{"id":"retrieval-supported-generation","name":"From lexical retrieval to model answers with evidence","shortName":"From lexical retrieval to model answers with evidence","role":"Track evidence retrieval, answer likelihood and claim support as separate stages.","description":"Retrieval supplies candidate evidence to a generator. A retrieved document can be relevant without supporting the generated claim, and a supported statement can still fail to answer the question. A model may ignore supplied evidence or use a stale source. The RAG paper formalizes one family of probabilistic retrieval/generation models; not every application with search implements that marginalization. The atlas’s existing retrieval specimen is a deterministic scored-source baseline. This lesson adds worked model probabilities and support accounting, without presenting that baseline as a trained generator.","mechanism":["Preserve query, source version, retrieved ranking and the exact evidence spans.","State whether the generator conditions on one concatenated context or a mixture over documents.","Score each answer claim for support and question relevance independently.","Compare source removal, contradictory-source and irrelevant-but-true controls."],"learningObjective":"Track evidence retrieval, answer likelihood and claim support as separate stages.","engineeringExample":"In a two-document mixture, retrieval weights are [0.7,0.3] and the probability of one complete candidate answer under the two documents is [0.8,0.2]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.","evidenceNotes":"Mode: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.","specs":[{"label":"Learning mode","value":"worked example","context":"The executable decoder has 423 parameters; worked extensions do not imply an additional simulator."},{"label":"Worked contract","value":"Track evidence retrieval, answer likelihood and claim support as separate stages.","context":"Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood."}],"interfaces":[{"name":"Inputs and state","direction":"in","signal":"Preserve query, source version, retrieved ranking and the exact evidence spans."},{"name":"Result and check","direction":"out","signal":"Track evidence retrieval, answer likelihood and claim support as separate stages."}],"tradeoff":"More retrieved material can improve evidence coverage while adding irrelevant or contradictory context. Generating citations is not equivalent to demonstrating support.","whyAI":"Measure quality, uncertainty and failure under a stated system/task contract. Track evidence retrieval, answer likelihood and claim support as separate stages.","misconception":"It is a mistake to conclude: The fraction of its claims supported by sources.","check":{"question":"What does the value 0.62 represent?","options":["The probability every source is current and accurate.","The candidate answer probability under this declared document mixture.","The fraction of its claims supported by sources."],"answer":1,"explanation":"The candidate answer probability under this declared document mixture. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood."},"prerequisites":["tool-loop","evaluation","attention"],"related":["tool-loop","evaluation","attention","probability-calibration-abstention","judge-rubric-agreement","test-time-search-verification","model-system-regression-contract"],"sources":[{"title":"Holistic Evaluation of Language Models","url":"https://arxiv.org/html/2211.09110v1"},{"title":"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks","url":"https://arxiv.org/html/2005.11401v4"}],"level":"model","scale":"Mathematical, data and behavior space; logical dimensions are not physical distance","kind":"model","branch":"model-lifecycle-evaluation","subbranch":"From lexical retrieval to model answers with evidence","workstream":"W11","visualFamily":"mathematical","labId":"library","labScope":"Model lifecycle studio: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.","science":[{"title":"From lexical retrieval to model answers with evidence · worked relation","equation":"p(y\\mid x)=\\sum_z p(z\\mid x)p(y\\mid x,z),\\qquad S=\\frac{N_{\\rm supported\\ claims}}{N_{\\rm scored\\ claims}}","explanation":"In a two-document mixture, retrieval weights are [0.7,0.3] and the probability of one complete candidate answer under the two documents is [0.8,0.2]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.","assumptions":"Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood."}],"history":[]}