{
  "schemaVersion": 1,
  "id": "atlas-lesson-retrieval-supported-generation",
  "title": "From lexical retrieval to model answers with evidence",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Track evidence retrieval, answer likelihood and claim support as separate stages.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/retrieval-supported-generation/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "model-lifecycle-evaluation",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Mode: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/retrieval-supported-generation-6c04d6d40bef835c.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#retrieval-supported-generation",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "ebf700e084ba3a3b334f9e6b3f1c81407b6cc3fe938f390dadcbeb3bc5895b5e",
  "contentLength": 6677,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-retrieval-supported-generation.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-retrieval-supported-generation.md",
  "sections": [
    {
      "id": "section-1",
      "title": "From lexical retrieval to model answers with evidence",
      "start": 43,
      "end": 455
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 455,
      "end": 1433
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1433,
      "end": 1461
    },
    {
      "id": "section-4",
      "title": "From lexical retrieval to model answers with evidence · worked relation",
      "start": 1461,
      "end": 2697
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2697,
      "end": 3168
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 3168,
      "end": 3609
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3609,
      "end": 4042
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4042,
      "end": 4271
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4271,
      "end": 4300
    },
    {
      "id": "section-10",
      "title": "What does the value 0.62 represent?",
      "start": 4300,
      "end": 4861
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4861,
      "end": 5486
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5486,
      "end": 5898
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5898,
      "end": 5921
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5921,
      "end": 6180
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 6180,
      "end": 6677
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#retrieval-supported-generation"
    },
    {
      "title": "Holistic Evaluation of Language Models",
      "url": "https://arxiv.org/html/2211.09110v1"
    },
    {
      "title": "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks",
      "url": "https://arxiv.org/html/2005.11401v4"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/retrieval-supported-generation-6c04d6d40bef835c.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Tools, memory & agent loop",
      "url": "https://semiaifoundry.com/learn/atlas/tool-loop/"
    },
    {
      "title": "Evaluation & the AGI frontier",
      "url": "https://semiaifoundry.com/learn/atlas/evaluation/"
    },
    {
      "title": "Self-attention",
      "url": "https://semiaifoundry.com/learn/atlas/attention/"
    },
    {
      "title": "Confidence, errors and refusal to guess",
      "url": "https://semiaifoundry.com/learn/atlas/probability-calibration-abstention/"
    },
    {
      "title": "Human and model judgments under a rubric",
      "url": "https://semiaifoundry.com/learn/atlas/judge-rubric-agreement/"
    },
    {
      "title": "Sampling more candidates versus selecting correct ones",
      "url": "https://semiaifoundry.com/learn/atlas/test-time-search-verification/"
    },
    {
      "title": "Version the entire evaluated system",
      "url": "https://semiaifoundry.com/learn/atlas/model-system-regression-contract/"
    }
  ],
  "recordHash": "58960398bc0547515764325023630914fdea26509564bebf14a57a957c6f6822",
  "contentFormat": "text/markdown",
  "content": "model-lifecycle-evaluation · Atlas v1.3.0\n\n# From lexical retrieval to model answers with evidence\n\nTrack evidence retrieval, answer likelihood and claim support as separate stages.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#retrieval-supported-generation)Mathematical, data and behavior space; logical dimensions are not physical distance\n\nLearning objectiveTrack evidence retrieval, answer likelihood and claim support as separate stages.\n\n## The mechanism\n\nRetrieval supplies candidate evidence to a generator. A retrieved document can be relevant without supporting the generated claim, and a supported statement can still fail to answer the question. A model may ignore supplied evidence or use a stale source. The RAG paper formalizes one family of probabilistic retrieval/generation models; not every application with search implements that marginalization. The atlas’s existing retrieval specimen is a deterministic scored-source baseline. This lesson adds worked model probabilities and support accounting, without presenting that baseline as a trained generator.\n\n1.  Preserve query, source version, retrieved ranking and the exact evidence spans.\n2.  State whether the generator conditions on one concatenated context or a mixture over documents.\n3.  Score each answer claim for support and question relevance independently.\n4.  Compare source removal, contradictory-source and irrelevant-but-true controls.\n\n## Science and assumptions\n\n### From lexical retrieval to model answers with evidence · worked relation\n\n$$\np(y\\mid x)=\\sum_z p(z\\mid x)p(y\\mid x,z),\\qquad S=\\frac{N_{\\rm supported\\ claims}}{N_{\\rm scored\\ claims}}\n$$\n\nTrack evidence retrieval, answer likelihood and claim support as separate stages.\n\nSymbols and units\n\nx\n\nQuestion or input\n\ntext\n\ny\n\nComplete candidate answer\n\ntext\n\nz\n\nRetrieved document in the declared finite mixture\n\ndocument\n\np\n\nConditional probabilities under their stated models\n\ndimensionless\n\nS\n\nClaim support fraction under the scoring rubric\n\ndimensionless\n\nN\n\nClaims at the stated scoring boundary\n\nclaims\n\nIn a two-document mixture, retrieval weights are \\[0.7,0.3\\] and the probability of one complete candidate answer under the two documents is \\[0.8,0.2\\]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.\n\nAssumptions Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.\n\n## Engineering example\n\nIn a two-document mixture, retrieval weights are \\[0.7,0.3\\] and the probability of one complete candidate answer under the two documents is \\[0.8,0.2\\]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.\n\n## Distinctions and tradeoffs\n\nIt is a mistake to conclude: The fraction of its claims supported by sources.\n\nMore retrieved material can improve evidence coverage while adding irrelevant or contradictory context. Generating citations is not equivalent to demonstrating support.\n\nMeasure quality, uncertainty and failure under a stated system/task contract. Track evidence retrieval, answer likelihood and claim support as separate stages.\n\n## Specifications and interfaces\n\nLearning mode\n\nworked example\n\nThe executable decoder has 423 parameters; worked extensions do not imply an additional simulator.\n\nWorked contract\n\nTrack evidence retrieval, answer likelihood and claim support as separate stages.\n\nExact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.\n\n### Interfaces\n\nInputs and state Input\n\nPreserve query, source version, retrieved ranking and the exact evidence spans.\n\nResult and check Output\n\nTrack evidence retrieval, answer likelihood and claim support as separate stages.\n\n## Check your understanding\n\n### What does the value 0.62 represent?\n\n1.  The probability every source is current and accurate.\n2.  The candidate answer probability under this declared document mixture.\n3.  The fraction of its claims supported by sources.\n\nAnswer and reasoning\n\nThe candidate answer probability under this declared document mixture.\n\nThe candidate answer probability under this declared document mixture. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.\n\n## Evidence and scope\n\nMode: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.\n\nModel lifecycle studio: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.\n\n## Sources\n\n-   [Holistic Evaluation of Language Models](https://arxiv.org/html/2211.09110v1)\n-   [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/html/2005.11401v4)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/retrieval-supported-generation-6c04d6d40bef835c.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)\n-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)\n-   [Self-attention](https://semiaifoundry.com/learn/atlas/attention/)\n\n### Related lessons\n\n-   [Confidence, errors and refusal to guess](https://semiaifoundry.com/learn/atlas/probability-calibration-abstention/)\n-   [Human and model judgments under a rubric](https://semiaifoundry.com/learn/atlas/judge-rubric-agreement/)\n-   [Sampling more candidates versus selecting correct ones](https://semiaifoundry.com/learn/atlas/test-time-search-verification/)\n-   [Version the entire evaluated system](https://semiaifoundry.com/learn/atlas/model-system-regression-contract/)"
}