The mechanism
Retrieval supplies candidate evidence to a generator. A retrieved document can be relevant without supporting the generated claim, and a supported statement can still fail to answer the question. A model may ignore supplied evidence or use a stale source. The RAG paper formalizes one family of probabilistic retrieval/generation models; not every application with search implements that marginalization. The atlas’s existing retrieval specimen is a deterministic scored-source baseline. This lesson adds worked model probabilities and support accounting, without presenting that baseline as a trained generator.
- Preserve query, source version, retrieved ranking and the exact evidence spans.
- State whether the generator conditions on one concatenated context or a mixture over documents.
- Score each answer claim for support and question relevance independently.
- Compare source removal, contradictory-source and irrelevant-but-true controls.
Science and assumptions
From lexical retrieval to model answers with evidence · worked relation
Track evidence retrieval, answer likelihood and claim support as separate stages.
Symbols and units
- x
- Question or input
text
- y
- Complete candidate answer
text
- z
- Retrieved document in the declared finite mixture
document
- p
- Conditional probabilities under their stated models
dimensionless
- S
- Claim support fraction under the scoring rubric
dimensionless
- N
- Claims at the stated scoring boundary
claims
In a two-document mixture, retrieval weights are [0.7,0.3] and the probability of one complete candidate answer under the two documents is [0.8,0.2]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.
Assumptions Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.
Engineering example
In a two-document mixture, retrieval weights are [0.7,0.3] and the probability of one complete candidate answer under the two documents is [0.8,0.2]. Its mixture probability is 0.7×0.8+0.3×0.2=0.62. Separately, an answer with two supported claims out of three has support fraction 2/3; that is not the probability 0.62. A statement that a server uses 400 W may be supported but irrelevant to a question asking how many servers are available.
Distinctions and tradeoffs
It is a mistake to conclude: The fraction of its claims supported by sources.
More retrieved material can improve evidence coverage while adding irrelevant or contradictory context. Generating citations is not equivalent to demonstrating support.
Measure quality, uncertainty and failure under a stated system/task contract. Track evidence retrieval, answer likelihood and claim support as separate stages.
Specifications and interfaces
- Learning mode
- worked example
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
- Worked contract
- Track evidence retrieval, answer likelihood and claim support as separate stages.
Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.
Interfaces
- Inputs and state Input
- Preserve query, source version, retrieved ranking and the exact evidence spans.
- Result and check Output
- Track evidence retrieval, answer likelihood and claim support as separate stages.
Check your understanding
What does the value 0.62 represent?
- The probability every source is current and accurate.
- The candidate answer probability under this declared document mixture.
- The fraction of its claims supported by sources.
Answer and reasoning
The candidate answer probability under this declared document mixture.
The candidate answer probability under this declared document mixture. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.
Evidence and scope
Mode: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.
Model lifecycle studio: worked example. Exact normalized two-document teaching mixture; RAG top-k truncation can require a different approximation boundary. Support is rubric evidence, not model likelihood.