The mechanism
Synthetic supervision has a lineage: a generator produces a candidate, a verifier or judge assigns a signal, and a selection rule decides whether to train on it. Each stage can introduce a different bias. Repeating a generator’s answer does not create independent evidence, and selecting only accepted outputs can hide the verifier’s blind spots. Generation seeds, prompts, model identities and verifier versions therefore belong to the dataset record. Final evaluation needs independently specified cases and scoring rules, rather than the same filter that selected training material.
- Retain rejected candidates as an audit sample alongside accepted examples.
- Evaluate an independent reference on a bounded problem where the answer is actually computable.
- Separate selection rate, precision among accepted examples and coverage of problem types.
- Compare the trained model on a reserved task family; selection statistics are not model-quality measurements.
Science and assumptions
Synthetic supervision and selection loops · worked relation
Track a generated example through verification and selection without converting acceptance into ground truth.
Symbols and units
- N
- Counts under the stated candidate or unique-pair boundary
examples
- precision_selected
- Independent correctness among accepted candidates
dimensionless
Four authored candidates answer 2+2: 4, 5, 4 and 6. A faulty verifier accepts even numbers, so it accepts three candidates, including the wrong 6. Selection rate is 3/4; correctness among accepted candidates is 2/3. Deduplicating identical problem/answer pairs leaves {4,6}, whose correctness is 1/2. This denominator change reveals why duplicate correct answers can make a selected corpus look more reliable.
Assumptions Explicit arithmetic reference and deterministic faulty filter; positive denominators. This is a worked selection audit, not a synthetic-data training simulator.
Engineering example
Four authored candidates answer 2+2: 4, 5, 4 and 6. A faulty verifier accepts even numbers, so it accepts three candidates, including the wrong 6. Selection rate is 3/4; correctness among accepted candidates is 2/3. Deduplicating identical problem/answer pairs leaves {4,6}, whose correctness is 1/2. This denominator change reveals why duplicate correct answers can make a selected corpus look more reliable.
Distinctions and tradeoffs
It is a mistake to conclude: Deterministic verification guarantees mathematical truth.
A restrictive verifier can improve accepted precision while reducing diversity. A permissive or biased verifier can amplify generator errors through repeated training.
Make the effective training distribution inspectable rather than treating tokens as a homogeneous commodity. Track a generated example through verification and selection without converting acceptance into ground truth.
Specifications and interfaces
- Learning mode
- worked example
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
- Worked contract
- Track a generated example through verification and selection without converting acceptance into ground truth.
Explicit arithmetic reference and deterministic faulty filter; positive denominators. This is a worked selection audit, not a synthetic-data training simulator.
Interfaces
- Inputs and state Input
- Retain rejected candidates as an audit sample alongside accepted examples.
- Result and check Output
- Track a generated example through verification and selection without converting acceptance into ground truth.
Check your understanding
What does the accepted answer 6 demonstrate?
- The filter accepts some incorrect answers, even though its rule is deterministic.
- Deterministic verification guarantees mathematical truth.
- A high selection rate proves generalization.
Answer and reasoning
The filter accepts some incorrect answers, even though its rule is deterministic.
The filter accepts some incorrect answers, even though its rule is deterministic. Explicit arithmetic reference and deterministic faulty filter; positive denominators. This is a worked selection audit, not a synthetic-data training simulator.
Evidence and scope
Mode: worked example. Explicit arithmetic reference and deterministic faulty filter; positive denominators. This is a worked selection audit, not a synthetic-data training simulator. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.
Model lifecycle studio: worked example. Explicit arithmetic reference and deterministic faulty filter; positive denominators. This is a worked selection audit, not a synthetic-data training simulator.