{"id":"replay-retention-forgetting","name":"Retention, replay and transfer","shortName":"Retention, replay and transfer","role":"Reason about a fixed-budget retention tradeoff without treating replay as a guarantee.","description":"Replay mixes old training examples into adaptation. It changes the expected objective and consumes part of the same update budget. Comparing a longer replay run with a shorter new-only run confounds mixture with computation. Retention must be evaluated on reserved old-domain examples, not merely on repeated training items. In the lifecycle task the same prompt has opposite desired answers in copy and swap. No deterministic single-answer policy without a domain identifier can satisfy both simultaneously. That impossibility is a property of this constructed task, not a universal claim about continual learning.","mechanism":["Hold the starting checkpoint, total draws and optimizer policy fixed.","Select a declared old-domain replay probability.","Record realized domain draws rather than assuming the expected fraction occurred exactly.","Compare both reserved-domain losses, and explain whether conflict or insufficient acquisition limits the result."],"learningObjective":"Reason about a fixed-budget retention tradeoff without treating replay as a guarantee.","engineeringExample":"For one conflicting binary prompt let p be the old-answer probability and q the old-example mixture fraction. Expected NLL is −q ln p−(1−q)ln(1−p), minimized at p=q. With q=0.25, old/new losses at that optimum are ln4=1.386294 and −ln0.75=0.287682 nats. Greedy output selects the new answer, so old accuracy is zero even though replay has raised old-answer probability above zero.","evidenceNotes":"Mode: executable replay; worked ideal optimum. One identical prompt with contradictory binary labels, unlimited fitting of its probability and no domain tag. Finite decoder optimization need not reach this ideal optimum. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.","specs":[{"label":"Learning mode","value":"executable replay; worked ideal optimum","context":"The executable decoder has 423 parameters; worked extensions do not imply an additional simulator."},{"label":"Worked contract","value":"Reason about a fixed-budget retention tradeoff without treating replay as a guarantee.","context":"One identical prompt with contradictory binary labels, unlimited fitting of its probability and no domain tag. Finite decoder optimization need not reach this ideal optimum."}],"interfaces":[{"name":"Inputs and state","direction":"in","signal":"Hold the starting checkpoint, total draws and optimizer policy fixed."},{"name":"Result and check","direction":"out","signal":"Reason about a fixed-budget retention tradeoff without treating replay as a guarantee."}],"tradeoff":"Replay allocates scarce updates to retention; it can reduce new-task progress and cannot solve an information-theoretic label conflict by itself.","whyAI":"Adapt an existing checkpoint while measuring retained and acquired behavior. Reason about a fixed-budget retention tradeoff without treating replay as a guarantee.","misconception":"It is a mistake to conclude: Likelihood and accuracy are the same statistic.","check":{"question":"Why can replay improve old-answer likelihood without restoring old-answer accuracy?","options":["Probability can rise while remaining below the greedy decision threshold.","Likelihood and accuracy are the same statistic.","Replay guarantees both incompatible labels are always correct."],"answer":0,"explanation":"Probability can rise while remaining below the greedy decision threshold. One identical prompt with contradictory binary labels, unlimited fitting of its probability and no domain tag. Finite decoder optimization need not reach this ideal optimum."},"prerequisites":["domain-adaptive-continuation"],"related":["domain-adaptive-continuation","model-lifecycle-stage-contract","long-context-adaptation","checkpoint-averaging-boundaries"],"sources":[{"title":"Don’t Stop Pretraining","url":"https://aclanthology.org/2020.acl-main.740.pdf"},{"title":"2 OLMo 2 Furious","url":"https://arxiv.org/html/2501.00656v1"}],"level":"model","scale":"Mathematical, data and behavior space; logical dimensions are not physical distance","kind":"model","branch":"model-lifecycle-continuation","subbranch":"Retention, replay and transfer","workstream":"W11","visualFamily":"mathematical","labId":"library","labScope":"Model lifecycle studio: executable replay; worked ideal optimum. One identical prompt with contradictory binary labels, unlimited fitting of its probability and no domain tag. Finite decoder optimization need not reach this ideal optimum.","science":[{"title":"Retention, replay and transfer · worked relation","equation":"L(p)=-q\\log p-(1-q)\\log(1-p),\\qquad p^*=q\\quad(0<q<1)","explanation":"For one conflicting binary prompt let p be the old-answer probability and q the old-example mixture fraction. Expected NLL is −q ln p−(1−q)ln(1−p), minimized at p=q. With q=0.25, old/new losses at that optimum are ln4=1.386294 and −ln0.75=0.287682 nats. Greedy output selects the new answer, so old accuracy is zero even though replay has raised old-answer probability above zero.","assumptions":"One identical prompt with contradictory binary labels, unlimited fitting of its probability and no domain tag. Finite decoder optimization need not reach this ideal optimum."}],"history":[]}