# Describe a stage by its changed contract Canonical: https://semiaifoundry.com/learn/atlas/model-lifecycle-stage-contract/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html model-lifecycle-continuation · Atlas v1.3.0 # Describe a stage by its changed contract Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#model-lifecycle-stage-contract)Mathematical, data and behavior space; logical dimensions are not physical distance Learning objectiveClassify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. ## The mechanism Names such as pretraining, mid-training and post-training summarize recipes but do not uniquely specify a mathematical operation. A useful stage contract identifies the starting checkpoint, example distribution, objective, parameter subset, optimizer state policy and acceptance measurements. Continued language modeling can change only the data mixture while preserving the same next-token objective. SFT can preserve the decoder while changing target masks. DPO changes the training signal to relative chosen/rejected likelihoods against a reference. Inference changes activations and selected outputs without necessarily changing learned weights. 1. Record a checkpoint identity and tokenizer/config identity before branching. 2. State which data and targets enter the loss, and which parameters may change. 3. Declare retained or reset optimizer/RNG/schedule state. 4. Compare acquisition, retention and failures on development cases before exposing a final test. ## Science and assumptions ### Describe a stage by its changed contract · worked relation $$ \mathcal C=(\theta_0,\mathcal D,\mathcal L,\mathcal T,\mathcal O,\mathcal E) $$ Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. Symbols and units C Stage contract tuple record theta\_0 Starting checkpoint parameters D Training data distribution distribution L Objective and denominator function T Trainable parameter subset set O Optimizer RNG and schedule policy state contract E Evaluation and retention gates protocol A base checkpoint contains 423 trainable weights. A continuation branch retains all 423, its Adam moments and RNG but changes the copy/swap sampling distribution. An SFT branch also retains 423 trainable weights but uses only two response targets per sequence. A rank-one adapter on a separate 4×6 matrix would train 10 factors instead of 24 base entries, but its objective could still be SFT, DPO or another differentiable loss. Assumptions Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. ## Engineering example A base checkpoint contains 423 trainable weights. A continuation branch retains all 423, its Adam moments and RNG but changes the copy/swap sampling distribution. An SFT branch also retains 423 trainable weights but uses only two response targets per sequence. A rank-one adapter on a separate 4×6 matrix would train 10 factors instead of 24 base entries, but its objective could still be SFT, DPO or another differentiable loss. ## Distinctions and tradeoffs It is a mistake to conclude: Giving the unchanged loss a new stage name. Stage names aid communication, while explicit contracts take more effort and reveal incompatibilities that a name can conceal. Adapt an existing checkpoint while measuring retained and acquired behavior. Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. ## Specifications and interfaces Learning mode executable stage branches The executable decoder has 423 parameters; worked extensions do not imply an additional simulator. Worked contract Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. ### Interfaces Inputs and state Input Record a checkpoint identity and tokenizer/config identity before branching. Result and check Output Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation. ## Check your understanding ### Which change alone makes a continuation stage a different objective? 1. Saving the same weights under a different filename. 2. Replacing token NLL with a specified preference-pair loss. 3. Giving the unchanged loss a new stage name. Answer and reasoning Replacing token NLL with a specified preference-pair loss. Replacing token NLL with a specified preference-pair loss. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. ## Evidence and scope Mode: executable stage branches. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied. Model lifecycle studio: executable stage branches. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. ## Sources - [2 OLMo 2 Furious](https://arxiv.org/html/2501.00656v1) - [Don’t Stop Pretraining](https://aclanthology.org/2020.acl-main.740.pdf) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/model-lifecycle-stage-contract-7ddd81044bf2edbc.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [Training data pipeline](https://semiaifoundry.com/learn/atlas/training-data/) - [Loss, gradients & optimizer](https://semiaifoundry.com/learn/atlas/loss-optimizer/) - [Post-training & feedback](https://semiaifoundry.com/learn/atlas/post-training/) ### Related lessons - [Continue language modeling on a new domain](https://semiaifoundry.com/learn/atlas/domain-adaptive-continuation/) - [Retention, replay and transfer](https://semiaifoundry.com/learn/atlas/replay-retention-forgetting/) - [Context extension is a learned-use test](https://semiaifoundry.com/learn/atlas/long-context-adaptation/) - [Compatible weight averaging and regressions](https://semiaifoundry.com/learn/atlas/checkpoint-averaging-boundaries/)