The mechanism
Names such as pretraining, mid-training and post-training summarize recipes but do not uniquely specify a mathematical operation. A useful stage contract identifies the starting checkpoint, example distribution, objective, parameter subset, optimizer state policy and acceptance measurements. Continued language modeling can change only the data mixture while preserving the same next-token objective. SFT can preserve the decoder while changing target masks. DPO changes the training signal to relative chosen/rejected likelihoods against a reference. Inference changes activations and selected outputs without necessarily changing learned weights.
- Record a checkpoint identity and tokenizer/config identity before branching.
- State which data and targets enter the loss, and which parameters may change.
- Declare retained or reset optimizer/RNG/schedule state.
- Compare acquisition, retention and failures on development cases before exposing a final test.
Science and assumptions
Describe a stage by its changed contract · worked relation
Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation.
Symbols and units
- C
- Stage contract tuple
record
- theta_0
- Starting checkpoint
parameters
- D
- Training data distribution
distribution
- L
- Objective and denominator
function
- T
- Trainable parameter subset
set
- O
- Optimizer RNG and schedule policy
state contract
- E
- Evaluation and retention gates
protocol
A base checkpoint contains 423 trainable weights. A continuation branch retains all 423, its Adam moments and RNG but changes the copy/swap sampling distribution. An SFT branch also retains 423 trainable weights but uses only two response targets per sequence. A rank-one adapter on a separate 4×6 matrix would train 10 factors instead of 24 base entries, but its objective could still be SFT, DPO or another differentiable loss.
Assumptions Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model.
Engineering example
A base checkpoint contains 423 trainable weights. A continuation branch retains all 423, its Adam moments and RNG but changes the copy/swap sampling distribution. An SFT branch also retains 423 trainable weights but uses only two response targets per sequence. A rank-one adapter on a separate 4×6 matrix would train 10 factors instead of 24 base entries, but its objective could still be SFT, DPO or another differentiable loss.
Distinctions and tradeoffs
It is a mistake to conclude: Giving the unchanged loss a new stage name.
Stage names aid communication, while explicit contracts take more effort and reveal incompatibilities that a name can conceal.
Adapt an existing checkpoint while measuring retained and acquired behavior. Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation.
Specifications and interfaces
- Learning mode
- executable stage branches
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
- Worked contract
- Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation.
Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model.
Interfaces
- Inputs and state Input
- Record a checkpoint identity and tokenizer/config identity before branching.
- Result and check Output
- Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation.
Check your understanding
Which change alone makes a continuation stage a different objective?
- Saving the same weights under a different filename.
- Replacing token NLL with a specified preference-pair loss.
- Giving the unchanged loss a new stage name.
Answer and reasoning
Replacing token NLL with a specified preference-pair loss.
Replacing token NLL with a specified preference-pair loss. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model.
Evidence and scope
Mode: executable stage branches. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.
Model lifecycle studio: executable stage branches. Contract notation is an authored organizational tool, not a universal naming standard. The tuple does not itself train a model.