Instruction tuning and assistant-only loss

The mechanism

Supervised fine-tuning supplies desired outputs for prompts. The model can receive both prompt and response tokens during teacher forcing while the objective trains only selected response targets. Seeing a prompt as context does not require predicting every prompt token in the loss. Role delimiters, assistant starts and end-of-turn markers require an explicit template convention; an off-by-one mask can train the wrong role or omit the terminating token. The executable lifecycle example uses one authored answer marker and supervises the answer plus EOS. Multi-turn templates are a worked extension, not an implemented chat framework.

Render the exact template and retain role ownership for every token.
Shift the target sequence before placing the loss mask.
Include each desired assistant target and the declared end marker; exclude prompt-only targets.
Divide by supervised targets and compare gradients under an intentionally wrong mask.

Science and assumptions

Instruction tuning and assistant-only loss · worked relation

L_{SFT}=-\frac{\sum_t m_t\log p_\theta(y_t\mid x,y_{<t})}{\sum_t m_t}

Identify exactly which shifted response targets contribute to supervised fine-tuning.

Symbols and units

L_SFT: Masked supervised target loss
nats per included target
m_t: Assistant-target inclusion flag after shifting
dimensionless
x: Prompt context
tokens
y_t: Supervised response target
token identifier
p_theta: Conditional target probability
dimensionless

For a six-target dialogue, prompt-target losses are [2,3,1,2] nats and response-target losses are [0.4,0.6]. All-token mean is 9/6=1.5 nats. Assistant-only mean is 1/2=0.5 nats. The prompt tokens still influence the response through attention, so their embeddings can receive gradients even though their own target losses are masked. A target mask is not a promise that all prompt-path parameters remain frozen.

Assumptions Positive included-target count and a stated role/template boundary. Core mask covers one answer and EOS; multi-turn masking here is worked-only.

Engineering example

Distinctions and tradeoffs

It is a mistake to conclude: No, target masking freezes every parameter used by the prompt.

Response-only supervision focuses the direct objective while still relying on prompt representations. Template errors can silently change what is learned.

Separate training signals and algorithms, then inspect actual policy changes and proxy failures. Identify exactly which shifted response targets contribute to supervised fine-tuning.

Specifications and interfaces

Learning mode: executable one-turn response mask
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
Worked contract: Identify exactly which shifted response targets contribute to supervised fine-tuning.
Positive included-target count and a stated role/template boundary. Core mask covers one answer and EOS; multi-turn masking here is worked-only.

Interfaces

Inputs and state Input: Render the exact template and retain role ownership for every token.
Result and check Output: Identify exactly which shifted response targets contribute to supervised fine-tuning.

Check your understanding

Can prompt embeddings receive gradients when prompt target losses are zeroed?

Yes, because response predictions depend on prompt representations.
No, target masking freezes every parameter used by the prompt.
Only if the prompt is incorrectly used as its own target.

Answer and reasoning

Yes, because response predictions depend on prompt representations.

Yes, because response predictions depend on prompt representations. Positive included-target count and a stated role/template boundary. Core mask covers one answer and EOS; multi-turn masking here is worked-only.

Evidence and scope

Mode: executable one-turn response mask. Positive included-target count and a stated role/template boundary. Core mask covers one answer and EOS; multi-turn masking here is worked-only. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.

Model lifecycle studio: executable one-turn response mask. Positive included-target count and a stated role/template boundary. Core mask covers one answer and EOS; multi-turn masking here is worked-only.