{"id":"output-head","level":"model","name":"Output head & decoding","shortName":"Output head & decoding","role":"Convert the final representation into the next token.","description":"The output projection assigns a logit to each candidate vocabulary token. A softmax can turn these scores into a probability distribution, after which a decoding rule selects an output. Greedy decoding takes a maximum; sampling introduces controlled randomness. The chosen ID is appended to the sequence and the process continues. A fluent output is therefore produced by many repeated computations, with behavior influenced by training, context, decoding, and any external tools.\n\nThe output head supplies logits for a probability distribution. Negative log likelihood compares that distribution with the observed target and differentiates to probability minus target indicator. Greedy decoding and categorical sampling consume the same probabilities differently. Neither updates the model’s weights by itself. The lifecycle generator retains a separate inference RNG so that requesting a sampled answer cannot change later training draws.","mechanism":["Project the final hidden representation to vocabulary logits.","Apply the decoding policy to select a token ID.","Append that token and compute the next step.","Apply stable log-softmax to all vocabulary logits.","Gather the target probability and compute the logit derivative.","Select greedy or seeded categorical output, keeping generation state separate from optimizer state."],"specs":[{"label":"Logits","value":"V scores","context":"One score per vocabulary entry."},{"label":"Autoregression","value":"One dependent step after another","context":"Batching and speculative methods alter execution scheduling."}],"interfaces":[{"name":"Final block","direction":"in","signal":"Hidden representation"},{"name":"User / next step","direction":"out","signal":"Selected token ID and decoded text"}],"tradeoff":"Sampling changes diversity and reproducibility. Output probability is not a calibrated guarantee that a statement is true.","whyAI":"Repeated decoding turns numerical transformations into an interactive response.","related":["tokenizer","kv-cache","serving","reverse-mode-autodiff","tiny-causal-decoder"],"sources":[{"title":"Brown et al. — Language Models are Few-Shot Learners","url":"https://arxiv.org/abs/2005.14165"},{"title":"Attention Is All You Need","url":"https://arxiv.org/html/1706.03762v7"}],"scale":"Logical structure · mapped onto physical hardware","kind":"output-head","misconception":"The output head chooses vocabulary symbols; factual verification requires additional evidence.","check":{"question":"Does a high next-token probability establish factual truth?","options":["Yes","No","Only with more GPUs"],"answer":1,"explanation":"The distribution reflects the trained prediction task and context."},"branch":"Generation","subbranch":"Token probabilities","science":[{"title":"A sequence probability factors into conditional predictions","equation":"p(x₁,…,xT) = ∏ₜ p(xₜ | x<t); p(xₜ=i | x<t) = exp(z_i)/Σⱼexp(z_j)","explanation":"The chain rule factors a joint sequence distribution into conditional steps. Vocabulary logits z become normalized next-token probabilities. Greedy or sampling-based selection then turns the distribution into a particular sequence. A likely continuation can still assert something false, because probability under a language model is not a truth certificate.","assumptions":"This is a standard autoregressive categorical decoder. It does not specify any model’s exposed sampling controls or internal scheduling; speculative decoding can verify several proposed tokens together while preserving a target distribution."},{"title":"Likelihood, cross-entropy and sampled output · worked relation","equation":"p_i=\\frac{e^{z_i}}{\\sum_j e^{z_j}},\\qquad L=-\\log p_y,\\qquad \\frac{\\partial L}{\\partial z_i}=p_i-\\mathbf1[i=y]","explanation":"For logits [0,ln2,ln3], probabilities are [1/6,1/3,1/2]. With target class three, loss is ln2=0.693147 and gradient is [1/6,1/3,−1/2]. Greedy decoding chooses class three, while one categorical sample can choose another class. A sample different from the greedy class is not evidence that the probabilities or gradient were incorrectly computed.","assumptions":"Finite logits, natural logarithms and one categorical target. Stable implementation subtracts maximum logit; that shift preserves the distribution."}],"history":[{"year":2020,"title":"GPT-3 demonstrates broad in-context task conditioning","significance":"Brown and colleagues evaluated GPT-3 on many tasks using textual instructions and examples without task-specific gradient updates, illustrating the difference between conditioning generation and retraining weights.","source":{"title":"Brown et al. — Language Models are Few-Shot Learners","url":"https://arxiv.org/abs/2005.14165"}}],"prerequisites":["attention","feed-forward","residual-normalization"],"learningObjective":"Convert a small logit vector to next-token probabilities and distinguish selection probability from truth.","engineeringExample":"Synthetic logits [0, ln 2, ln 3] exponentiate to [1, 2, 3], so softmax gives [1/6, 1/3, 1/2]. Greedy decoding chooses the third candidate; sampling can choose any of the three. A two-token sequence with conditional probabilities 1/2 then 1/4 has probability 1/8.\n\nFor logits [0,ln2,ln3], probabilities are [1/6,1/3,1/2]. With target class three, loss is ln2=0.693147 and gradient is [1/6,1/3,−1/2]. Greedy decoding chooses class three, while one categorical sample can choose another class. A sample different from the greedy class is not evidence that the probabilities or gradient were incorrectly computed.","evidenceNotes":"This is the standard categorical autoregressive calculation already stated in the lesson. Toy logits are not model observations, and generation probability is not factual confidence or a claim about a private system’s decoding controls.\n\nMode: executable output probabilities and generation. Finite logits, natural logarithms and one categorical target. Stable implementation subtracts maximum logit; that shift preserves the distribution. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.","checks":[{"question":"Why can a categorical sample differ from the greedy token?","options":["Sampling follows the whole distribution, including lower-probability outcomes.","Sampling performs a hidden optimizer update.","The largest logit is never used in sampling."],"answer":0,"explanation":"Sampling follows the whole distribution, including lower-probability outcomes. Finite logits, natural logarithms and one categorical target. Stable implementation subtracts maximum logit; that shift preserves the distribution."}],"labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}