{"id":"loss-optimizer","level":"model","name":"Loss, gradients & optimizer","shortName":"Loss, gradients & optimizer","role":"Turn prediction errors into parameter updates.","description":"A loss assigns a numerical training objective to the model’s predictions. Backpropagation applies the chain rule through the computation graph to obtain gradients. An optimizer uses those gradients and, often, additional state to update weights. Adam maintains estimates related to gradient moments. This backward computation and optimizer storage distinguish training from ordinary inference. Distributed execution must coordinate the appropriate gradients or shards before consistent updates can proceed.\n\nA training objective, a decision threshold and an operational acceptance criterion answer different questions. Updating weights changes the predictive function; moving a threshold changes decisions for the same probabilities. Neither operation should be concealed inside an evaluation comparison. A frozen artifact and separate test set make these changes traceable into later deployment decisions.\n\nAn optimizer consumes a derived gradient; a supplied-gradient calculator teaches only the last part of that chain. The lifecycle experience adds an actual decoder loss and reverse pass before applying either SGD or Adam. Adam’s moments and step counter are learning state, not display history. Global norm clipping occurs before those moments. Changing the objective changes the gradient even if the optimizer setting is held fixed.","mechanism":["Run the forward computation and evaluate the loss.","Backpropagate derivatives through the recorded operations.","Combine required distributed information and update parameters.","The optimizer updates model parameters using the defined training loss and regularization.","Evaluation keeps thresholds and accepted error tradeoffs explicit instead of claiming that lower loss necessarily produces a better application.","Compute the gradient from the current batch loss and retain its denominator.","Apply the declared global clipping rule.","Update moments, bias-correct them and apply the chosen learning rate; save all resulting state."],"specs":[{"label":"Gradient","value":"∂loss / ∂weight","context":"Local sensitivity of the objective."},{"label":"Adam state","value":"First and second moment estimates","context":"Adds storage beyond the weights."}],"interfaces":[{"name":"Batch and predictions","direction":"in","signal":"Objective and activations"},{"name":"Model parameters","direction":"out","signal":"Updated weight arrays"}],"tradeoff":"Optimizer state and saved activations consume substantial memory. Checkpointing can exchange extra computation for lower activation storage.","whyAI":"Learning is an optimization process executed by physical arithmetic and memory systems.","related":["training-data","residual-normalization","parallelism","checkpoint","model-artifact-rollout-drift","evaluation","causal-pretraining-objective","optimizer-schedule-stability","microbatch-equivalent-updates"],"sources":[{"title":"Kingma & Ba — Adam","url":"https://arxiv.org/abs/1412.6980"},{"title":"PyTorch — CrossEntropyLoss","url":"https://docs.pytorch.org/docs/2.14/generated/torch.nn.CrossEntropyLoss.html"},{"title":"scikit-learn · Common pitfalls and data leakage","url":"https://scikit-learn.org/stable/common_pitfalls.html#data-leakage"},{"title":"2 OLMo 2 Furious","url":"https://arxiv.org/html/2501.00656v1"},{"title":"PyTorch Adam algorithm","url":"https://docs.pytorch.org/docs/2.14/generated/torch.optim.Adam.html"}],"scale":"Logical structure · mapped onto physical hardware","kind":"loss-optimizer","misconception":"Backpropagation computes derivatives; the optimizer decides the update.","check":{"question":"Which process changes learned weights in a normal training step?","options":["Displaying a response","Applying an optimizer update","Reading a KV cache"],"answer":1,"explanation":"The optimizer updates parameters using gradients and its update rule."},"branch":"Learning","subbranch":"Differentiation and weight updates","science":[{"title":"Adam adapts updates using gradient moments","equation":"gₜ = ∇θℒₜ; mₜ = β₁mₜ₋₁+(1−β₁)gₜ; vₜ = β₂vₜ₋₁+(1−β₂)gₜ²; θₜ₊₁ = θₜ−η·m̂ₜ/(√v̂ₜ+ε)","explanation":"The first moment m smooths gradient direction; the second moment v tracks squared gradient magnitude. Bias-corrected estimates m̂ and v̂ scale each coordinate’s step. The optimizer consequently needs state arrays in addition to weights and gradients, connecting a mathematical update rule directly to accelerator memory requirements.","assumptions":"Products, squares, and divisions are coordinate-wise. With zero initialization, m̂ₜ=mₜ/(1−β₁ᵗ) and v̂ₜ=vₜ/(1−β₂ᵗ). This is ordinary Adam; decoupled weight decay, clipping, loss scaling, and distributed sharding require additional rules."},{"title":"Compute gradients before applying the optimizer · worked relation","equation":"m_t=\\beta_1m_{t-1}+(1-\\beta_1)g_t,\\quad v_t=\\beta_2v_{t-1}+(1-\\beta_2)g_t^2,\\quad\\theta^+=\\theta-\\eta\\frac{m_t/(1-\\beta_1^t)}{\\sqrt{v_t/(1-\\beta_2^t)}+\\epsilon}","explanation":"For a scalar gradient g=0.5, weight 2 and learning rate 0.1, SGD gives 1.95. For the first Adam step with zero moments, beta1=0.9 and beta2=0.999, m=0.05 and v=0.00025; bias-corrected values are 0.5 and 0.25. With epsilon=1e-8 the Adam result is approximately 1.900000002. It differs from SGD because Adam normalizes by the second-moment scale.","assumptions":"No weight decay or AMSGrad; elementwise squares/division. First-step worked values use an unclipped scalar gradient of 0.5."}],"history":[{"year":2014,"title":"Adam combines momentum with adaptive scaling","significance":"Kingma and Ba released the Adam preprint in December 2014; its conference publication followed at ICLR 2015. The method maintains first and second gradient-moment estimates.","source":{"title":"Kingma & Ba — Adam: A Method for Stochastic Optimization","url":"https://arxiv.org/abs/1412.6980"}}],"prerequisites":["training-data","output-head"],"learningObjective":"Trace one Adam update from a gradient through moment state to a changed parameter.","engineeringExample":"Synthetic first Adam step: θ = 1, gradient g = 2, zero initial moments, β₁ = 0.9, β₂ = 0.999 and learning rate 0.01 give m₁ = 0.2 and v₁ = 0.004. Bias correction gives m̂₁ = 2 and v̂₁ = 4; with ε = 10⁻⁸, θ becomes approximately 0.99.\n\nFor fixed predicted probabilities [0.2,0.6,0.8], a threshold of 0.5 gives decisions [0,1,1], while threshold 0.7 gives [0,0,1]. The weights and all three probabilities are unchanged. With labels [0,0,1], the second threshold removes one false positive on these three cases. This does not prove that 0.7 is better on future data; selecting it using the held-out test would also compromise that test’s independence.\n\nFor a scalar gradient g=0.5, weight 2 and learning rate 0.1, SGD gives 1.95. For the first Adam step with zero moments, beta1=0.9 and beta2=0.999, m=0.05 and v=0.00025; bias-corrected values are 0.5 and 0.25. With epsilon=1e-8 the Adam result is approximately 1.900000002. It differs from SGD because Adam normalizes by the second-moment scale.","evidenceNotes":"This follows the ordinary Adam equations in the linked paper. The gradient is supplied rather than derived here; clipping, weight decay, loss scaling and sharded optimizer state are excluded. Computing a gradient and applying an update remain separate steps.\n\nThe threshold arithmetic is authored. The existing sensor experience executes actual logistic updates, while these three probabilities are a separate worked decision example.\n\nMode: executable derived-gradient optimizer. No weight decay or AMSGrad; elementwise squares/division. First-step worked values use an unclipped scalar gradient of 0.5. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.","checks":[{"question":"A decision threshold changes from 0.5 to 0.7 with fixed model outputs. What necessarily changes?","options":["The fitted weights","The training labels","Potential binary decisions, without requiring retraining"],"answer":2,"explanation":"Thresholding is a downstream decision rule; it can change labels without changing the fitted predictor."},{"question":"Why is the first Adam change about 0.1 rather than 0.05?","options":["Adam ignores gradients and always chooses random weights.","SGD and Adam use identical state equations.","The bias-corrected gradient is divided by its second-moment root."],"answer":2,"explanation":"The bias-corrected gradient is divided by its second-moment root. No weight decay or AMSGrad; elementwise squares/division. First-step worked values use an unclipped scalar gradient of 0.5."}],"labId":"library","scope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.","investigationId":"execution","labScope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions."}