semiAIfoundry

model-lifecycle-base-model · Atlas v1.3.0

A differentiable computation graph

Accumulate every branch contribution to a reused parameter and check it by finite differences.

Explore in AtlasMathematical, data and behavior space; logical dimensions are not physical distance

Learning objectiveAccumulate every branch contribution to a reused parameter and check it by finite differences.

The mechanism

A forward computation creates intermediate values. Reverse-mode differentiation traverses the dependency graph backward, multiplying each incoming derivative by the local derivative and adding contributions where paths meet. Reusing one parameter in several operations creates several paths to the loss; its final gradient is their sum. The optimizer acts only after these derivatives have been computed. This distinction lets a learner locate an incorrect derivative independently of an incorrect update rule. Numerical finite differences provide a separate local check, although their accuracy depends on step size and smoothness.

  1. Save the forward operands required by local derivative rules.
  2. Seed the scalar loss derivative with one.
  3. Traverse operations in reverse dependency order and add, rather than overwrite, shared-input gradients.
  4. Compare selected parameter derivatives with central differences while holding every other input fixed.

Science and assumptions

A differentiable computation graph · worked relation

vˉi+=vˉj∂vj∂vi,∂L∂w≈L(w+h)−L(w−h)2h\bar v_i\mathrel{+}=\bar v_j\frac{\partial v_j}{\partial v_i},\qquad \frac{\partial L}{\partial w}\approx\frac{L(w+h)-L(w-h)}{2h}

Accumulate every branch contribution to a reused parameter and check it by finite differences.

Symbols and units
v_i,v_j
Intermediate scalar values

dimensionless

bar v_i
Accumulated derivative of loss with respect to v_i

loss per value

w
Reused trainable scalar

dimensionless

h
Finite-difference perturbation

dimensionless

L
Scalar objective

dimensionless

Let u=wx and L=(u+w)^2/2. At w=2 and x=3, u=6, u+w=8 and L=32. The path through u contributes 8×3=24 to dL/dw; the direct w path contributes 8, giving 32. Treating the second path as a replacement would give 8, while omitting it would give 24. For this quadratic, [L(w+h)-L(w-h)]/(2h) is 32 in exact arithmetic for any nonzero h.

Assumptions Smooth scalar graph and nonzero finite-difference step; floating-point cancellation can limit very small h. Lifecycle tape executes these mechanisms for all 423 parameters.

Engineering example

Let u=wx and L=(u+w)^2/2. At w=2 and x=3, u=6, u+w=8 and L=32. The path through u contributes 8×3=24 to dL/dw; the direct w path contributes 8, giving 32. Treating the second path as a replacement would give 8, while omitting it would give 24. For this quadratic, [L(w+h)-L(w-h)]/(2h) is 32 in exact arithmetic for any nonzero h.

Distinctions and tradeoffs

It is a mistake to conclude: 24, because only the multiplication path is differentiable.

Retaining intermediates reduces recomputation but consumes memory. Gradient checks are expensive if run on every parameter of a large model.

Join existing forward mechanisms into an initialized model with real gradients and measurable predictions. Accumulate every branch contribution to a reused parameter and check it by finite differences.

Specifications and interfaces

Learning mode
executable core

The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.

Worked contract
Accumulate every branch contribution to a reused parameter and check it by finite differences.

Smooth scalar graph and nonzero finite-difference step; floating-point cancellation can limit very small h. Lifecycle tape executes these mechanisms for all 423 parameters.

Interfaces

Inputs and state Input
Save the forward operands required by local derivative rules.
Result and check Output
Accumulate every branch contribution to a reused parameter and check it by finite differences.

Check your understanding

What is the correct derivative with respect to w?

  1. 24, because only the multiplication path is differentiable.
  2. 8, because the later path overwrites the earlier one.
  3. 32, because both computational paths contribute.
Answer and reasoning

32, because both computational paths contribute.

32, because both computational paths contribute. Smooth scalar graph and nonzero finite-difference step; floating-point cancellation can limit very small h. Lifecycle tape executes these mechanisms for all 423 parameters.

Evidence and scope

Mode: executable core. Smooth scalar graph and nonzero finite-difference step; floating-point cancellation can limit very small h. Lifecycle tape executes these mechanisms for all 423 parameters. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.

Model lifecycle studio: executable core. Smooth scalar graph and nonzero finite-difference step; floating-point cancellation can limit very small h. Lifecycle tape executes these mechanisms for all 423 parameters.

Sources

Published lesson data · Atlas releases and licensing

Continue exploring

Foundations

Related lessons