Train a denoiser and sample iteratively

The mechanism

Generative training can operate on continuous values rather than discrete next-token labels. In a DDPM noise-prediction objective, a clean example is mixed with Gaussian noise at a chosen level and the model predicts that noise. Generation later runs a specified reverse sampling process; a single good denoising prediction is not a complete generated sample. Flow matching instead regresses a vector field along a declared probability path. The target, time convention and sampler therefore differ even when both start from noise. Neither mechanism is implemented by the tiny causal decoder.

Specify whether time indexes a discrete diffusion level or a continuous path.
Construct the noisy/interpolated input and exact regression target from retained random values.
Compute the declared coordinate-sum or coordinate-mean loss.
Evaluate samples from a complete stated sampler with fixed seeds and quality metrics separate from regression loss.

Science and assumptions

Train a denoiser and sample iteratively · worked relation

x_t=\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\epsilon,\quad L_\epsilon=\lVert\epsilon-\epsilon_\theta(x_t,t)\rVert^2,\qquad x_t^{flow}=(1-t)x_0+t x_1,\quad u=x_1-x_0

Compute a noisy training target and distinguish noise regression from a flow vector field.

Symbols and units

x_0: Clean diffusion value or explicitly named flow start
dimensionless
x_1: Data endpoint in the flow example
dimensionless
alpha bar_t: Cumulative diffusion signal factor
dimensionless
epsilon: Sampled standard Gaussian noise
dimensionless
t: Declared diffusion index or normalized flow time
index or dimensionless
u: Straight-path target velocity
value per normalized time
L_epsilon: Coordinate-summed noise error
squared value

For DDPM-style training, x0=2, cumulative alpha=0.64 and sampled noise epsilon=−1 give xt=0.8×2+0.6×(−1)=1. A noise prediction −0.5 has squared error 0.25. In a separate straight flow path from noise x0=−1 to data x1=2, time t=0.25 gives xt=−0.25 and target velocity x1−x0=3. The noise target −1 and flow target 3 refer to different constructions.

Assumptions One-dimensional authored samples; diffusion and flow use separate time/start conventions explicitly. No complete sampler, likelihood bound or image-quality benchmark is executed.

Engineering example

Distinctions and tradeoffs

It is a mistake to conclude: All noise-based objectives must have the same target.

Continuous objectives offer different representation and sampling choices. More sampling steps incur additional work but do not alone guarantee useful or faithful outputs.

Teach modality-specific representation, objective and evaluation choices. Compute a noisy training target and distinguish noise regression from a flow vector field.

Specifications and interfaces

Learning mode: worked example
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
Worked contract: Compute a noisy training target and distinguish noise regression from a flow vector field.
One-dimensional authored samples; diffusion and flow use separate time/start conventions explicitly. No complete sampler, likelihood bound or image-quality benchmark is executed.

Interfaces

Inputs and state Input: Specify whether time indexes a discrete diffusion level or a continuous path.
Result and check Output: Compute a noisy training target and distinguish noise regression from a flow vector field.

Check your understanding

Why are the two regression targets different?

The flow velocity is the model’s next-token probability.
They supervise different declared paths and predicted quantities.
All noise-based objectives must have the same target.

Answer and reasoning

They supervise different declared paths and predicted quantities.

They supervise different declared paths and predicted quantities. One-dimensional authored samples; diffusion and flow use separate time/start conventions explicitly. No complete sampler, likelihood bound or image-quality benchmark is executed.

Evidence and scope

Mode: worked example. One-dimensional authored samples; diffusion and flow use separate time/start conventions explicitly. No complete sampler, likelihood bound or image-quality benchmark is executed. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.

Model lifecycle studio: worked example. One-dimensional authored samples; diffusion and flow use separate time/start conventions explicitly. No complete sampler, likelihood bound or image-quality benchmark is executed.