Predict transitions conditional on actions

The mechanism

An action-conditioned world model predicts consequences from state and action inputs. A visually plausible continuation from observational video is not sufficient evidence that the model distinguishes interventions. Training requires action-linked transitions with a stated coordinate system and observation timing. One-step prediction error can be small while errors compound during repeated predicted steps, especially when a policy seeks states absent from training. V-JEPA 2-AC and Dreamer use different learned-state mechanisms; the worked scalar transition below isolates their shared action-dependence requirement without replicating either.

Retain state, executed action and next observation as one time-aligned transition.
Fit or inspect a predictor that receives the action explicitly.
Hold initial state fixed and compare alternative action predictions.
Evaluate one-step and multi-step errors separately on held-out trajectories and policy-induced states.

Science and assumptions

Predict transitions conditional on actions · worked relation

\hat x_{t+1}=f_\theta(\hat x_t,a_t),\qquad e_H=|\hat x_{t+H}-x_{t+H}|

Change an action while holding state fixed and compare predicted transitions with a held-out environment.

Symbols and units

x,hat x: Observed and predicted scalar position in the example
m
a: Declared normalized action command
dimensionless
f_theta: Learned or supplied transition predictor
function
t,H: Transition index and rollout horizon
steps
e_H: H-step position error
m

In an authored environment $x_{\mathrm{next}}=x+2a$ , x=1 with action +0.5 yields 2 and action −0.5 yields 0. A model $x_{\mathrm{hat,next}}=x+1.8a$ predicts 1.9 and 0.1, giving absolute error 0.1 each. Repeating action +0.5 for five predicted steps from x=1 gives model state 5.5 versus true state 6: error grows to 0.5. This exact linear example demonstrates accumulation, not a universal growth law.

Assumptions Example gain is 2 m per normalized command per step; model gain is 1.8. No clipping, noise or hidden state. This is worked arithmetic, not a trained world-model experience.

Engineering example

Distinctions and tradeoffs

It is a mistake to conclude: The generated scene merely looks realistic.

Action data supports control-relevant prediction but is limited by collected interventions and embodiment. A policy can exploit errors beyond that support.

Distinguish predicting observations, choosing actions and controlling a changing environment. Change an action while holding state fixed and compare predicted transitions with a held-out environment.

Specifications and interfaces

Learning mode: worked example
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
Worked contract: Change an action while holding state fixed and compare predicted transitions with a held-out environment.
Example gain is 2 m per normalized command per step; model gain is 1.8. No clipping, noise or hidden state. This is worked arithmetic, not a trained world-model experience.

Interfaces

Inputs and state Input: Retain state, executed action and next observation as one time-aligned transition.
Result and check Output: Change an action while holding state fixed and compare predicted transitions with a held-out environment.

Check your understanding

What distinguishes action-conditioned prediction from an action-free continuation?

Changing only the action can change the predicted next state.
The generated scene merely looks realistic.
Every future state is already included in the input.

Answer and reasoning

Changing only the action can change the predicted next state.

Changing only the action can change the predicted next state. Example gain is 2 m per normalized command per step; model gain is 1.8. No clipping, noise or hidden state. This is worked arithmetic, not a trained world-model experience.

Evidence and scope

Mode: worked example. Example gain is 2 m per normalized command per step; model gain is 1.8. No clipping, noise or hidden state. This is worked arithmetic, not a trained world-model experience. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.

Model lifecycle studio: worked example. Example gain is 2 m per normalized command per step; model gain is 1.8. No clipping, noise or hidden state. This is worked arithmetic, not a trained world-model experience.