# Recurrent state and convolution Canonical: https://semiaifoundry.com/learn/atlas/scalar-state-space-recurrence/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Contrasting computational structures · Atlas v1.3.0 # Recurrent state and convolution Carry history in a state; recover the same output by an explicitly unrolled kernel. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#scalar-state-space-recurrence)Abstract computation with explicit coordinate, sample and operation units Learning objectiveExplain why a recurrence and a convolution agree for the declared time-invariant system, and why a reset changes their history boundary. ## The mechanism A sequence model needs a representation of what preceded the current observation. A linear state-space system updates a hidden state from its previous value and a new input, then reads out an output. For a time-invariant system, repeatedly substituting the state update expresses that output as a convolution with an impulse-response kernel. The recurrent and convolutional forms are two computations of the same declared map, with different scheduling and storage opportunities. The atlas reduces this relationship to one scalar state so every contribution can be inspected. Changing retention changes the temporal weighting. A reset deliberately discards earlier contributions. Modern structured and selective state-space architectures build on richer matrices, discretization, learned parameters and sometimes input-dependent transitions; their capabilities do not follow from this scalar example. 1. Start from zero state and sixteen specified input samples, with input gain b = 0.3. 2. At each sample compute h = a × previous h + b × input; retain exactly one state scalar. 3. Independently multiply every input since the last reset by b a raised to its age and sum the contributions. 4. Compare the two results and vary a across decay, sign alternation and growth. ## Science and assumptions ### Recurrent state and convolution · governing computation $$ h_t=a h_{t-1}+b x_t=\sum_{j=r}^{t}b a^{t-j}x_j $$ Explain why a recurrence and a convolution agree for the declared time-invariant system, and why a reset changes their history boundary. Symbols and units h\_t Retained scalar state state units a Time-invariant retention dimensionless b Input gain state units per input unit x\_j Input sample input units r Start of the current zero-state history interval sample index With a = 0.7, b = 0.3 and the first four inputs \[1, 0, 0, 0\], recurrent states are \[0.3, 0.21, 0.147, 0.1029\]. The impulse kernel has those same first four values. Resetting before sample 2 makes that sample and the next zero, because the earlier impulse is no longer in the retained state. Setting a = −0.7 alternates the sign; a = 1.05 grows an isolated impulse instead of forgetting it. Assumptions Zero initial state; a and b are constant within the episode; one scalar state; no trained S4, selective state-space model or learned long-memory guarantee. ## Engineering example With a = 0.7, b = 0.3 and the first four inputs \[1, 0, 0, 0\], recurrent states are \[0.3, 0.21, 0.147, 0.1029\]. The impulse kernel has those same first four values. Resetting before sample 2 makes that sample and the next zero, because the earlier impulse is no longer in the retained state. Setting a = −0.7 alternates the sign; a = 1.05 grows an isolated impulse instead of forgetting it. ## Distinctions and tradeoffs One persistent scalar and a long sequence do not imply perfect long-term memory or task understanding. A compact recurrent state makes incremental execution possible, while parallel convolution can expose more training parallelism. Neither representation removes the need to choose and learn a suitable memory structure. State-space and recurrent models provide a different way to represent temporal dependencies and workload memory from storing attention keys and values. ## Specifications and interfaces Computational contract Carry history in a state; recover the same output by an explicitly unrolled kernel. Zero initial state; a and b are constant within the episode; one scalar state; no trained S4, selective state-space model or learned long-memory guarantee. Learning evidence Inspectable executable specimen Inputs, intermediate values, exact output and a declared comparison or failure are available in Contrasting AI mechanisms. ### Interfaces Specified input Input Start from zero state and sixteen specified input samples, with input gain b = 0.3. Computed output Output Explain why a recurrence and a convolution agree for the declared time-invariant system, and why a reset changes their history boundary. ## Check your understanding ### Why must an input from before an explicit zero-state reset be excluded from the unrolled sum? 1. A reset changes the input sample rate 2. The reset discards the state that carried that contribution 3. A convolution can never represent recurrence Answer and reasoning The reset discards the state that carried that contribution The two forms agree only when they use the same initial/reset state and history interval. ## Evidence and scope Zero initial state; a and b are constant within the episode; one scalar state; no trained S4, selective state-space model or learned long-memory guarantee. Numerical settings are original teaching fixtures. The named experience executes the stated bounded algorithm; no production performance or independent human review is claimed. Zero initial state; a and b are constant within the episode; one scalar state; no trained S4, selective state-space model or learned long-memory guarantee. ## Sources - [Gu et al., state spaces and recurrence/convolution §2.3–2.4](https://arxiv.org/html/2111.00396v3) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/scalar-state-space-recurrence-8e4b73d079f45047.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [Tensor shapes, indices and contractions](https://semiaifoundry.com/learn/atlas/tensor-index-contracts/) ### Related lessons - [Graph message passing and node identity](https://semiaifoundry.com/learn/atlas/graph-message-passing/) - [Self-attention](https://semiaifoundry.com/learn/atlas/attention/) - [Distributed checkpoint and recovery state](https://semiaifoundry.com/learn/atlas/checkpoint/)