# Forecasts are tested forward in time

Canonical: https://semiaifoundry.com/learn/atlas/temporal-forecast-validation/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Models become evaluated applications · Atlas v1.3.0

# Forecasts are tested forward in time

Horizon, rolling origin, seasonal baseline, probabilistic intervals and leakage.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#temporal-forecast-validation)Logical state, task evidence and declared units; no physical assembly scale

Learning objectivePreserve time availability and compare forecasts and interval coverage on the same declared horizons.

## The mechanism

A forecast is a prediction made at a time using only information available then. Its horizon determines how far ahead the target lies. Training and evaluation must preserve that availability boundary: randomly mixing future and past observations can leak information into a test that appears predictive. Rolling-origin evaluation advances the cutoff and tests later horizons, while simple seasonal or last-value baselines provide a meaningful comparison. Probabilistic forecasts add intervals or distributions whose coverage and calibration require evaluation. Chronos supplies primary context for learning time-series representations through scaling and quantization. The worked numbers here compare explicit forecasts; they do not run Chronos or establish a forecasting advantage on a real dataset.

The application-contract experience fits an actual five-coefficient ridge model to hours 1–23. Each rolling origin uses its latest observation, then recursively feeds predictions through the selected horizon. Later observations begin a new forecast but do not update model coefficients. A persistence baseline uses the exact same targets. A future-label intervention produces zero error while explicitly violating information availability.

1.  A forecast origin defines the available observations and the later target horizon.
2.  Training, normalization and model selection use only the permitted earlier information.
3.  Predictions are compared with a declared simple baseline at the same origins and horizons.
4.  Error and interval coverage are reported with sample size and dependence limits, rather than implied certainty.

## Science and assumptions

### Forecasts are tested forward in time · explicit relationship

$$
\operatorname{MAE}=\frac{1}{H}\sum_{h=1}^{H}|y_h-\widehat y_h|
$$

Mean absolute error averages the absolute forecast error over the declared positive number of horizons.

Symbols and units

H

Number of evaluated horizons in this example

count

yₕ

Realized target at horizon h

target units

ŷₕ

Forecast issued at the stated origin

target units

MAE

Mean absolute error

target units

At one origin, future values are \[12,15,13\]. A candidate predicts \[10,14,16\], so absolute errors \[2,1,3\] give mean absolute error 2. A last-value baseline of 11 predicts \[11,11,11\], giving errors \[1,4,2\] and mean 7/3. The candidate is better on this tiny three-target example, not necessarily on other origins. If two of the three realized values lie inside a stated 90% interval, observed coverage is 2/3; three possibly dependent targets cannot establish long-run 90% calibration.

Assumptions All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.

## Engineering example

At one origin, future values are \[12,15,13\]. A candidate predicts \[10,14,16\], so absolute errors \[2,1,3\] give mean absolute error 2. A last-value baseline of 11 predicts \[11,11,11\], giving errors \[1,4,2\] and mean 7/3. The candidate is better on this tiny three-target example, not necessarily on other origins. If two of the three realized values lie inside a stated 90% interval, observed coverage is 2/3; three possibly dependent targets cannot establish long-run 90% calibration.

With a +4-unit shift beginning at hour 32, the default one-hour forecast scores 24 target rows. Its mean absolute error is about 3.26680 synthetic load units, while persistence achieves 1.10411 on the same rows. More fitting machinery does not guarantee the better forecast. At an eight-hour horizon the number of scored origins is 17, and the separately matched baseline must be reported again.

## Distinctions and tradeoffs

A low error after a random time split can reflect leakage rather than predictive information.

Longer horizons and changing regimes can degrade accuracy; narrower intervals look precise but must retain evaluated coverage.

Forecasting applications need temporal evidence discipline in addition to a model capable of processing sequences.

## Specifications and interfaces

Mechanism boundary

Could the forecast have been made using information available then?

All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.

Evidence type

Authored worked example

Worked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.

### Interfaces

Declared input and state Input

A forecast origin defines the available observations and the later target horizon.

Checked outcome Output

Preserve time availability and compare forecasts and interval coverage on the same declared horizons.

## Check your understanding

### Why must a normalization fit exclude future evaluation values?

1.  Because otherwise the forecast uses information unavailable at its origin
2.  Because future data has no units
3.  Because all time series must be constant

Answer and reasoning

Because otherwise the forecast uses information unavailable at its origin

A forecast’s evidence boundary is what was available when it was issued.

## Evidence and scope

This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.

Executable ridge and recursive forecasting use original deterministic hourly data. They are not a real demand forecast, independent-sample confidence estimate or neural forecasting benchmark.

The worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.

## Sources

-   [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815v3)
-   [scikit-learn · Common pitfalls and data leakage](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/temporal-forecast-validation-17cdba5c8b1b6fd3.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [Training data pipeline](https://semiaifoundry.com/learn/atlas/training-data/)
-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)

### Related lessons

-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)
-   [A code patch needs execution evidence](https://semiaifoundry.com/learn/atlas/coding-agent-patch-evidence/)