{
  "schemaVersion": 1,
  "id": "atlas-lesson-temporal-forecast-validation",
  "title": "Forecasts are tested forward in time",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Horizon, rolling origin, seasonal baseline, probabilistic intervals and leakage.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/temporal-forecast-validation/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Models become evaluated applications",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.\n\nExecutable ridge and recursive forecasting use original deterministic hourly data. They are not a real demand forecast, independent-sample confidence estimate or neural forecasting benchmark.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/temporal-forecast-validation-17cdba5c8b1b6fd3.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#temporal-forecast-validation",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "24bdce8899e64545d0114cd057b054f8485d9dafd611527859fc8ebf76ba8038",
  "contentLength": 7279,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-temporal-forecast-validation.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-temporal-forecast-validation.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Forecasts are tested forward in time",
      "start": 53,
      "end": 457
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 457,
      "end": 2109
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 2109,
      "end": 2137
    },
    {
      "id": "section-4",
      "title": "Forecasts are tested forward in time · explicit relationship",
      "start": 2137,
      "end": 3247
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 3247,
      "end": 4162
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 4162,
      "end": 4532
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 4532,
      "end": 4989
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4989,
      "end": 5248
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 5248,
      "end": 5277
    },
    {
      "id": "section-10",
      "title": "Why must a normalization fit exclude future evaluation values?",
      "start": 5277,
      "end": 5677
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 5677,
      "end": 6424
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 6424,
      "end": 6853
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 6853,
      "end": 6876
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 6876,
      "end": 7064
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 7064,
      "end": 7279
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#temporal-forecast-validation"
    },
    {
      "title": "Chronos: Learning the Language of Time Series",
      "url": "https://arxiv.org/abs/2403.07815v3"
    },
    {
      "title": "scikit-learn · Common pitfalls and data leakage",
      "url": "https://scikit-learn.org/stable/common_pitfalls.html#data-leakage"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/temporal-forecast-validation-17cdba5c8b1b6fd3.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Training data pipeline",
      "url": "https://semiaifoundry.com/learn/atlas/training-data/"
    },
    {
      "title": "Evaluation & the AGI frontier",
      "url": "https://semiaifoundry.com/learn/atlas/evaluation/"
    },
    {
      "title": "Tools, memory & agent loop",
      "url": "https://semiaifoundry.com/learn/atlas/tool-loop/"
    },
    {
      "title": "A code patch needs execution evidence",
      "url": "https://semiaifoundry.com/learn/atlas/coding-agent-patch-evidence/"
    }
  ],
  "recordHash": "860c4165b66eafcd09c88e0e34fe7a22cb9afdb71681aca6abf7c2d41ce58a57",
  "contentFormat": "text/markdown",
  "content": "Models become evaluated applications · Atlas v1.3.0\n\n# Forecasts are tested forward in time\n\nHorizon, rolling origin, seasonal baseline, probabilistic intervals and leakage.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#temporal-forecast-validation)Logical state, task evidence and declared units; no physical assembly scale\n\nLearning objectivePreserve time availability and compare forecasts and interval coverage on the same declared horizons.\n\n## The mechanism\n\nA forecast is a prediction made at a time using only information available then. Its horizon determines how far ahead the target lies. Training and evaluation must preserve that availability boundary: randomly mixing future and past observations can leak information into a test that appears predictive. Rolling-origin evaluation advances the cutoff and tests later horizons, while simple seasonal or last-value baselines provide a meaningful comparison. Probabilistic forecasts add intervals or distributions whose coverage and calibration require evaluation. Chronos supplies primary context for learning time-series representations through scaling and quantization. The worked numbers here compare explicit forecasts; they do not run Chronos or establish a forecasting advantage on a real dataset.\n\nThe application-contract experience fits an actual five-coefficient ridge model to hours 1–23. Each rolling origin uses its latest observation, then recursively feeds predictions through the selected horizon. Later observations begin a new forecast but do not update model coefficients. A persistence baseline uses the exact same targets. A future-label intervention produces zero error while explicitly violating information availability.\n\n1.  A forecast origin defines the available observations and the later target horizon.\n2.  Training, normalization and model selection use only the permitted earlier information.\n3.  Predictions are compared with a declared simple baseline at the same origins and horizons.\n4.  Error and interval coverage are reported with sample size and dependence limits, rather than implied certainty.\n\n## Science and assumptions\n\n### Forecasts are tested forward in time · explicit relationship\n\n$$\n\\operatorname{MAE}=\\frac{1}{H}\\sum_{h=1}^{H}|y_h-\\widehat y_h|\n$$\n\nMean absolute error averages the absolute forecast error over the declared positive number of horizons.\n\nSymbols and units\n\nH\n\nNumber of evaluated horizons in this example\n\ncount\n\nyₕ\n\nRealized target at horizon h\n\ntarget units\n\nŷₕ\n\nForecast issued at the stated origin\n\ntarget units\n\nMAE\n\nMean absolute error\n\ntarget units\n\nAt one origin, future values are \\[12,15,13\\]. A candidate predicts \\[10,14,16\\], so absolute errors \\[2,1,3\\] give mean absolute error 2. A last-value baseline of 11 predicts \\[11,11,11\\], giving errors \\[1,4,2\\] and mean 7/3. The candidate is better on this tiny three-target example, not necessarily on other origins. If two of the three realized values lie inside a stated 90% interval, observed coverage is 2/3; three possibly dependent targets cannot establish long-run 90% calibration.\n\nAssumptions All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.\n\n## Engineering example\n\nAt one origin, future values are \\[12,15,13\\]. A candidate predicts \\[10,14,16\\], so absolute errors \\[2,1,3\\] give mean absolute error 2. A last-value baseline of 11 predicts \\[11,11,11\\], giving errors \\[1,4,2\\] and mean 7/3. The candidate is better on this tiny three-target example, not necessarily on other origins. If two of the three realized values lie inside a stated 90% interval, observed coverage is 2/3; three possibly dependent targets cannot establish long-run 90% calibration.\n\nWith a +4-unit shift beginning at hour 32, the default one-hour forecast scores 24 target rows. Its mean absolute error is about 3.26680 synthetic load units, while persistence achieves 1.10411 on the same rows. More fitting machinery does not guarantee the better forecast. At an eight-hour horizon the number of scored origins is 17, and the separately matched baseline must be reported again.\n\n## Distinctions and tradeoffs\n\nA low error after a random time split can reflect leakage rather than predictive information.\n\nLonger horizons and changing regimes can degrade accuracy; narrower intervals look precise but must retain evaluated coverage.\n\nForecasting applications need temporal evidence discipline in addition to a model capable of processing sequences.\n\n## Specifications and interfaces\n\nMechanism boundary\n\nCould the forecast have been made using information available then?\n\nAll forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.\n\nEvidence type\n\nAuthored worked example\n\nWorked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.\n\n### Interfaces\n\nDeclared input and state Input\n\nA forecast origin defines the available observations and the later target horizon.\n\nChecked outcome Output\n\nPreserve time availability and compare forecasts and interval coverage on the same declared horizons.\n\n## Check your understanding\n\n### Why must a normalization fit exclude future evaluation values?\n\n1.  Because otherwise the forecast uses information unavailable at its origin\n2.  Because future data has no units\n3.  Because all time series must be constant\n\nAnswer and reasoning\n\nBecause otherwise the forecast uses information unavailable at its origin\n\nA forecast’s evidence boundary is what was available when it was issued.\n\n## Evidence and scope\n\nThis is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. All forecasts precede their targets; values are synthetic. A small set of correlated horizons is not a broad accuracy or calibration estimate.\n\nExecutable ridge and recursive forecasting use original deterministic hourly data. They are not a real demand forecast, independent-sample confidence estimate or neural forecasting benchmark.\n\nThe worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.\n\n## Sources\n\n-   [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815v3)\n-   [scikit-learn · Common pitfalls and data leakage](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/temporal-forecast-validation-17cdba5c8b1b6fd3.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Training data pipeline](https://semiaifoundry.com/learn/atlas/training-data/)\n-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)\n\n### Related lessons\n\n-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)\n-   [A code patch needs execution evidence](https://semiaifoundry.com/learn/atlas/coding-agent-patch-evidence/)"
}