{
  "schemaVersion": 1,
  "id": "atlas-lesson-long-context-adaptation",
  "title": "Context extension is a learned-use test",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Separate accepted context length, causal correctness, memory growth and demonstrated long-distance use.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/long-context-adaptation/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "model-lifecycle-continuation",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Mode: worked example. Uncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/long-context-adaptation-2a11cf9f6ad5a9fe.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#long-context-adaptation",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "627d090d975687c89504322e74b906bf34096c51bc4b601edacdef80b325cceb",
  "contentLength": 6480,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-long-context-adaptation.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-long-context-adaptation.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Context extension is a learned-use test",
      "start": 45,
      "end": 480
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 480,
      "end": 1417
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1417,
      "end": 1445
    },
    {
      "id": "section-4",
      "title": "Context extension is a learned-use test · worked relation",
      "start": 1445,
      "end": 2467
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2467,
      "end": 2898
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2898,
      "end": 3347
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3347,
      "end": 3810
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3810,
      "end": 4071
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4071,
      "end": 4100
    },
    {
      "id": "section-10",
      "title": "Which observation establishes storage capacity but not learned long-distance use?",
      "start": 4100,
      "end": 4759
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4759,
      "end": 5400
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5400,
      "end": 5724
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5724,
      "end": 5747
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5747,
      "end": 6115
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 6115,
      "end": 6480
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#long-context-adaptation"
    },
    {
      "title": "YaRN: Efficient Context Window Extension of Large Language Models",
      "url": "https://arxiv.org/html/2309.00071v2"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/long-context-adaptation-2a11cf9f6ad5a9fe.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Position representation",
      "url": "https://semiaifoundry.com/learn/atlas/positional-encoding/"
    },
    {
      "title": "Self-attention",
      "url": "https://semiaifoundry.com/learn/atlas/attention/"
    },
    {
      "title": "Key–value cache",
      "url": "https://semiaifoundry.com/learn/atlas/kv-cache/"
    },
    {
      "title": "Continue language modeling on a new domain",
      "url": "https://semiaifoundry.com/learn/atlas/domain-adaptive-continuation/"
    },
    {
      "title": "Describe a stage by its changed contract",
      "url": "https://semiaifoundry.com/learn/atlas/model-lifecycle-stage-contract/"
    },
    {
      "title": "Retention, replay and transfer",
      "url": "https://semiaifoundry.com/learn/atlas/replay-retention-forgetting/"
    },
    {
      "title": "Compatible weight averaging and regressions",
      "url": "https://semiaifoundry.com/learn/atlas/checkpoint-averaging-boundaries/"
    }
  ],
  "recordHash": "45ea46ac699f2cb9c5000c4d74895a6492d45ba8edec2dc15af0a9466853780a",
  "contentFormat": "text/markdown",
  "content": "model-lifecycle-continuation · Atlas v1.3.0\n\n# Context extension is a learned-use test\n\nSeparate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#long-context-adaptation)Mathematical, data and behavior space; logical dimensions are not physical distance\n\nLearning objectiveSeparate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\n## The mechanism\n\nA model accepting a longer input is only the first part of context extension. Positional representations must remain meaningful over new distances, masks must still enforce causality, and training/evaluation must test whether earlier evidence is actually used. YaRN studies a particular rotary-position extension method; its mechanism is not the fixed sinusoidal position rule of the 423-parameter core. Retrieval at long distances, distractor resistance and instruction following can fail separately. Increasing the storage limit alone establishes none of those abilities.\n\n1.  Declare original and extended positional configurations and the data used for adaptation.\n2.  Test future-token invariance independently of task correctness.\n3.  Move a required symbol to different distances and compare an evidence-removed control.\n4.  Inventory KV elements at each length with explicit head, layer and dtype conventions.\n\n## Science and assumptions\n\n### Context extension is a learned-use test · worked relation\n\n$$\nB_{KV}=2LTH_{KV}d_hb\n$$\n\nSeparate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\nSymbols and units\n\nB\\_KV\n\nKey plus value payload\n\nbytes\n\nL\n\nLayer count\n\nlayers\n\nT\n\nStored token positions\n\ntokens\n\nH\\_KV\n\nKV head count\n\nheads\n\nd\\_h\n\nFeatures per KV head\n\nfeatures\n\nb\n\nStored bytes per element\n\nbytes per element\n\nWith two layers, two KV heads, four features per head and two bytes per stored element, a full KV cache uses 2×layers×tokens×heads×features×bytes. At 128 tokens it is 8192 bytes; at 512 it is 32768 bytes. This fourfold storage increase follows from the declared shape. If a 512-token interface still answers only 5/10 far-evidence cases, accepted length 512 is not evidence of successful long-context use.\n\nAssumptions Uncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN.\n\n## Engineering example\n\nWith two layers, two KV heads, four features per head and two bytes per stored element, a full KV cache uses 2×layers×tokens×heads×features×bytes. At 128 tokens it is 8192 bytes; at 512 it is 32768 bytes. This fourfold storage increase follows from the declared shape. If a 512-token interface still answers only 5/10 far-evidence cases, accepted length 512 is not evidence of successful long-context use.\n\n## Distinctions and tradeoffs\n\nIt is a mistake to conclude: Answers change correctly when the required far-away evidence changes.\n\nLonger contexts consume storage and attention work; positional adaptation and useful information retrieval require their own evidence.\n\nAdapt an existing checkpoint while measuring retained and acquired behavior. Separate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\n## Specifications and interfaces\n\nLearning mode\n\nworked example\n\nThe executable decoder has 423 parameters; worked extensions do not imply an additional simulator.\n\nWorked contract\n\nSeparate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\nUncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN.\n\n### Interfaces\n\nInputs and state Input\n\nDeclare original and extended positional configurations and the data used for adaptation.\n\nResult and check Output\n\nSeparate accepted context length, causal correctness, memory growth and demonstrated long-distance use.\n\n## Check your understanding\n\n### Which observation establishes storage capacity but not learned long-distance use?\n\n1.  A reserved long-distance task is solved under a fixed scoring rule.\n2.  The system accepts 512 tokens and allocates the corresponding KV payload.\n3.  Answers change correctly when the required far-away evidence changes.\n\nAnswer and reasoning\n\nThe system accepts 512 tokens and allocates the corresponding KV payload.\n\nThe system accepts 512 tokens and allocates the corresponding KV payload. Uncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN.\n\n## Evidence and scope\n\nMode: worked example. Uncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.\n\nModel lifecycle studio: worked example. Uncompressed full KV payload with no sharing or overhead. This lesson is a worked extension; the lifecycle core has at most eight input positions and does not implement YaRN.\n\n## Sources\n\n-   [YaRN: Efficient Context Window Extension of Large Language Models](https://arxiv.org/html/2309.00071v2)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/long-context-adaptation-2a11cf9f6ad5a9fe.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Position representation](https://semiaifoundry.com/learn/atlas/positional-encoding/)\n-   [Self-attention](https://semiaifoundry.com/learn/atlas/attention/)\n-   [Key–value cache](https://semiaifoundry.com/learn/atlas/kv-cache/)\n-   [Continue language modeling on a new domain](https://semiaifoundry.com/learn/atlas/domain-adaptive-continuation/)\n\n### Related lessons\n\n-   [Describe a stage by its changed contract](https://semiaifoundry.com/learn/atlas/model-lifecycle-stage-contract/)\n-   [Retention, replay and transfer](https://semiaifoundry.com/learn/atlas/replay-retention-forgetting/)\n-   [Compatible weight averaging and regressions](https://semiaifoundry.com/learn/atlas/checkpoint-averaging-boundaries/)"
}