{
  "schemaVersion": 1,
  "id": "atlas-lesson-l1-cache",
  "title": "L1 and texture cache",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Automatically retains recently accessed data and combines nearby thread requests near an SM.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/l1-cache/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Memory hierarchy",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The ratio is workload- and access-pattern-specific.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/l1-cache-7101d934167fccaf.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#l1-cache",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "3b88af0f776fa234b097935207579e43029892c059b47823b988154e6448f168",
  "contentLength": 5662,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-l1-cache.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-l1-cache.md",
  "sections": [
    {
      "id": "section-1",
      "title": "L1 and texture cache",
      "start": 33,
      "end": 339
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 339,
      "end": 1444
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1444,
      "end": 1472
    },
    {
      "id": "section-4",
      "title": "Useful bytes versus bytes actually moved",
      "start": 1472,
      "end": 2752
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2752,
      "end": 2907
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2907,
      "end": 3495
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3495,
      "end": 3905
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3905,
      "end": 4068
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4068,
      "end": 4097
    },
    {
      "id": "section-10",
      "title": "Which access pattern generally makes better use of memory transactions?",
      "start": 4097,
      "end": 4552
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4552,
      "end": 4781
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4781,
      "end": 5198
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5198,
      "end": 5221
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5221,
      "end": 5324
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5324,
      "end": 5662
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#l1-cache"
    },
    {
      "title": "NVIDIA CUDA Blackwell Tuning Guide",
      "url": "https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html"
    },
    {
      "title": "NVIDIA CUDA C++ Best Practices Guide",
      "url": "https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/l1-cache-7101d934167fccaf.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Six-transistor static memory cell",
      "url": "https://semiaifoundry.com/learn/atlas/sram/"
    },
    {
      "title": "Streaming multiprocessor",
      "url": "https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/"
    },
    {
      "title": "Shared memory SRAM",
      "url": "https://semiaifoundry.com/learn/atlas/shared-sram/"
    },
    {
      "title": "Shared L2 cache",
      "url": "https://semiaifoundry.com/learn/atlas/l2-cache/"
    },
    {
      "title": "Register file",
      "url": "https://semiaifoundry.com/learn/atlas/register-file/"
    }
  ],
  "recordHash": "9523acec3a267d7ddf16691bef0f54a14224d7d1b550b767a6ccb0b3b79f7a61",
  "contentFormat": "text/markdown",
  "content": "Memory hierarchy · Atlas v1.3.0\n\n# L1 and texture cache\n\nAutomatically retains recently accessed data and combines nearby thread requests near an SM.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#l1-cache)On-die memory near SMs; illustrative cache lines\n\nLearning objectiveDefine useful bytes separately from transferred bytes.\n\n## The mechanism\n\nThe L1 cache is the first automatic cache close to an SM’s execution units. When threads request memory, hardware checks whether useful data are already nearby. Hits can avoid a longer journey; misses continue toward lower levels of the hierarchy. Consecutive thread addresses can be coalesced into fewer memory transactions, while scattered addresses may move many unused bytes. Blackwell shares a physical capacity budget among L1, texture functionality, and the shared-memory carveout. This differs from the explicit ownership and lifetime of software-managed shared memory. A cache hit serves a load locally, while a miss requests data farther down the hierarchy. Reshaping a warp’s addresses can change transferred bytes even when it computes the same values.\n\n1.  Threads issue addresses, and the memory system combines compatible nearby requests into transactions.\n2.  A cache lookup supplies data that are already present near the SM; a miss continues toward the next memory level.\n3.  Returned data may populate the cache, making later local or nearby accesses less expensive.\n\n## Science and assumptions\n\n### Useful bytes versus bytes actually moved\n\n$$\n\\eta_{\\mathrm{useful}}=\\frac{M_{\\mathrm{useful}}}{M_{\\mathrm{transferred}}},\\qquad B_{\\mathrm{useful}}=\\eta_{\\mathrm{useful}}B_{\\mathrm{transferred}}\n$$\n\nUseful-byte efficiency is useful bytes divided by transferred bytes. Multiplying transferred bandwidth by that efficiency gives useful bandwidth.\n\nSymbols and units\n\nη\\_useful\n\nUseful-byte fraction for the declared traffic accounting\n\nfraction\n\nM\\_useful\n\nUseful bytes in the counted transfers\n\nbytes\n\nM\\_transferred\n\nAll bytes physically moved for those transfers\n\nbytes\n\nB\\_useful\n\nUseful byte rate\n\nbytes/s\n\nB\\_transferred\n\nTransferred byte rate over the same interval\n\nbytes/s\n\nMuseful counts bytes the computation needs; Mtransferred counts all bytes fetched in the memory transactions serving those needs. A scattered access pattern can move many unused bytes, reducing effective bandwidth even if the physical interface is busy. Coalescing neighboring thread requests and reusing cache lines improve the fraction of movement that performs useful work.\n\nAssumptions Accounting over the same interval and memory boundary. Count unique or repeated demand consistently, and treat compression, cache hits, write traffic, and replayed transactions separately when profiling.\n\n## Engineering example\n\nIf only half of transferred cache-line bytes feed useful arithmetic, η\\_useful is 0.5 even though the interface moved every byte.\n\n## Distinctions and tradeoffs\n\nA 256 KB combined L1/texture/shared-memory capacity does not mean 256 KB of L1 plus another 256 KB of shared memory. They share a capacity budget.\n\nA cache helps when nearby or recent data will be reused, but unpredictable access can defeat locality. Increasing software-managed shared memory can improve explicit reuse while reducing space available to automatic caching.\n\nEmbedding lookups, normalization, and intermediate tensor access can become memory limited. Data layout determines whether the hierarchy moves useful numbers or mostly unused bytes.\n\n## Specifications and interfaces\n\nCombined capacity\n\n256 KB per SM\n\nBlackwell B200 L1/texture/shared-memory aggregate; not 256 KB of L1 plus shared memory.\n\nAllocation\n\nConfigurable carveout\n\nA larger shared-memory reservation leaves a different cache allocation.\n\nCoalescing example\n\n32-byte transaction segments\n\nCUDA programming guidance; actual request handling depends on instruction and access pattern.\n\n### Interfaces\n\nMemory request Input\n\nWarp load/store addresses\n\nCache hit Output\n\nLocally available data\n\nCache miss Output\n\nRequest toward L2 and device memory\n\n## Check your understanding\n\n### Which access pattern generally makes better use of memory transactions?\n\n1.  Nearby threads requesting neighboring useful values.\n2.  Every thread requesting an unrelated distant address.\n3.  Every load deliberately bypassing all reuse.\n\nAnswer and reasoning\n\nNearby threads requesting neighboring useful values.\n\nCoalescing can serve neighboring addresses with fewer transactions, reducing the number of unused bytes moved for the same computation.\n\n## Evidence and scope\n\nThe ratio is workload- and access-pattern-specific.\n\nUse the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.\n\n## Sources\n\n-   [NVIDIA CUDA Blackwell Tuning Guide](https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html)\n-   [NVIDIA CUDA C++ Best Practices Guide](https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/l1-cache-7101d934167fccaf.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Six-transistor static memory cell](https://semiaifoundry.com/learn/atlas/sram/)\n\n### Related lessons\n\n-   [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/)\n-   [Shared memory SRAM](https://semiaifoundry.com/learn/atlas/shared-sram/)\n-   [Shared L2 cache](https://semiaifoundry.com/learn/atlas/l2-cache/)\n-   [Register file](https://semiaifoundry.com/learn/atlas/register-file/)"
}