{
  "schemaVersion": 1,
  "id": "atlas-lesson-tensor-memory",
  "title": "Tensor Memory",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Provides dedicated on-chip storage for fifth-generation Tensor Core operands and accumulators.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/tensor-memory/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Memory hierarchy",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The logical dimensions are those cited from PTX ISA 8.8 for sm_100a/sm_100f. The 32-column allocation is an arithmetic example, subject to actual allocation and residency rules; it is not a physical-cell layout or a portable GPU feature.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-memory-c62eedf232bc3a66.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#tensor-memory",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "cdca447b678a9855cebe0829b68979168c8bd0fe1880db5fa74773f6004a14d2",
  "contentLength": 6250,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-memory.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-memory.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Tensor Memory",
      "start": 33,
      "end": 383
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 383,
      "end": 1573
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1573,
      "end": 1601
    },
    {
      "id": "section-4",
      "title": "Logical shape determines storage capacity",
      "start": 1601,
      "end": 2831
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2831,
      "end": 3118
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 3118,
      "end": 3662
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3662,
      "end": 3963
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3963,
      "end": 4157
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4157,
      "end": 4186
    },
    {
      "id": "section-10",
      "title": "What is Tensor Memory’s main role?",
      "start": 4186,
      "end": 4629
    },
    {
      "id": "section-11",
      "title": "Historical context",
      "start": 4629,
      "end": 4652
    },
    {
      "id": "section-12",
      "title": "2025 · CUTLASS 3.8 exposes Blackwell Tensor Memory",
      "start": 4652,
      "end": 5020
    },
    {
      "id": "section-13",
      "title": "Evidence and scope",
      "start": 5020,
      "end": 5450
    },
    {
      "id": "section-14",
      "title": "Sources",
      "start": 5450,
      "end": 5858
    },
    {
      "id": "section-15",
      "title": "Continue exploring",
      "start": 5858,
      "end": 5881
    },
    {
      "id": "section-16",
      "title": "Foundations",
      "start": 5881,
      "end": 6058
    },
    {
      "id": "section-17",
      "title": "Related lessons",
      "start": 6058,
      "end": 6250
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#tensor-memory"
    },
    {
      "title": "NVIDIA CUTLASS 3.8 release discussion",
      "url": "https://github.com/NVIDIA/cutlass/discussions/2125"
    },
    {
      "title": "NVIDIA PTX ISA 8.8: fifth-generation Tensor Core instructions",
      "url": "https://docs.nvidia.com/cuda/pdf/ptx_isa_8.8.pdf"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-memory-c62eedf232bc3a66.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Tensor Core matrix unit",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-core/"
    },
    {
      "title": "Shared memory SRAM",
      "url": "https://semiaifoundry.com/learn/atlas/shared-sram/"
    },
    {
      "title": "Register file",
      "url": "https://semiaifoundry.com/learn/atlas/register-file/"
    },
    {
      "title": "Tensor Memory Accelerator",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-memory-accelerator/"
    }
  ],
  "recordHash": "3b94285d22a023c9a9c4d05b89ab28f711f67fe2cd42c53642c8d8e2df829f8c",
  "contentFormat": "text/markdown",
  "content": "Memory hierarchy · Atlas v1.3.0\n\n# Tensor Memory\n\nProvides dedicated on-chip storage for fifth-generation Tensor Core operands and accumulators.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#tensor-memory)Logical 128 × 512 address array inside an SM\n\nLearning objectiveConvert a documented logical Tensor Memory allocation into bytes without confusing it with HBM or TMA.\n\n## The mechanism\n\nTensor Memory is a Blackwell storage space specialized for Tensor Core operations. It is neither HBM nor the Tensor Memory Accelerator. Supported instructions allocate columns, move suitable fragments, launch matrix operations, and later retrieve results. Giving tensor operations their own storage changes how software pipelines matrix multiplication and manages register pressure. The programming model has explicit access and allocation rules, so the whole array is not a freely interchangeable replacement for ordinary shared memory. Column allocation and instruction layouts determine which tensor fragments can remain in this storage across successive matrix operations. The documented shape is an architectural address view, not a photograph or map of physical memory cells.\n\n1.  A supported kernel allocates columns in Tensor Memory and arranges tensor fragments according to the instruction’s required layout.\n2.  Tensor Core operations consume operands and update accumulators using this specialized nearby storage.\n3.  The kernel synchronizes, extracts completed results through supported instructions, and releases the allocation when it is no longer needed.\n\n## Science and assumptions\n\n### Logical shape determines storage capacity\n\n$$\n\\begin{aligned}C&=128\\times512\\times32\\,\\mathrm{bits}\\\\&=262{,}144\\,\\mathrm{bytes}=256\\,\\mathrm{KiB}\\end{aligned}\n$$\n\nThe documented array has 128 rows by 512 columns with 32 bits per cell: 2,097,152 bits, or 262,144 bytes, or 256 KiB.\n\nEight bits make one byte. This is the architectural array capacity, not a per-block allocation guarantee.\n\nSymbols and units\n\nC\n\nCapacity of the documented logical Tensor Memory array\n\nbytes\n\n128\n\nLogical row count\n\nrows\n\n512\n\nLogical column count\n\ncolumns\n\n32 bits\n\nStorage per logical cell\n\nbits/cell\n\nKiB\n\nBinary kibibyte\n\n1024 bytes\n\nThe documented $sm_{\\mathrm{100a}}/sm_{\\mathrm{100f}}$ address view has 128 rows, 512 columns, and 32 bits per cell. Dividing bits by eight gives bytes. This organization explains why column allocation reserves an entire vertical set of cells and why instruction layouts govern access. The number describes the architectural array, not a claim that every simultaneously resident block can consume an independent full-size copy.\n\nAssumptions Logical Tensor Memory organization from the PTX ISA. Allocation, residency, and access rules still apply. KiB uses 1,024 bytes, distinguishing it from decimal KB.\n\n## Engineering example\n\nUsing the lesson’s documented 128-row, 32-bit-cell address view, a hypothetical allocation of 32 columns reserves 128 × 32 × 32 / 8 = 16,384 bytes = 16 KiB. The full 512-column view is 256 KiB; that does not grant every resident block an independent full array.\n\n## Distinctions and tradeoffs\n\nTensor Memory is storage; the Tensor Memory Accelerator is a copy engine. Similar names describe different parts of the data path.\n\nDedicated storage relieves some pressure on general registers, but introduces layout and allocation constraints. Efficient kernels must overlap movement and compute while respecting synchronization rules.\n\nLarge matrix products repeatedly update partial sums before storing final values. Keeping these accumulators next to the tensor machinery improves the opportunity for reuse.\n\n## Specifications and interfaces\n\nAddress organization\n\n128 rows × 512 columns\n\nPTX sm\\_100a/sm\\_100f Tensor Memory view.\n\nCell size\n\n32 bits\n\nTotal logical array is 256 KiB by direct multiplication.\n\nAllocation unit\n\n32 columns\n\nAllocated column counts are powers of two under the documented rules.\n\n### Interfaces\n\nTile transfer Input\n\nSupported tensor fragments\n\nTensor execution Output\n\nOperands and accumulation storage\n\nResult extraction Output\n\nTensor Memory loads into thread registers\n\n## Check your understanding\n\n### What is Tensor Memory’s main role?\n\n1.  Converting AC power into DC power.\n2.  Holding specialized Tensor Core operands and accumulators on the chip.\n3.  Routing Ethernet packets between racks.\n\nAnswer and reasoning\n\nHolding specialized Tensor Core operands and accumulators on the chip.\n\nTMEM is an architectural storage space used by fifth-generation Tensor Core instructions. It is not a data center memory appliance or a copy engine.\n\n## Historical context\n\n### 2025 · CUTLASS 3.8 exposes Blackwell Tensor Memory\n\nThe February 2025 CUTLASS 3.8 release added Blackwell SM100 support and exposed Tensor Memory as a distinct storage location, along with operations that move values between registers, shared memory, and Tensor Memory.\n\n[NVIDIA CUTLASS 3.8 release discussion](https://github.com/NVIDIA/cutlass/discussions/2125)\n\n## Evidence and scope\n\nThe logical dimensions are those cited from PTX ISA 8.8 for $sm_{\\mathrm{100a}}/sm_{\\mathrm{100f}}.$ The 32-column allocation is an arithmetic example, subject to actual allocation and residency rules; it is not a physical-cell layout or a portable GPU feature.\n\nNo current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.\n\n## Sources\n\n-   [NVIDIA PTX ISA 8.8: fifth-generation Tensor Core instructions](https://docs.nvidia.com/cuda/pdf/ptx_isa_8.8.pdf)\n-   [NVIDIA CUTLASS 3.8 release](https://github.com/NVIDIA/cutlass/discussions/2125)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/tensor-memory-c62eedf232bc3a66.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/)\n-   [Shared memory SRAM](https://semiaifoundry.com/learn/atlas/shared-sram/)\n\n### Related lessons\n\n-   [Register file](https://semiaifoundry.com/learn/atlas/register-file/)\n-   [Tensor Memory Accelerator](https://semiaifoundry.com/learn/atlas/tensor-memory-accelerator/)"
}