{
  "schemaVersion": 1,
  "id": "atlas-lesson-tensor-core",
  "title": "Tensor Core matrix unit",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Multiplies small matrix tiles and accumulates their products with supported numerical formats.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/tensor-core/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Compute architecture",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Tile dimensions and accumulation precision are configuration choices.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-core-e72167e744c85d77.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#tensor-core",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "60638d7cb96c98275455c4d114ec9b8f393e8bf1514e03f8ea1952c8f65f27d6",
  "contentLength": 6666,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-core.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-core.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Tensor Core matrix unit",
      "start": 37,
      "end": 371
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 371,
      "end": 1571
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1571,
      "end": 1599
    },
    {
      "id": "section-4",
      "title": "Why matrix size and reuse matter",
      "start": 1599,
      "end": 2817
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2817,
      "end": 2964
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2964,
      "end": 3559
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3559,
      "end": 3897
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3897,
      "end": 4021
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4021,
      "end": 4050
    },
    {
      "id": "section-10",
      "title": "What does a Tensor Core primarily accelerate?",
      "start": 4050,
      "end": 4508
    },
    {
      "id": "section-11",
      "title": "Historical context",
      "start": 4508,
      "end": 4531
    },
    {
      "id": "section-12",
      "title": "2017 · Volta introduces NVIDIA Tensor Cores",
      "start": 4531,
      "end": 5028
    },
    {
      "id": "section-13",
      "title": "Evidence and scope",
      "start": 5028,
      "end": 5254
    },
    {
      "id": "section-14",
      "title": "Sources",
      "start": 5254,
      "end": 5992
    },
    {
      "id": "section-15",
      "title": "Continue exploring",
      "start": 5992,
      "end": 6015
    },
    {
      "id": "section-16",
      "title": "Foundations",
      "start": 6015,
      "end": 6124
    },
    {
      "id": "section-17",
      "title": "Related lessons",
      "start": 6124,
      "end": 6666
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#tensor-core"
    },
    {
      "title": "NVIDIA: Volta GPU platform launch",
      "url": "https://nvidianews.nvidia.com/news/nvidia-launches-revolutionary-volta-gpu-platform-fueling-next-era-of-ai-and-high-performance-computing"
    },
    {
      "title": "NVIDIA CUTLASS: Blackwell specifics",
      "url": "https://docs.nvidia.com/cutlass/4.5.1/media/docs/cpp/blackwell.html"
    },
    {
      "title": "NVIDIA GB200 NVL72 specifications",
      "url": "https://www.nvidia.com/en-us/data-center/gb200-nvl72/"
    },
    {
      "title": "NVIDIA Matrix Multiplication Background Guide",
      "url": "https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-core-e72167e744c85d77.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Multiply–accumulate and fused arithmetic",
      "url": "https://semiaifoundry.com/learn/atlas/mac/"
    },
    {
      "title": "Streaming multiprocessor",
      "url": "https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/"
    },
    {
      "title": "Tensor Memory",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-memory/"
    },
    {
      "title": "Shared memory SRAM",
      "url": "https://semiaifoundry.com/learn/atlas/shared-sram/"
    },
    {
      "title": "Tensor Memory Accelerator",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-memory-accelerator/"
    },
    {
      "title": "Self-attention",
      "url": "https://semiaifoundry.com/learn/atlas/attention/"
    },
    {
      "title": "Compiler, intermediate code, and device executable",
      "url": "https://semiaifoundry.com/learn/atlas/compiler/"
    }
  ],
  "recordHash": "448736299efee7620efd0bda8f29c7bf5344bc124b9bcb2be623f7ccfa878e10",
  "contentFormat": "text/markdown",
  "content": "Compute architecture · Atlas v1.3.0\n\n# Tensor Core matrix unit\n\nMultiplies small matrix tiles and accumulates their products with supported numerical formats.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#tensor-core)Subregion of an SM; logical matrix tiles enlarged for inspection\n\nLearning objectiveTranslate a matrix tile into operations and data movement.\n\n## The mechanism\n\nA Tensor Core implements matrix multiply and accumulate: D equals A times B plus C. It does not contain a neural network, words, or a complete attention head. Software divides larger tensors into compatible tiles, arranges their layout, and feeds these units repeatedly. Different instructions accept different input formats and accumulation rules. Low precision reduces the bits moved and processed, while scale factors help represent useful numerical ranges. A peak throughput figure assumes suitable operations, shapes, clocks, and a continuous supply of operands. A matrix tile combines row and column elements into partial products, accumulates their sums, and contributes results to a larger matrix. That arithmetic relationship describes the operation, not a literal photograph of a Tensor Core layout.\n\n1.  Software divides a large matrix product into compatible tiles and selects a supported input format, layout, and scaling scheme.\n2.  The tensor instruction multiplies corresponding tile elements and accumulates partial sums according to its numerical rules.\n3.  Partial tiles are combined or reused, then written into the larger output tensor for the next operation.\n\n## Science and assumptions\n\n### Why matrix size and reuse matter\n\n$$\nF\\approx2mnk,\\qquad I_{\\mathrm{ideal}}\\approx\\frac{2mnk}{b(mk+kn+mn)}\n$$\n\nDense matrix work is approximately two times m times n times k. Ideal intensity divides that work by the bytes needed to read both inputs once and write the output once.\n\nSymbols and units\n\nF\n\nDense matrix-product arithmetic count\n\nFLOP\n\nm\n\nRows of A and C\n\nelements\n\nn\n\nColumns of B and C\n\nelements\n\nk\n\nShared reduction dimension\n\nelements\n\nb\n\nStored size per matrix element\n\nbytes/element\n\nI\\_ideal\n\nIntensity when A and B are read once and C written once\n\nFLOP/byte\n\nA has m rows and k columns; B has k rows and n columns. Their product needs approximately mnk fused multiply-adds, conventionally counted as two floating-point operations each. If all elements occupy b bytes, the denominator describes reading A and B once and writing the output once. Increasing reuse can make compute grow faster than external data movement.\n\nAssumptions Ideal dense C = AB traffic, with each matrix transferred once and no initial C read. Accumulation precision can differ in real kernels; a D = AB + C operation must include its extra C traffic. This is an arithmetic model, not measured Tensor Core efficiency.\n\n## Engineering example\n\nFor m×k by k×n, the multiply-accumulate count is 2mnk; operand reuse determines whether the tile reaches a compute bound.\n\n## Distinctions and tradeoffs\n\nTensor Core sparse peak performance is not a dense-model guarantee. The supported sparsity pattern and appropriate instructions are required to realize the sparse rating.\n\nLower precision can raise throughput and reduce memory traffic, but numerical error must be controlled. Sparse peak rates require the supported sparsity pattern and cannot be applied to arbitrary dense weights.\n\nAttention projections and feed-forward layers are dominated by matrix products. Efficient tensor kernels turn this recurring mathematical structure into much more work per joule.\n\n## Specifications and interfaces\n\nOperation\n\nD = A × B + C\n\nMatrix multiply-accumulate, or MMA.\n\nGeneration\n\nFifth-generation Tensor Cores\n\nDatacenter Blackwell; supported instruction details depend on architecture target.\n\nNVFP4 pair peak\n\n40 sparse / 20 dense PFLOPS\n\nOfficial GB200 Superchip table: two GPUs, theoretical tensor peak.\n\n### Interfaces\n\nOperands Input\n\nA and B tiles plus scale metadata\n\nAccumulator Input\n\nC tile\n\nMatrix result Output\n\nD tile\n\n## Check your understanding\n\n### What does a Tensor Core primarily accelerate?\n\n1.  Storing model files after power is removed.\n2.  Matrix multiply-accumulate operations on compatible tiles.\n3.  Replacing the CPU operating system.\n\nAnswer and reasoning\n\nMatrix multiply-accumulate operations on compatible tiles.\n\nNeural-network projections are built from large matrix products. Tensor Cores accelerate small pieces of those products, which software assembles into the complete result.\n\n## Historical context\n\n### 2017 · Volta introduces NVIDIA Tensor Cores\n\nNVIDIA announced Volta and Tesla V100 on May 10, 2017. Dedicated Tensor Cores accelerated matrix arithmetic, aligning silicon execution more directly with the multiply-and-accumulate structure of deep learning instead of relying only on general-purpose arithmetic lanes.\n\n[NVIDIA: Volta GPU platform launch](https://nvidianews.nvidia.com/news/nvidia-launches-revolutionary-volta-gpu-platform-fueling-next-era-of-ai-and-high-performance-computing)\n\n## Evidence and scope\n\nTile dimensions and accumulation precision are configuration choices.\n\nInspect row–column products and accumulated results in a small integer matrix; this does not simulate the individual gate network.\n\n## Sources\n\n-   [NVIDIA CUTLASS: Blackwell specifics](https://docs.nvidia.com/cutlass/4.5.1/media/docs/cpp/blackwell.html)\n-   [NVIDIA GB200 NVL72 specifications](https://www.nvidia.com/en-us/data-center/gb200-nvl72/)\n-   [NVIDIA Matrix Multiplication Background Guide](https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html)\n-   [NVIDIA: Volta GPU platform launch](https://nvidianews.nvidia.com/news/nvidia-launches-revolutionary-volta-gpu-platform-fueling-next-era-of-ai-and-high-performance-computing)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/tensor-core-e72167e744c85d77.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Multiply–accumulate and fused arithmetic](https://semiaifoundry.com/learn/atlas/mac/)\n\n### Related lessons\n\n-   [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/)\n-   [Tensor Memory](https://semiaifoundry.com/learn/atlas/tensor-memory/)\n-   [Shared memory SRAM](https://semiaifoundry.com/learn/atlas/shared-sram/)\n-   [Tensor Memory Accelerator](https://semiaifoundry.com/learn/atlas/tensor-memory-accelerator/)\n-   [Self-attention](https://semiaifoundry.com/learn/atlas/attention/)\n-   [Compiler, intermediate code, and device executable](https://semiaifoundry.com/learn/atlas/compiler/)"
}