{"id":"tensor-core","level":"die","name":"Tensor Core matrix unit","shortName":"Tensor Core","role":"Multiplies small matrix tiles and accumulates their products with supported numerical formats.","description":"A Tensor Core implements matrix multiply and accumulate: D equals A times B plus C. It does not contain a neural network, words, or a complete attention head. Software divides larger tensors into compatible tiles, arranges their layout, and feeds these units repeatedly. Different instructions accept different input formats and accumulation rules. Low precision reduces the bits moved and processed, while scale factors help represent useful numerical ranges. A peak throughput figure assumes suitable operations, shapes, clocks, and a continuous supply of operands. A matrix tile combines row and column elements into partial products, accumulates their sums, and contributes results to a larger matrix. That arithmetic relationship describes the operation, not a literal photograph of a Tensor Core layout.","specs":[{"label":"Operation","value":"D = A × B + C","context":"Matrix multiply-accumulate, or MMA."},{"label":"Generation","value":"Fifth-generation Tensor Cores","context":"Datacenter Blackwell; supported instruction details depend on architecture target."},{"label":"NVFP4 pair peak","value":"40 sparse / 20 dense PFLOPS","context":"Official GB200 Superchip table: two GPUs, theoretical tensor peak."}],"interfaces":[{"name":"Operands","direction":"in","signal":"A and B tiles plus scale metadata"},{"name":"Accumulator","direction":"in","signal":"C tile"},{"name":"Matrix result","direction":"out","signal":"D tile"}],"tradeoff":"Lower precision can raise throughput and reduce memory traffic, but numerical error must be controlled. Sparse peak rates require the supported sparsity pattern and cannot be applied to arbitrary dense weights.","whyAI":"Attention projections and feed-forward layers are dominated by matrix products. Efficient tensor kernels turn this recurring mathematical structure into much more work per joule.","related":["streaming-multiprocessor","tensor-memory","shared-sram","tensor-memory-accelerator","mac","attention","compiler"],"sources":[{"title":"NVIDIA CUTLASS: Blackwell specifics","url":"https://docs.nvidia.com/cutlass/4.5.1/media/docs/cpp/blackwell.html"},{"title":"NVIDIA GB200 NVL72 specifications","url":"https://www.nvidia.com/en-us/data-center/gb200-nvl72/"},{"title":"NVIDIA Matrix Multiplication Background Guide","url":"https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html"},{"title":"NVIDIA: Volta GPU platform launch","url":"https://nvidianews.nvidia.com/news/nvidia-launches-revolutionary-volta-gpu-platform-fueling-next-era-of-ai-and-high-performance-computing"}],"scale":"Subregion of an SM; logical matrix tiles enlarged for inspection","kind":"matrix-multiply-tiles","mechanism":["Software divides a large matrix product into compatible tiles and selects a supported input format, layout, and scaling scheme.","The tensor instruction multiplies corresponding tile elements and accumulates partial sums according to its numerical rules.","Partial tiles are combined or reused, then written into the larger output tensor for the next operation."],"misconception":"Tensor Core sparse peak performance is not a dense-model guarantee. The supported sparsity pattern and appropriate instructions are required to realize the sparse rating.","check":{"question":"What does a Tensor Core primarily accelerate?","options":["Storing model files after power is removed.","Matrix multiply-accumulate operations on compatible tiles.","Replacing the CPU operating system."],"answer":1,"explanation":"Neural-network projections are built from large matrix products. Tensor Cores accelerate small pieces of those products, which software assembles into the complete result."},"branch":"Compute architecture","subbranch":"Execution engines","science":[{"title":"Why matrix size and reuse matter","equation":"F ≈ 2mnk; Iideal ≈ 2mnk / [b(mk + kn + mn)]","explanation":"A has m rows and k columns; B has k rows and n columns. Their product needs approximately mnk fused multiply-adds, conventionally counted as two floating-point operations each. If all elements occupy b bytes, the denominator describes reading A and B once and writing the output once. Increasing reuse can make compute grow faster than external data movement.","assumptions":"Ideal dense C = AB traffic, with each matrix transferred once and no initial C read. Accumulation precision can differ in real kernels; a D = AB + C operation must include its extra C traffic. This is an arithmetic model, not measured Tensor Core efficiency."}],"history":[{"year":2017,"title":"Volta introduces NVIDIA Tensor Cores","significance":"NVIDIA announced Volta and Tesla V100 on May 10, 2017. Dedicated Tensor Cores accelerated matrix arithmetic, aligning silicon execution more directly with the multiply-and-accumulate structure of deep learning instead of relying only on general-purpose arithmetic lanes.","source":{"title":"NVIDIA: Volta GPU platform launch","url":"https://nvidianews.nvidia.com/news/nvidia-launches-revolutionary-volta-gpu-platform-fueling-next-era-of-ai-and-high-performance-computing"}}],"learningObjective":"Translate a matrix tile into operations and data movement.","engineeringExample":"For m×k by k×n, the multiply-accumulate count is 2mnk; operand reuse determines whether the tile reaches a compute bound.","evidenceNotes":"Tile dimensions and accumulation precision are configuration choices.","visualFamily":"architecture-array","prerequisites":["mac"],"labId":"matrix","scope":"Inspect row–column products and accumulated results in a small integer matrix; this does not simulate the individual gate network.","investigationId":"switching","labScope":"Inspect row–column products and accumulated results in a small integer matrix; this does not simulate the individual gate network."}