{
  "schemaVersion": 1,
  "id": "atlas-lesson-systolic-array-dataflow",
  "title": "Systolic arrays and tensor dataflow",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Reuses operands through a regular network of multiply–accumulate elements.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/systolic-array-dataflow/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Processing architectures",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "TPU generations are named separately. This teaching array is not a transistor-accurate reconstruction or a cross-vendor peak comparison.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/systolic-array-dataflow-8d5f972ce8b5a1fc.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#systolic-array-dataflow",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "b7e06f41b4191868602473d14a2bc2dca5a391de93cc870be4c1cbaa33cdec1a",
  "contentLength": 5200,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-systolic-array-dataflow.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-systolic-array-dataflow.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Systolic arrays and tensor dataflow",
      "start": 41,
      "end": 406
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 406,
      "end": 1641
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1641,
      "end": 1669
    },
    {
      "id": "section-4",
      "title": "Dense matrix work",
      "start": 1669,
      "end": 2339
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2339,
      "end": 2612
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2612,
      "end": 3096
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3096,
      "end": 3385
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3385,
      "end": 3542
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3542,
      "end": 3571
    },
    {
      "id": "section-10",
      "title": "What chiefly allows an array to reduce distant memory traffic?",
      "start": 3571,
      "end": 3969
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3969,
      "end": 4315
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4315,
      "end": 4783
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4783,
      "end": 4806
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4806,
      "end": 4992
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 4992,
      "end": 5200
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#systolic-array-dataflow"
    },
    {
      "title": "In-Datacenter Performance Analysis of a Tensor Processing Unit",
      "url": "https://arxiv.org/abs/1704.04760"
    },
    {
      "title": "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings",
      "url": "https://arxiv.org/abs/2304.01433v3"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/systolic-array-dataflow-8d5f972ce8b5a1fc.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Multiply–accumulate and fused arithmetic",
      "url": "https://semiaifoundry.com/learn/atlas/mac/"
    },
    {
      "title": "Shared memory SRAM",
      "url": "https://semiaifoundry.com/learn/atlas/shared-sram/"
    },
    {
      "title": "Tensor Core matrix unit",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-core/"
    },
    {
      "title": "Trainium and Inferentia execution paths",
      "url": "https://semiaifoundry.com/learn/atlas/aws-neuron-dataflow/"
    }
  ],
  "recordHash": "293ae2774f5797cbead74aa75160527dc9603d2ccad701debc5a4388d45c6eaa",
  "contentFormat": "text/markdown",
  "content": "Processing architectures · Atlas v1.3.0\n\n# Systolic arrays and tensor dataflow\n\nReuses operands through a regular network of multiply–accumulate elements.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#systolic-array-dataflow)Processing element to accelerator package; array tiles exaggerated\n\nLearning objectiveRelate a matrix tiling and dataflow choice to reuse, filling and array utilization.\n\n## The mechanism\n\nA systolic array moves operands rhythmically between neighboring processing elements. Instead of repeatedly fetching every pair from a distant memory, each value contributes to several multiply–accumulate operations as it crosses the array. A dataflow choice determines whether weights, inputs or partial sums remain stationary for a tile. Filling and draining the array creates overhead, especially for small or awkward matrix shapes. Google’s original TPU is a disclosed matrix-engine example; TPU v4 adds system and embedding support. The array is only part of the machine: nonlinear operations, sparse lookups, compiler scheduling and communication still require other resources.\n\n1.  Tile the matrix multiplication so operands and partial sums fit the target buffers and array shape.\n2.  Skew the entering streams so each processing element receives the matching operands at the matching cycle.\n3.  Multiply and accumulate while forwarding reusable operands to neighboring elements.\n4.  Drain completed partial sums, accounting for edge tiles and bubbles during startup or shape changes.\n5.  Schedule vector, lookup and communication operations around the array so non-matrix work does not strand its capacity.\n\n## Science and assumptions\n\n### Dense matrix work\n\n$$\nF\\approx2mnk\n$$\n\nDense matrix multiplication takes approximately two times output rows times output columns times reduction width floating-point operations.\n\nSymbols and units\n\nF\n\nDense matrix-product arithmetic work\n\nFLOP\n\nm\n\nRows in the left matrix and output\n\nelements\n\nn\n\nColumns in the right matrix and output\n\nelements\n\nk\n\nReduction dimension shared by the inputs\n\nelements\n\nFor A of shape m×k and B of shape k×n, F counts multiply and add separately. Dimensions count elements, and F counts arithmetic operations.\n\nAssumptions Large dense matrices; exact edge counting differs slightly and sparse execution needs a separate convention.\n\n## Engineering example\n\nSynthetic multiplication of two 4×4 matrices contains 64 products and approximately 128 operations by the two-operation MAC convention. An animation should track one operand’s reuse rather than pretend that every product requires a fresh HBM read.\n\n## Distinctions and tradeoffs\n\nA tensor accelerator need not be one universal systolic design, and optical links in TPU v4 do not mean its arithmetic is optical.\n\nRegular local traffic can be efficient for dense matrices but mismatched shapes and irregular indexing demand padding, remapping or another engine.\n\nTransformer projections and feed-forward layers contain reusable matrix work, while embeddings and token-dependent control expose the limits of a matrix-only explanation.\n\n## Specifications and interfaces\n\nOperation convention\n\nOne multiply plus one add = two operations\n\nUse this convention only when explicitly comparing compatible arithmetic.\n\nTile utilization\n\nUseful array cells and cycles\n\nShape padding and fill/drain cycles reduce realized throughput.\n\n### Interfaces\n\nTile buffers Input\n\nOrdered operand streams and scheduling metadata\n\nAccumulator or result buffer Output\n\nPartial or completed matrix tiles\n\n## Check your understanding\n\n### What chiefly allows an array to reduce distant memory traffic?\n\n1.  Operand reuse through local connections\n2.  Ignoring matrix dependencies\n3.  Using the largest possible advertised FLOPS number\n\nAnswer and reasoning\n\nOperand reuse through local connections\n\nValues can participate in several products after entering the array, avoiding repeated fetches from the selected outer memory level.\n\n## Evidence and scope\n\nTPU generations are named separately. This teaching array is not a transistor-accurate reconstruction or a cross-vendor peak comparison.\n\nCompare unblocked and tiled matrix traffic for fixed arithmetic; architecture-specific instructions, fusion and attention algorithms are context rather than simulated implementations.\n\n## Sources\n\n-   [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://arxiv.org/abs/1704.04760)\n-   [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://arxiv.org/abs/2304.01433v3)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/systolic-array-dataflow-8d5f972ce8b5a1fc.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Multiply–accumulate and fused arithmetic](https://semiaifoundry.com/learn/atlas/mac/)\n-   [Shared memory SRAM](https://semiaifoundry.com/learn/atlas/shared-sram/)\n\n### Related lessons\n\n-   [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/)\n-   [Trainium and Inferentia execution paths](https://semiaifoundry.com/learn/atlas/aws-neuron-dataflow/)"
}