{
  "schemaVersion": 1,
  "id": "atlas-lesson-cpu-vector-matrix",
  "title": "Scalar, vector and CPU matrix units",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Executes different shapes of arithmetic within a general-purpose processor.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/cpu-vector-matrix/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Processing architectures",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Intel and Arm examples illustrate programming mechanisms. They do not establish equal precision support or a measured performance comparison.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/cpu-vector-matrix-e144948c627c0f3e.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#cpu-vector-matrix",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "9b61120e1f7dcec78fb64f517b22ac3663a1cf53339d84bb8d95b0d476e58530",
  "contentLength": 4983,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-cpu-vector-matrix.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-cpu-vector-matrix.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Scalar, vector and CPU matrix units",
      "start": 41,
      "end": 400
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 400,
      "end": 1656
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1656,
      "end": 1684
    },
    {
      "id": "section-4",
      "title": "Lane utilization",
      "start": 1684,
      "end": 2262
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2262,
      "end": 2485
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2485,
      "end": 2930
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 2930,
      "end": 3198
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3198,
      "end": 3373
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3373,
      "end": 3402
    },
    {
      "id": "section-10",
      "title": "A vector width is eight elements and the loop contains ten elements. What limits useful-lane utilization in two issued groups?",
      "start": 3402,
      "end": 3870
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3870,
      "end": 4180
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4180,
      "end": 4660
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4660,
      "end": 4683
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4683,
      "end": 4881
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 4881,
      "end": 4983
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#cpu-vector-matrix"
    },
    {
      "title": "What Is Intel Advanced Matrix Extensions?",
      "url": "https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/what-is-intel-amx.html"
    },
    {
      "title": "Arm Neoverse V2 Core Technical Reference Manual",
      "url": "https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/cpu-vector-matrix-e144948c627c0f3e.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Multiply–accumulate and fused arithmetic",
      "url": "https://semiaifoundry.com/learn/atlas/mac/"
    },
    {
      "title": "Out-of-order CPU pipeline",
      "url": "https://semiaifoundry.com/learn/atlas/cpu-out-of-order/"
    },
    {
      "title": "Tensor Core matrix unit",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-core/"
    }
  ],
  "recordHash": "aacf2021353caff6572616a6d596ed7653bc7d8a4e7ffb6ec276d3ed4a748e28",
  "contentFormat": "text/markdown",
  "content": "Processing architectures · Atlas v1.3.0\n\n# Scalar, vector and CPU matrix units\n\nExecutes different shapes of arithmetic within a general-purpose processor.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#cpu-vector-matrix)Core to multi-socket server; diagrams are logical\n\nLearning objectiveExplain why scalar, vector and matrix execution can coexist and require different data preparation.\n\n## The mechanism\n\nA scalar instruction updates one value; a vector instruction applies an operation across lanes; a matrix instruction consumes a tile containing many values. These are execution resources inside a CPU, not mutually exclusive kinds of computer. Intel AMX exposes tile storage and tile matrix multiplication, while Arm SVE lets software express vector work with predicates and a vector-length-aware model. Loads, address calculation and branches still surround the arithmetic. Efficient code arranges contiguous data, handles tails and chooses supported types so that wider arithmetic produces useful results instead of multiplying padding or waiting for operands.\n\n1.  Separate loop control and address calculations from the arithmetic repeated across data elements.\n2.  Group independent elements into vector lanes and mask the final partial group rather than reading invalid data.\n3.  For a matrix operation, load tiles that permit each operand to contribute to multiple accumulations.\n4.  Select instructions whose input format, accumulator format and rounding satisfy the model’s numerical requirements.\n5.  Measure complete loop time, including layout conversion and memory access, rather than timing only the arithmetic instruction.\n\n## Science and assumptions\n\n### Lane utilization\n\n$$\nu=\\frac{n}{w\\lceil n/w\\rceil}\n$$\n\nTail-packing utilization is useful elements divided by vector width times the number of vector instructions rounded up.\n\nSymbols and units\n\nu\n\nTail-packing lane utilization\n\nfraction\n\nn\n\nUseful elements processed, greater than zero\n\nelements\n\nw\n\nFixed vector-lane width\n\nelements/instruction\n\nn is useful elements and w is lanes per instruction; u has no units. It measures tail packing, not all causes of CPU utilization loss.\n\nAssumptions One operation per element with fixed vector width and no additional masking.\n\n## Engineering example\n\nSynthetic loop: ten independent additions on an eight-lane unit need two groups, giving 10/16 = 62.5% useful lane positions. A larger batch reduces this tail fraction without changing vector width.\n\n## Distinctions and tradeoffs\n\nA CPU with matrix instructions remains a CPU; adding a specialized unit does not remove its control pipeline or memory hierarchy.\n\nWider units increase arithmetic throughput but raise operand demand and can leave lanes idle on short or irregular arrays.\n\nCPU vector and matrix capabilities can accelerate preprocessing, embedding reductions and smaller inference batches while preserving general-purpose control.\n\n## Specifications and interfaces\n\nVector occupancy\n\nUseful lanes / issued lanes\n\nDimensionless utilization for a particular instruction.\n\nArithmetic type\n\nInput and accumulator specified separately\n\nA low-precision input does not imply an equally narrow accumulator.\n\n### Interfaces\n\nCache and load units Input\n\nContiguous or gathered operand vectors and tiles\n\nRegister or tile results Output\n\nPartial sums consumed by subsequent operations\n\n## Check your understanding\n\n### A vector width is eight elements and the loop contains ten elements. What limits useful-lane utilization in two issued groups?\n\n1.  Ten useful lanes out of sixteen issued lanes\n2.  All sixteen lanes produce valid outputs\n3.  Only the first eight elements can be computed\n\nAnswer and reasoning\n\nTen useful lanes out of sixteen issued lanes\n\nA masked tail can compute the remaining two elements safely, but six issued lane positions perform no useful element work.\n\n## Evidence and scope\n\nIntel and Arm examples illustrate programming mechanisms. They do not establish equal precision support or a measured performance comparison.\n\nNo current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.\n\n## Sources\n\n-   [What Is Intel Advanced Matrix Extensions?](https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/what-is-intel-amx.html)\n-   [Arm Neoverse V2 Core Technical Reference Manual](https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/cpu-vector-matrix-e144948c627c0f3e.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Multiply–accumulate and fused arithmetic](https://semiaifoundry.com/learn/atlas/mac/)\n-   [Out-of-order CPU pipeline](https://semiaifoundry.com/learn/atlas/cpu-out-of-order/)\n\n### Related lessons\n\n-   [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/)"
}