{
  "schemaVersion": 1,
  "id": "atlas-lesson-tensor-index-contracts",
  "title": "Tensor shapes, indices and contractions",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Specifies what each axis means before the same numbers are scheduled or stored on hardware.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/tensor-index-contracts/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "mathematical-foundations",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-index-contracts-2c14febbab1621be.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#tensor-index-contracts",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "36e423b707b3ea221bb9de018038453f72f81d1111923e080bda95605ec84ef3",
  "contentLength": 6304,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-index-contracts.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-tensor-index-contracts.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Tensor shapes, indices and contractions",
      "start": 41,
      "end": 431
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 431,
      "end": 1723
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1723,
      "end": 1751
    },
    {
      "id": "section-4",
      "title": "Tensor shapes, indices and contractions",
      "start": 1751,
      "end": 2594
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2594,
      "end": 2933
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2933,
      "end": 3455
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3455,
      "end": 4059
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4059,
      "end": 4382
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4382,
      "end": 4411
    },
    {
      "id": "section-10",
      "title": "Which change leaves C = AB semantically unchanged?",
      "start": 4411,
      "end": 5081
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 5081,
      "end": 5572
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5572,
      "end": 5905
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5905,
      "end": 5928
    },
    {
      "id": "section-14",
      "title": "Related lessons",
      "start": 5928,
      "end": 6304
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#tensor-index-contracts"
    },
    {
      "title": "Goodfellow, Bengio and Courville · Linear algebra",
      "url": "https://www.deeplearningbook.org/contents/linear_algebra.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/tensor-index-contracts-2c14febbab1621be.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Embedding table",
      "url": "https://semiaifoundry.com/learn/atlas/embedding/"
    },
    {
      "title": "Multiply–accumulate and fused arithmetic",
      "url": "https://semiaifoundry.com/learn/atlas/mac/"
    },
    {
      "title": "Self-attention",
      "url": "https://semiaifoundry.com/learn/atlas/attention/"
    },
    {
      "title": "Sharded matrix multiplication and result ownership",
      "url": "https://semiaifoundry.com/learn/atlas/sharded-matmul-ownership/"
    }
  ],
  "recordHash": "d0c75d37d544fb25053e475c55ea4a630118a61850cae0eef8aad5c4afba511f",
  "contentFormat": "text/markdown",
  "content": "mathematical-foundations · Atlas v1.3.0\n\n# Tensor shapes, indices and contractions\n\nSpecifies what each axis means before the same numbers are scheduled or stored on hardware.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#tensor-index-contracts)Mathematical and data space; dimensions are not physical distance\n\nLearning objectiveTrace every scalar contribution into one output and detect an incorrect contraction axis.\n\n## The mechanism\n\nA tensor is an indexed collection of values. Shape records axis lengths; it does not by itself record which axis means token, feature, batch or head. A contraction pairs chosen axes and sums their products. The unsummed axes survive in the output. Matrix multiplication is one such contraction: output row i and column j identify a sum over the shared index k. A transpose, a physical memory reorder and a reinterpretation of an axis are different operations. A shape-compatible program can still compute the wrong function when its axis meanings are mistaken. The experiment keeps explicit indices beside each operand and partial sum so that arithmetic and layout can later be connected without losing the original contract.\n\n1.  Name each input axis and check that contracted dimensions agree. A tensor with two rows and two columns can still represent different semantic objects.\n2.  Choose an output coordinate. Fetch matching A\\[i,k\\] and B\\[k,j\\], multiply them, and add the product to that output’s accumulator.\n3.  Advance k until every contribution is included, then move to another output. Keep the final result separate from a partial sum.\n4.  Compare a separate dense reference and preserve index meaning when a compiler changes placement, tiling or ownership.\n\n## Science and assumptions\n\n### Tensor shapes, indices and contractions\n\n$$\nC_{ij}=\\sum_{k=0}^{K-1}A_{ik}B_{kj}\n$$\n\nEach output entry sums the matching scalar products along the contracted axis k.\n\nSymbols and units\n\nAᵢₖ,Bₖⱼ\n\nIndexed input elements\n\nvalue\n\nCᵢⱼ\n\nCompleted output element\n\nvalue\n\ni,j,k\n\nRow, column and reduction indices\n\nindex\n\nK\n\nShared dimension\n\ncount\n\nWith A=\\[\\[1,2\\],\\[3,4\\]\\] and B=\\[\\[5,6\\],\\[7,8\\]\\], C\\[0,0\\]=1×5+2×7=19 and C=\\[\\[19,22\\],\\[43,50\\]\\]. Changing A\\[0,1\\] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.\n\nAssumptions Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.\n\n## Engineering example\n\nWith A=\\[\\[1,2\\],\\[3,4\\]\\] and B=\\[\\[5,6\\],\\[7,8\\]\\], C\\[0,0\\]=1×5+2×7=19 and C=\\[\\[19,22\\],\\[43,50\\]\\]. Changing A\\[0,1\\] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.\n\n## Distinctions and tradeoffs\n\nA tensor’s axes are not automatically physical coordinates, and reshaping an array does not generally transpose its values.\n\nA layout that improves locality must preserve axis semantics. Materializing a transpose can cost traffic; changing a label alone cannot perform its mathematical effect.\n\nAttention projections, gradient contractions and distributed sharding depend on the same index discipline. Correct dimensions are necessary but insufficient evidence of correct tensor semantics.\n\n## Specifications and interfaces\n\nTeaching specimen\n\nTensor contracts\n\nWith A=\\[\\[1,2\\],\\[3,4\\]\\] and B=\\[\\[5,6\\],\\[7,8\\]\\], C\\[0,0\\]=1×5+2×7=19 and C=\\[\\[19,22\\],\\[43,50\\]\\]. Changing A\\[0,1\\] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.\n\nEvidence boundary\n\nExplicit computed example\n\nFinite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.\n\n### Interfaces\n\nInputs and convention Input\n\nName each input axis and check that contracted dimensions agree. A tensor with two rows and two columns can still represent different semantic objects.\n\nComputed result and scope Output\n\nTrace every scalar contribution into one output and detect an incorrect contraction axis.\n\n## Check your understanding\n\n### Which change leaves C = AB semantically unchanged?\n\n1.  Changing storage layout while preserving the indexed values consumed by the contraction\n2.  Relabeling any two equal-length axes without checking their meaning\n3.  Concatenating partial sums instead of adding matching output coordinates\n\nAnswer and reasoning\n\nChanging storage layout while preserving the indexed values consumed by the contraction\n\nChanging storage layout while preserving the indexed values consumed by the contraction. Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.\n\n## Evidence and scope\n\nThe workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.\n\nOpen Mathematical foundations for the bounded executable mechanism and its record. General numerical labs are an additional reference library.\n\n## Sources\n\n-   [Goodfellow, Bengio and Courville · Linear algebra](https://www.deeplearningbook.org/contents/linear_algebra.html)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/tensor-index-contracts-2c14febbab1621be.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Related lessons\n\n-   [Embedding table](https://semiaifoundry.com/learn/atlas/embedding/)\n-   [Multiply–accumulate and fused arithmetic](https://semiaifoundry.com/learn/atlas/mac/)\n-   [Self-attention](https://semiaifoundry.com/learn/atlas/attention/)\n-   [Sharded matrix multiplication and result ownership](https://semiaifoundry.com/learn/atlas/sharded-matmul-ownership/)"
}