semiAIfoundry

mathematical-foundations · Atlas v1.3.0

Tensor shapes, indices and contractions

Specifies what each axis means before the same numbers are scheduled or stored on hardware.

Explore in AtlasMathematical and data space; dimensions are not physical distance

Learning objectiveTrace every scalar contribution into one output and detect an incorrect contraction axis.

The mechanism

A tensor is an indexed collection of values. Shape records axis lengths; it does not by itself record which axis means token, feature, batch or head. A contraction pairs chosen axes and sums their products. The unsummed axes survive in the output. Matrix multiplication is one such contraction: output row i and column j identify a sum over the shared index k. A transpose, a physical memory reorder and a reinterpretation of an axis are different operations. A shape-compatible program can still compute the wrong function when its axis meanings are mistaken. The experiment keeps explicit indices beside each operand and partial sum so that arithmetic and layout can later be connected without losing the original contract.

  1. Name each input axis and check that contracted dimensions agree. A tensor with two rows and two columns can still represent different semantic objects.
  2. Choose an output coordinate. Fetch matching A[i,k] and B[k,j], multiply them, and add the product to that output’s accumulator.
  3. Advance k until every contribution is included, then move to another output. Keep the final result separate from a partial sum.
  4. Compare a separate dense reference and preserve index meaning when a compiler changes placement, tiling or ownership.

Science and assumptions

Tensor shapes, indices and contractions

Cij=∑k=0K−1AikBkjC_{ij}=\sum_{k=0}^{K-1}A_{ik}B_{kj}

Each output entry sums the matching scalar products along the contracted axis k.

Symbols and units
Aᵢₖ,Bₖⱼ
Indexed input elements

value

Cᵢⱼ
Completed output element

value

i,j,k
Row, column and reduction indices

index

K
Shared dimension

count

With A=[[1,2],[3,4]] and B=[[5,6],[7,8]], C[0,0]=1×5+2×7=19 and C=[[19,22],[43,50]]. Changing A[0,1] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.

Assumptions Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.

Engineering example

With A=[[1,2],[3,4]] and B=[[5,6],[7,8]], C[0,0]=1×5+2×7=19 and C=[[19,22],[43,50]]. Changing A[0,1] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.

Distinctions and tradeoffs

A tensor’s axes are not automatically physical coordinates, and reshaping an array does not generally transpose its values.

A layout that improves locality must preserve axis semantics. Materializing a transpose can cost traffic; changing a label alone cannot perform its mathematical effect.

Attention projections, gradient contractions and distributed sharding depend on the same index discipline. Correct dimensions are necessary but insufficient evidence of correct tensor semantics.

Specifications and interfaces

Teaching specimen
Tensor contracts

With A=[[1,2],[3,4]] and B=[[5,6],[7,8]], C[0,0]=1×5+2×7=19 and C=[[19,22],[43,50]]. Changing A[0,1] changes the first output row while leaving the second row unchanged. Eight scalar products appear in the trace. These are exact small integer operations, not eight measured GPU instructions.

Evidence boundary
Explicit computed example

Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.

Interfaces

Inputs and convention Input
Name each input axis and check that contracted dimensions agree. A tensor with two rows and two columns can still represent different semantic objects.
Computed result and scope Output
Trace every scalar contribution into one output and detect an incorrect contraction axis.

Check your understanding

Which change leaves C = AB semantically unchanged?

  1. Changing storage layout while preserving the indexed values consumed by the contraction
  2. Relabeling any two equal-length axes without checking their meaning
  3. Concatenating partial sums instead of adding matching output coordinates
Answer and reasoning

Changing storage layout while preserving the indexed values consumed by the contraction

Changing storage layout while preserving the indexed values consumed by the contraction. Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.

Evidence and scope

The workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Finite rectangular arrays, matching contracted dimension and a declared arithmetic type. The workbench uses small integers; rounded reduction order is studied separately.

Open Mathematical foundations for the bounded executable mechanism and its record. General numerical labs are an additional reference library.

Sources

Published lesson data · Atlas releases and licensing

Continue exploring

Related lessons