The mechanism
Lowering makes an abstract tensor operation progressively more concrete. The compiler chooses layouts, partitions work, selects library calls or generated kernels and plans when storage can be reused. A transpose can sometimes be represented by changing indexing, while another consumer requires a physical data rearrangement. Tile shapes must fit local resources and supported arithmetic instructions. Quantized values additionally need scales, zero points or conversions preserved through transformations. OpenXLA’s GPU architecture documentation supplies one representative route. Compiler decisions therefore affect both arithmetic efficiency and bytes moved, and inspecting the optimized representation often explains a surprising performance result.
A layout choice is meaningful when the indexed values, storage ownership and next consumer are named together. A transpose may be a view with changed strides or a materialized reorder; the latter consumes bytes and time that the former does not automatically incur. A schedule that improves locality must retain the operator’s axis semantics and account for any conversion needed by its consumer.
- Select a physical layout compatible with the dominant consumers and the target memory transaction rules.
- Partition large operations into legal tiles or shards and select the corresponding hardware or library path.
- Insert materialized layout conversions only where indexing transformations cannot satisfy a consumer.
- Assign storage according to tensor lifetimes, aliasing rules and the desired execution overlap.
- Generate target code and inspect the plan for unnecessary copies, unsupported operations and excessive temporary allocation.
- Logical coordinate-to-value meaning survives layout changes even when physical addresses differ.
- The next consumer determines whether a strided view is acceptable or a contiguous materialization and synchronization are needed.
Science and assumptions
Dense stride mapping
The element’s byte address is base address plus bytes per element times the sum of each logical index times its element stride.
Symbols and units
- address(i,j)
- Byte address of the element at logical indices i,j
byte address
- base
- Byte address of the logical origin
byte address
- s
- Bytes per element
bytes/element
- i,j
- Valid row/column indices
integer indices
- stride₀,stride₁
- Element offset per one-index step on each axis
elements/index step
s is bytes/element and strides count elements; the expression yields a byte address for logical indices i,j.
Assumptions Dense strided storage with valid bounds; tiled or compressed layouts require a richer mapping.
Engineering example
Synthetic 4×8 row-major tensor has strides (8,1). Swapping logical axes can produce an 8×4 view with strides (1,8), avoiding a copy until a consumer demands contiguous rows.
A row-major 2×3 array [[1,2,3],[4,5,6]] has element strides (3,1). Its 3×2 transpose view uses strides (1,3) and logically yields [[1,4],[2,5],[3,6]]. A separate contiguous two-byte representation of that result contains 12 payload bytes. If it is materialized by reading and writing each value once, the chosen local memory boundary sees 24 payload bytes, not merely the 12-byte output size.
Distinctions and tradeoffs
A transpose symbol does not always imply a full copy, while a mathematically simple operation can require substantial physical movement.
A layout that favors one operator may require expensive conversion for the next; aggressive storage reuse can restrict overlap between otherwise independent operations.
Compiler choices can turn an apparently small model edit into extra copies, recompilation or an unsupported slow path.
Specifications and interfaces
- Layout
- Index-to-address mapping
Logical dimensions and physical contiguous dimension can differ.
- Buffer lifetime
- Producer completion to final consumer
Storage reuse requires non-overlapping live ranges or correct aliasing semantics.
Interfaces
- Typed graph Input
- Tensor semantics, shapes and dependencies
- Lowered executable plan Output
- Layouts, buffer assignments, kernels and calls
Check your understanding
When can two temporary tensors safely share a buffer?
- When their required live ranges do not overlap under the execution schedule
- Whenever their names are different
- Only if all kernels run on the CPU
Answer and reasoning
When their required live ranges do not overlap under the execution schedule
The memory may be reused after the last consumer has finished, subject to aliasing and asynchronous execution rules.
What is the payload traffic for reading and writing six two-byte elements once during a transpose copy?
- 12 bytes
- 24 bytes
- Six bytes
Answer and reasoning
24 bytes
Six elements × two bytes × one read and one write is 24 payload bytes at that boundary.
Evidence and scope
OpenXLA is a representative compiler route. Exact passes, emitter choices and executable formats depend on software revision and target architecture.
These are logical address and payload counts for a synthetic array. They exclude cache effects, allocator overhead and measured compiler/device performance.
No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.