# GPU chiplets and memory locality

Canonical: https://semiaifoundry.com/learn/atlas/gpu-chiplet-locality/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Processing architectures · Atlas v1.3.0

# GPU chiplets and memory locality

Partitions compute and I/O physically while presenting a programmable accelerator.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#gpu-chiplet-locality)GPU execution group to package; schematic topology

Learning objectiveRead physical chiplet boundaries independently from the software device model.

## The mechanism

Breaking an accelerator into chiplets changes where data must travel. In AMD’s MI300 family, compute chiplets connect through I/O dies and a package-level fabric to memory. MI300X is a GPU accelerator, whereas MI300A combines CPU and GPU compute with shared HBM in an APU package. A software-visible address space can hide the physical route without making its bandwidth or contention disappear. Chiplets can place different circuit functions on suitable processes and reuse design blocks, but introduce additional interfaces, test requirements and package assembly dependencies. A package diagram therefore needs both physical boundaries and logical memory connections.

1.  Distinguish a logical accelerator device from the separate compute, I/O and memory dies inside its package.
2.  Map a tensor partition onto compute resources and identify the cache and memory path it will use.
3.  Account for data reused locally versus data repeatedly crossing the package fabric.
4.  Compare CPU/GPU shared-memory access in an APU with explicit host/device placement in a discrete accelerator system.
5.  Validate the mapping with target-specific tools rather than assuming that a shared address makes every route equivalent.

## Science and assumptions

### Cut bandwidth bound

$$
T\ge\frac{Q_{\mathrm{cut}}}{B_{\mathrm{cut}}}
$$

Transfer demand across a chiplet topology cut takes at least crossing bytes divided by usable cut bandwidth.

Symbols and units

T

Minimum time demanded by the selected topology cut

s

Q\_cut

Payload crossing that cut

bytes

B\_cut

Sustainable directional capacity usable across it

bytes/s

Qcut is bytes crossing a chosen internal topology cut and Bcut is sustainable bytes/s across that cut; T is seconds.

Assumptions Traffic is measured for that boundary; do not sum link nameplate rates that cannot all serve the same cut.

## Engineering example

Synthetic mapping: 12 GB crossing a 3 TB/s internal cut has a 4 ms transfer lower bound. Local reuse that halves crossing traffic halves this bound, even if arithmetic count stays constant.

## Distinctions and tradeoffs

A chiplet package is not automatically a cluster of independent GPUs, and unified memory does not imply uniform physical access costs.

Smaller dies can improve reuse and manufacturing flexibility; the additional links, packaging and distributed caches require careful bandwidth and energy budgets.

An operator placement decision can increase traffic across internal fabrics before any inter-server communication occurs.

## Specifications and interfaces

Comparison boundary

One package

An accelerator package may contain multiple logic dies and memory stacks.

Locality

Compute-to-cache-to-memory path

Physical routes remain relevant even under a unified address model.

### Interfaces

Compute chiplet Input

Cache misses and fabric requests

I/O die and HBM Output

Data responses and inter-chiplet transactions

## Check your understanding

### Why can two kernels with equal HBM traffic differ on a chiplet accelerator?

1.  They may create different internal-fabric traffic and locality
2.  HBM traffic uniquely determines runtime
3.  Every chiplet has an isolated programming language

Answer and reasoning

They may create different internal-fabric traffic and locality

Internal placement, cache behavior and fabric contention can differ even when total off-package memory bytes match.

## Evidence and scope

The MI300 family is a disclosed historical specimen. Public topology is sufficient for the mechanism; undocumented routing policy and measured locality penalties remain unknown.

Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.

## Sources

-   [AMD Instinct MI300 Series microarchitecture](https://rocm.docs.amd.com/en/latest/reference/gpu-arch/mi300.html)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/gpu-chiplet-locality-19bb68acbe4aa11e.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [GPU logic dies](https://semiaifoundry.com/learn/atlas/gpu-die/)
-   [On-chip interconnect fabric](https://semiaifoundry.com/learn/atlas/on-chip-network/)

### Related lessons

-   [Custom-chip volume economics](https://semiaifoundry.com/learn/atlas/chip-volume-economics/)