{"id":"gpu-chiplet-locality","level":"compute","name":"GPU chiplets and memory locality","shortName":"GPU chiplets and memory locality","role":"Partitions compute and I/O physically while presenting a programmable accelerator.","description":"Breaking an accelerator into chiplets changes where data must travel. In AMD’s MI300 family, compute chiplets connect through I/O dies and a package-level fabric to memory. MI300X is a GPU accelerator, whereas MI300A combines CPU and GPU compute with shared HBM in an APU package. A software-visible address space can hide the physical route without making its bandwidth or contention disappear. Chiplets can place different circuit functions on suitable processes and reuse design blocks, but introduce additional interfaces, test requirements and package assembly dependencies. A package diagram therefore needs both physical boundaries and logical memory connections.","specs":[{"label":"Comparison boundary","value":"One package","context":"An accelerator package may contain multiple logic dies and memory stacks."},{"label":"Locality","value":"Compute-to-cache-to-memory path","context":"Physical routes remain relevant even under a unified address model."}],"interfaces":[{"name":"Compute chiplet","direction":"in","signal":"Cache misses and fabric requests"},{"name":"I/O die and HBM","direction":"out","signal":"Data responses and inter-chiplet transactions"}],"tradeoff":"Smaller dies can improve reuse and manufacturing flexibility; the additional links, packaging and distributed caches require careful bandwidth and energy budgets.","whyAI":"An operator placement decision can increase traffic across internal fabrics before any inter-server communication occurs.","related":["gpu-die","on-chip-network","chip-volume-economics"],"sources":[{"title":"AMD Instinct MI300 Series microarchitecture","url":"https://rocm.docs.amd.com/en/latest/reference/gpu-arch/mi300.html"}],"scale":"GPU execution group to package; schematic topology","kind":"architecture","branch":"Processing architectures","subbranch":"GPU execution and placement","mechanism":["Distinguish a logical accelerator device from the separate compute, I/O and memory dies inside its package.","Map a tensor partition onto compute resources and identify the cache and memory path it will use.","Account for data reused locally versus data repeatedly crossing the package fabric.","Compare CPU/GPU shared-memory access in an APU with explicit host/device placement in a discrete accelerator system.","Validate the mapping with target-specific tools rather than assuming that a shared address makes every route equivalent."],"misconception":"A chiplet package is not automatically a cluster of independent GPUs, and unified memory does not imply uniform physical access costs.","check":{"question":"Why can two kernels with equal HBM traffic differ on a chiplet accelerator?","options":["They may create different internal-fabric traffic and locality","HBM traffic uniquely determines runtime","Every chiplet has an isolated programming language"],"answer":0,"explanation":"Internal placement, cache behavior and fabric contention can differ even when total off-package memory bytes match."},"science":[{"title":"Cut bandwidth bound","equation":"T ≥ Qcut / Bcut","explanation":"Qcut is bytes crossing a chosen internal topology cut and Bcut is sustainable bytes/s across that cut; T is seconds.","assumptions":"Traffic is measured for that boundary; do not sum link nameplate rates that cannot all serve the same cut."}],"prerequisites":["gpu-die","on-chip-network"],"learningObjective":"Read physical chiplet boundaries independently from the software device model.","workstream":"W07","visualFamily":"architecture-gpu","engineeringExample":"Synthetic mapping: 12 GB crossing a 3 TB/s internal cut has a 4 ms transfer lower bound. Local reuse that halves crossing traffic halves this bound, even if arithmetic count stays constant.","evidenceNotes":"The MI300 family is a disclosed historical specimen. Public topology is sufficient for the mechanism; undocumented routing policy and measured locality penalties remain unknown.","labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}