{"id":"grace-blackwell-superchip","level":"server","name":"Grace Blackwell Superchip","shortName":"GB200 module","role":"Couples one Grace CPU with two Blackwell GPUs so host and accelerator work can share a tightly integrated system.","description":"The Grace Blackwell Superchip is a system integration unit, not one monolithic piece of silicon. It combines one CPU and two GPUs, with separate CPU memory and GPU HBM connected through a coherent interconnect. The GPUs also participate in a larger NVLink domain when installed in an NVL72 rack. This creates several nested scales: transistors form dies, dies and HBM form GPU packages, packages join the CPU in a module, and modules join a rack. The GPU’s internal die-to-die link, CPU–GPU coherence path and GPU-to-GPU scale-up communication serve different boundaries and cannot be treated as one interchangeable connection. This published reference explains a modern architecture; it is not evidence about the hardware used by any specific proprietary model service.","specs":[{"label":"Processor composition","value":"1 Grace CPU + 2 Blackwell GPUs","context":"GB200 Superchip product configuration."},{"label":"GPU memory","value":"372 GB HBM3E, 16 TB/s aggregate","context":"Current official pair specification; distinct from B200 standalone variants."},{"label":"CPU–GPU link","value":"900 GB/s raw bidirectional NVLink-C2C","context":"Coherent interconnect, distinct from NVLink 5 and the internal GPU die link."}],"interfaces":[{"name":"Host preparation","direction":"in","signal":"Programs and CPU-resident data"},{"name":"Coherent exchange","direction":"out","signal":"CPU–GPU memory and control traffic"},{"name":"Rack fabric","direction":"out","signal":"GPU tensors moving through NVLink scale-up links"}],"tradeoff":"Tight integration can improve data movement and energy efficiency but ties the system to a particular memory and interconnect arrangement. Software must still place work and data carefully across the component boundaries.","whyAI":"This module joins general-purpose control with specialized matrix computation. Replicating and connecting it creates a larger platform for training and serving advanced models.","related":["host-cpu","gpu-die","hbm-stack","lpddr-memory","nvlink-port","compute-baseboard"],"sources":[{"title":"NVIDIA GB200 NVL72 specifications","url":"https://www.nvidia.com/en-us/data-center/gb200-nvl72/"},{"title":"NVIDIA Blackwell platform announcement","url":"https://nvidianews.nvidia.com/news/nvidia-blackwell-platform-arrives-to-power-a-new-era-of-computing"},{"title":"NVIDIA: Inside Grace CPU architecture","url":"https://developer.nvidia.com/blog/inside-nvidia-grace-cpu-nvidia-amps-up-superchip-engineering-for-hpc-and-ai/"},{"title":"NVIDIA CUDA Programming Guide: asynchronous data copies","url":"https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/async-copies.html"},{"title":"Intel Advisor: Amdahl’s Law and measurement","url":"https://www.intel.com/content/www/us/en/docs/advisor/user-guide/2024-2/use-amdahl-law.html"}],"scale":"Module-scale integration across several packages; schematic proportions","kind":"nested-superchip-system","mechanism":["One Grace CPU prepares and coordinates work while its LPDDR memory holds host-side state.","Two Blackwell GPUs compute tensor operations using their local HBM, exchanging required information with the CPU over NVLink-C2C.","The GPUs connect into a larger NVLink scale-up domain, allowing multiple modules to cooperate on workloads that exceed one module."],"misconception":"Superchip is a system-integration name, not proof of one monolithic die. Its CPU, GPUs, memories, and several different interconnects remain distinct physical components.","check":{"question":"Which description correctly separates the nested links?","options":["GPU internal die links, CPU–GPU C2C, and GPU scale-up NVLink serve different boundaries.","Every link in the module is the same kind of TSV.","The Ethernet NIC is the internal bridge between the two GPU logic dies."],"answer":0,"explanation":"Different physical boundaries require different interfaces. Keeping these layers distinct explains how chips become modules and modules become larger machines."},"branch":"System integration","subbranch":"Boards and modules","science":[{"title":"Offload is useful only when saved compute exceeds movement cost","equation":"TGPU-path ≈ F/PGPU + Min/Bin + Mout/Bout + Tlaunch","explanation":"F is operation count; PGPU is attained throughput; Min and Mout are transferred bytes; B values are effective one-direction path bandwidths. Compare the sum with the CPU execution time for the same operation. Reusing data already in HBM can remove repeated transfer terms, while coherent remote access still pays the relevant movement cost.","assumptions":"A sequential non-overlapped teaching model for a fixed workload and precision. Actual kernels may overlap transfers, hit caches, share memory coherently, or contend for bandwidth. Attained throughput and effective bandwidth must be measured rather than replaced blindly with product peaks."}],"history":[{"year":2024,"title":"Blackwell extends acceleration from multiple dies to rack-scale systems","significance":"Announced on March 18, 2024, Blackwell joined two large logic dies within one GPU and paired new numerical formats with stronger GPU interconnect. Grace Blackwell systems brought CPU, GPU memory, scale-up networking, power, and cooling into a more tightly coordinated architecture; no specific proprietary model’s deployment is implied.","source":{"title":"NVIDIA: Blackwell platform announcement","url":"https://nvidianews.nvidia.com/news/nvidia-blackwell-platform-arrives-to-power-a-new-era-of-computing"}}],"prerequisites":["host-cpu","gpu-die","lpddr-memory"],"learningObjective":"Budget compute, transfers and launch overhead for a CPU–GPU workload at a declared execution boundary.","engineeringExample":"Synthetic non-overlapped job: 2 TFLOP at an attained 100 TFLOP/s takes 20 ms; 1 GB in and 0.5 GB out at 100 GB/s take 10 and 5 ms. Adding 0.1 ms dispatch gives 35.1 ms. This example uses invented attained rates, not product peaks.","evidenceNotes":"The linked module sources establish CPU, GPU and memory boundaries. The calculation assumes serial phases and fixed precision; overlap, coherent accesses, caches and contention require a different execution model.","labId":"token-system-path","scope":"Follow a modeled batched decode critical path and energy budget; prefill, expert routing and scheduling differences are comparison context rather than separately simulated algorithms.","investigationId":"execution","labScope":"Follow a modeled batched decode critical path and energy budget; prefill, expert routing and scheduling differences are comparison context rather than separately simulated algorithms."}