The mechanism
The Grace Blackwell Superchip is a system integration unit, not one monolithic piece of silicon. It combines one CPU and two GPUs, with separate CPU memory and GPU HBM connected through a coherent interconnect. The GPUs also participate in a larger NVLink domain when installed in an NVL72 rack. This creates several nested scales: transistors form dies, dies and HBM form GPU packages, packages join the CPU in a module, and modules join a rack. The GPU’s internal die-to-die link, CPU–GPU coherence path and GPU-to-GPU scale-up communication serve different boundaries and cannot be treated as one interchangeable connection. This published reference explains a modern architecture; it is not evidence about the hardware used by any specific proprietary model service.
- One Grace CPU prepares and coordinates work while its LPDDR memory holds host-side state.
- Two Blackwell GPUs compute tensor operations using their local HBM, exchanging required information with the CPU over NVLink-C2C.
- The GPUs connect into a larger NVLink scale-up domain, allowing multiple modules to cooperate on workloads that exceed one module.
Science and assumptions
Offload is useful only when saved compute exceeds movement cost
Sequential GPU-path time is arithmetic work divided by attained throughput, plus incoming transfer time, outgoing transfer time and launch overhead.
Symbols and units
- T_GPU path
- Sequential GPU execution-and-transfer time
s
- F
- Arithmetic work for the specified operation and format
operations
- P_GPU
- Attained arithmetic throughput
operations/s
- M_in
- Bytes transferred into the device
bytes
- B_in
- Effective incoming path bandwidth
bytes/s
- M_out
- Bytes transferred out of the device
bytes
- B_out
- Effective outgoing path bandwidth
bytes/s
- T_launch
- Exposed launch overhead
s
F is operation count; PGPU is attained throughput; Min and Mout are transferred bytes; B values are effective one-direction path bandwidths. Compare the sum with the CPU execution time for the same operation. Reusing data already in HBM can remove repeated transfer terms, while coherent remote access still pays the relevant movement cost.
Assumptions A sequential non-overlapped teaching model for a fixed workload and precision. Actual kernels may overlap transfers, hit caches, share memory coherently, or contend for bandwidth. Attained throughput and effective bandwidth must be measured rather than replaced blindly with product peaks.
Engineering example
Synthetic non-overlapped job: 2 TFLOP at an attained 100 TFLOP/s takes 20 ms; 1 GB in and 0.5 GB out at 100 GB/s take 10 and 5 ms. Adding 0.1 ms dispatch gives 35.1 ms. This example uses invented attained rates, not product peaks.
Distinctions and tradeoffs
Superchip is a system-integration name, not proof of one monolithic die. Its CPU, GPUs, memories, and several different interconnects remain distinct physical components.
Tight integration can improve data movement and energy efficiency but ties the system to a particular memory and interconnect arrangement. Software must still place work and data carefully across the component boundaries.
This module joins general-purpose control with specialized matrix computation. Replicating and connecting it creates a larger platform for training and serving advanced models.
Specifications and interfaces
- Processor composition
- 1 Grace CPU + 2 Blackwell GPUs
GB200 Superchip product configuration.
- GPU memory
- 372 GB HBM3E, 16 TB/s aggregate
Current official pair specification; distinct from B200 standalone variants.
- CPU–GPU link
- 900 GB/s raw bidirectional NVLink-C2C
Coherent interconnect, distinct from NVLink 5 and the internal GPU die link.
Interfaces
- Host preparation Input
- Programs and CPU-resident data
- Coherent exchange Output
- CPU–GPU memory and control traffic
- Rack fabric Output
- GPU tensors moving through NVLink scale-up links
Check your understanding
Which description correctly separates the nested links?
- GPU internal die links, CPU–GPU C2C, and GPU scale-up NVLink serve different boundaries.
- Every link in the module is the same kind of TSV.
- The Ethernet NIC is the internal bridge between the two GPU logic dies.
Answer and reasoning
GPU internal die links, CPU–GPU C2C, and GPU scale-up NVLink serve different boundaries.
Different physical boundaries require different interfaces. Keeping these layers distinct explains how chips become modules and modules become larger machines.
Historical context
2024 · Blackwell extends acceleration from multiple dies to rack-scale systems
Announced on March 18, 2024, Blackwell joined two large logic dies within one GPU and paired new numerical formats with stronger GPU interconnect. Grace Blackwell systems brought CPU, GPU memory, scale-up networking, power, and cooling into a more tightly coordinated architecture; no specific proprietary model’s deployment is implied.
Evidence and scope
The linked module sources establish CPU, GPU and memory boundaries. The calculation assumes serial phases and fixed precision; overlap, coherent accesses, caches and contention require a different execution model.
Follow a modeled batched decode critical path and energy budget; prefill, expert routing and scheduling differences are comparison context rather than separately simulated algorithms.