# Integrated accelerator rack: GB200 NVL72 example Canonical: https://semiaifoundry.com/learn/atlas/rack/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Rack integration · Atlas v1.3.0 # Integrated accelerator rack: GB200 NVL72 example Integrates compute, communication, power, and cooling into one coordinated physical system. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#rack)Meter-scale rack; GB200 NVL72 reference anatomy Learning objectiveFind a rack bottleneck from power, memory and network ceilings. ## The mechanism A rack becomes an AI system when its components are integrated around shared communication and service infrastructure. The GB200 NVL72 example combines seventy-two Blackwell GPUs and thirty-six Grace CPUs in one rack-scale NVLink domain. Compute trays, switch trays, cable cartridges, power distribution, and cooling each occupy distinct roles. The atlas exposes them as separate but connected layers: operands move through the data fabric, electricity arrives through the power system, and heat leaves through cooling. The rack is not literally one semiconductor chip, despite descriptions of it acting like one large GPU. Software must still partition work, manage memory, and coordinate communication to turn its resources into useful training or inference. 1. Place compute and communication trays within the rack. 2. Connect the trays to shared data, power, and coolant paths. 3. Use distributed software to map a model onto the coordinated resources. ## Science and assumptions ### Communication-aware throughput ceiling $$ R\le\min(P_{\mathrm{peak}},B_{\mathrm{mem}}I_{\mathrm{mem}},B_{\mathrm{net}}I_{\mathrm{net}}) $$ Rack throughput is bounded by arithmetic capacity, memory bandwidth times memory intensity, and network bandwidth times network intensity; the smallest bound applies. Symbols and units R Useful arithmetic throughput under the declared work convention operations/s P\_peak Compatible arithmetic throughput ceiling operations/s B\_mem Useful memory bandwidth bytes/s I\_mem Useful operations per byte crossing the memory boundary operations/byte B\_net Useful network bandwidth bytes/s I\_net Useful operations per byte crossing the network boundary operations/byte $P_{\mathrm{peak}}$ is arithmetic capacity. $I_{\mathrm{mem}}$ and $I_{\mathrm{net}}$ count useful operations per byte crossing the chosen memory and network boundaries. The limiting resource can change with the workload partition. Assumptions Roofline-style explanatory upper bound with consistent operation formats and accounting boundaries; not a prediction of measured rack performance. ## Engineering example The smallest of $P_{\mathrm{peak}}$, $B_{\mathrm{mem}}$ $I_{\mathrm{mem}}$ and $B_{\mathrm{net}}$ $I_{\mathrm{net}}$ bounds the workload-specific rate in the first-order roofline. ## Distinctions and tradeoffs A 72-GPU rack is not one physical GPU die, and its aggregate bandwidth is not a single pairwise connection. Dense integration reduces communication distance. It concentrates power, cooling, and service requirements. The rack is a useful scale for model execution. Its effective output depends on the whole system remaining supplied and coordinated. ## Specifications and interfaces GB200 NVL72 example 72 GPUs + 36 CPUs A documented reference system, not all AI racks. Compute and switch trays 18 + 9 One-rack DGX NVL72 configuration. NVLink aggregate ≈130 TB/s Aggregate rack communication specification; not bandwidth between one GPU pair. ### Interfaces Facility services and jobs Input Power, cooling, network data, and scheduled work Computed results and heat Output Model outputs, exchanged tensors, and thermal load ## Check your understanding ### What makes the rack a usable AI system? 1. The coordinated compute, data, power, cooling, and software layers 2. Its cabinet color 3. GPU count alone Answer and reasoning The coordinated compute, data, power, cooling, and software layers All layers must work together; arithmetic units cannot run useful jobs without data, power, cooling, and coordination. ## Historical context ### 2024 · GB200 NVL72 rack-scale architecture announced The documented example integrates 72 GPUs through a large NVLink domain with purpose-built communication and liquid-cooling infrastructure. It illustrates system-level co-design rather than proving a model capability by itself. [NVIDIA — GB200 NVL72 Architecture, March 2024](https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/) ## Evidence and scope The terms require a common workload and time boundary. Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included. ## Sources - [NVIDIA — GB200 NVL72 Product Specifications](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) - [NVIDIA — DGX GB Rack Scale Systems Hardware Guide](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html) - [Berkeley Lab — Introduction to Roofline](https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/rack-d8e166de597e4f4d.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [Compute baseboard and tray](https://semiaifoundry.com/learn/atlas/compute-baseboard/) - [Rack power shelf and DC busbar](https://semiaifoundry.com/learn/atlas/power-shelf/) ### Related lessons - [GB200 compute tray](https://semiaifoundry.com/learn/atlas/compute-tray/) - [NVLink switch tray and scale-up fabric](https://semiaifoundry.com/learn/atlas/nvlink-switch/) - [NVLink copper cable cartridge](https://semiaifoundry.com/learn/atlas/cable-cartridge/) - [Coolant distribution unit and separated loops](https://semiaifoundry.com/learn/atlas/cdu/) - [Data, tensor, pipeline, and state parallelism](https://semiaifoundry.com/learn/atlas/parallelism/)