{"id":"rack","level":"rack","name":"Integrated accelerator rack: GB200 NVL72 example","shortName":"Accelerator rack","role":"Integrates compute, communication, power, and cooling into one coordinated physical system.","description":"A rack becomes an AI system when its components are integrated around shared communication and service infrastructure. The GB200 NVL72 example combines seventy-two Blackwell GPUs and thirty-six Grace CPUs in one rack-scale NVLink domain. Compute trays, switch trays, cable cartridges, power distribution, and cooling each occupy distinct roles. The atlas exposes them as separate but connected layers: operands move through the data fabric, electricity arrives through the power system, and heat leaves through cooling. The rack is not literally one semiconductor chip, despite descriptions of it acting like one large GPU. Software must still partition work, manage memory, and coordinate communication to turn its resources into useful training or inference.","specs":[{"label":"GB200 NVL72 example","value":"72 GPUs + 36 CPUs","context":"A documented reference system, not all AI racks."},{"label":"Compute and switch trays","value":"18 + 9","context":"One-rack DGX NVL72 configuration."},{"label":"NVLink aggregate","value":"≈130 TB/s","context":"Aggregate rack communication specification; not bandwidth between one GPU pair."}],"interfaces":[{"name":"Facility services and jobs","direction":"in","signal":"Power, cooling, network data, and scheduled work"},{"name":"Computed results and heat","direction":"out","signal":"Model outputs, exchanged tensors, and thermal load"}],"tradeoff":"Dense integration reduces communication distance. It concentrates power, cooling, and service requirements.","whyAI":"The rack is a useful scale for model execution. Its effective output depends on the whole system remaining supplied and coordinated.","related":["compute-tray","nvlink-switch","cable-cartridge","cdu","power-shelf","parallelism"],"sources":[{"title":"NVIDIA — GB200 NVL72 Product Specifications","url":"https://www.nvidia.com/en-us/data-center/gb200-nvl72/"},{"title":"NVIDIA — DGX GB Rack Scale Systems Hardware Guide","url":"https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"},{"title":"Berkeley Lab — Introduction to Roofline","url":"https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf"}],"scale":"Meter-scale rack; GB200 NVL72 reference anatomy","kind":"rack","mechanism":["Place compute and communication trays within the rack.","Connect the trays to shared data, power, and coolant paths.","Use distributed software to map a model onto the coordinated resources."],"misconception":"A 72-GPU rack is not one physical GPU die, and its aggregate bandwidth is not a single pairwise connection.","check":{"question":"What makes the rack a usable AI system?","options":["The coordinated compute, data, power, cooling, and software layers","Its cabinet color","GPU count alone"],"answer":0,"explanation":"All layers must work together; arithmetic units cannot run useful jobs without data, power, cooling, and coordination."},"branch":"Rack integration","subbranch":"Integrated compute","science":[{"title":"Communication-aware throughput ceiling","equation":"R ≤ min(P_peak, B_mem I_mem, B_net I_net)","explanation":"P_peak is arithmetic capacity. I_mem and I_net count useful operations per byte crossing the chosen memory and network boundaries. The limiting resource can change with the workload partition.","assumptions":"Roofline-style explanatory upper bound with consistent operation formats and accounting boundaries; not a prediction of measured rack performance."}],"history":[{"year":2024,"title":"GB200 NVL72 rack-scale architecture announced","significance":"The documented example integrates 72 GPUs through a large NVLink domain with purpose-built communication and liquid-cooling infrastructure. It illustrates system-level co-design rather than proving a model capability by itself.","source":{"title":"NVIDIA — GB200 NVL72 Architecture, March 2024","url":"https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/"}}],"learningObjective":"Find a rack bottleneck from power, memory and network ceilings.","engineeringExample":"The smallest of P_peak, B_mem I_mem and B_net I_net bounds the workload-specific rate in the first-order roofline.","evidenceNotes":"The terms require a common workload and time boundary.","visualFamily":"rack-system","prerequisites":["compute-baseboard","power-shelf"],"labId":"engineering","scope":"Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.","deepLabId":"hall","investigationId":"manufacturing","labScope":"Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included."}