{
  "schemaVersion": 1,
  "id": "atlas-lesson-rack",
  "title": "Integrated accelerator rack: GB200 NVL72 example",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Integrates compute, communication, power, and cooling into one coordinated physical system.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/rack/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Rack integration",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The terms require a common workload and time boundary.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/rack-d8e166de597e4f4d.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#rack",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "a471d6a93488d62c719eb753e115db0f198150a88e92c62eb86f829db2301da4",
  "contentLength": 5940,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-rack.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-rack.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Integrated accelerator rack: GB200 NVL72 example",
      "start": 33,
      "end": 370
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 370,
      "end": 1350
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1350,
      "end": 1378
    },
    {
      "id": "section-4",
      "title": "Communication-aware throughput ceiling",
      "start": 1378,
      "end": 2500
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2500,
      "end": 2705
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2705,
      "end": 3088
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3088,
      "end": 3391
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3391,
      "end": 3577
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3577,
      "end": 3606
    },
    {
      "id": "section-10",
      "title": "What makes the rack a usable AI system?",
      "start": 3606,
      "end": 3975
    },
    {
      "id": "section-11",
      "title": "Historical context",
      "start": 3975,
      "end": 3998
    },
    {
      "id": "section-12",
      "title": "2024 · GB200 NVL72 rack-scale architecture announced",
      "start": 3998,
      "end": 4454
    },
    {
      "id": "section-13",
      "title": "Evidence and scope",
      "start": 4454,
      "end": 4701
    },
    {
      "id": "section-14",
      "title": "Sources",
      "start": 4701,
      "end": 5230
    },
    {
      "id": "section-15",
      "title": "Continue exploring",
      "start": 5230,
      "end": 5253
    },
    {
      "id": "section-16",
      "title": "Foundations",
      "start": 5253,
      "end": 5451
    },
    {
      "id": "section-17",
      "title": "Related lessons",
      "start": 5451,
      "end": 5940
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#rack"
    },
    {
      "title": "NVIDIA — GB200 NVL72 Architecture, March 2024",
      "url": "https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/"
    },
    {
      "title": "NVIDIA — GB200 NVL72 Product Specifications",
      "url": "https://www.nvidia.com/en-us/data-center/gb200-nvl72/"
    },
    {
      "title": "NVIDIA — DGX GB Rack Scale Systems Hardware Guide",
      "url": "https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"
    },
    {
      "title": "Berkeley Lab — Introduction to Roofline",
      "url": "https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/rack-d8e166de597e4f4d.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Compute baseboard and tray",
      "url": "https://semiaifoundry.com/learn/atlas/compute-baseboard/"
    },
    {
      "title": "Rack power shelf and DC busbar",
      "url": "https://semiaifoundry.com/learn/atlas/power-shelf/"
    },
    {
      "title": "GB200 compute tray",
      "url": "https://semiaifoundry.com/learn/atlas/compute-tray/"
    },
    {
      "title": "NVLink switch tray and scale-up fabric",
      "url": "https://semiaifoundry.com/learn/atlas/nvlink-switch/"
    },
    {
      "title": "NVLink copper cable cartridge",
      "url": "https://semiaifoundry.com/learn/atlas/cable-cartridge/"
    },
    {
      "title": "Coolant distribution unit and separated loops",
      "url": "https://semiaifoundry.com/learn/atlas/cdu/"
    },
    {
      "title": "Data, tensor, pipeline, and state parallelism",
      "url": "https://semiaifoundry.com/learn/atlas/parallelism/"
    }
  ],
  "recordHash": "a132aadfebb63c81a319ea4de7bf0bf06ca067ae37d9397f8e05a2a13ac35641",
  "contentFormat": "text/markdown",
  "content": "Rack integration · Atlas v1.3.0\n\n# Integrated accelerator rack: GB200 NVL72 example\n\nIntegrates compute, communication, power, and cooling into one coordinated physical system.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#rack)Meter-scale rack; GB200 NVL72 reference anatomy\n\nLearning objectiveFind a rack bottleneck from power, memory and network ceilings.\n\n## The mechanism\n\nA rack becomes an AI system when its components are integrated around shared communication and service infrastructure. The GB200 NVL72 example combines seventy-two Blackwell GPUs and thirty-six Grace CPUs in one rack-scale NVLink domain. Compute trays, switch trays, cable cartridges, power distribution, and cooling each occupy distinct roles. The atlas exposes them as separate but connected layers: operands move through the data fabric, electricity arrives through the power system, and heat leaves through cooling. The rack is not literally one semiconductor chip, despite descriptions of it acting like one large GPU. Software must still partition work, manage memory, and coordinate communication to turn its resources into useful training or inference.\n\n1.  Place compute and communication trays within the rack.\n2.  Connect the trays to shared data, power, and coolant paths.\n3.  Use distributed software to map a model onto the coordinated resources.\n\n## Science and assumptions\n\n### Communication-aware throughput ceiling\n\n$$\nR\\le\\min(P_{\\mathrm{peak}},B_{\\mathrm{mem}}I_{\\mathrm{mem}},B_{\\mathrm{net}}I_{\\mathrm{net}})\n$$\n\nRack throughput is bounded by arithmetic capacity, memory bandwidth times memory intensity, and network bandwidth times network intensity; the smallest bound applies.\n\nSymbols and units\n\nR\n\nUseful arithmetic throughput under the declared work convention\n\noperations/s\n\nP\\_peak\n\nCompatible arithmetic throughput ceiling\n\noperations/s\n\nB\\_mem\n\nUseful memory bandwidth\n\nbytes/s\n\nI\\_mem\n\nUseful operations per byte crossing the memory boundary\n\noperations/byte\n\nB\\_net\n\nUseful network bandwidth\n\nbytes/s\n\nI\\_net\n\nUseful operations per byte crossing the network boundary\n\noperations/byte\n\n$P_{\\mathrm{peak}}$ is arithmetic capacity. $I_{\\mathrm{mem}}$ and $I_{\\mathrm{net}}$ count useful operations per byte crossing the chosen memory and network boundaries. The limiting resource can change with the workload partition.\n\nAssumptions Roofline-style explanatory upper bound with consistent operation formats and accounting boundaries; not a prediction of measured rack performance.\n\n## Engineering example\n\nThe smallest of $P_{\\mathrm{peak}}$, $B_{\\mathrm{mem}}$ $I_{\\mathrm{mem}}$ and $B_{\\mathrm{net}}$ $I_{\\mathrm{net}}$ bounds the workload-specific rate in the first-order roofline.\n\n## Distinctions and tradeoffs\n\nA 72-GPU rack is not one physical GPU die, and its aggregate bandwidth is not a single pairwise connection.\n\nDense integration reduces communication distance. It concentrates power, cooling, and service requirements.\n\nThe rack is a useful scale for model execution. Its effective output depends on the whole system remaining supplied and coordinated.\n\n## Specifications and interfaces\n\nGB200 NVL72 example\n\n72 GPUs + 36 CPUs\n\nA documented reference system, not all AI racks.\n\nCompute and switch trays\n\n18 + 9\n\nOne-rack DGX NVL72 configuration.\n\nNVLink aggregate\n\n≈130 TB/s\n\nAggregate rack communication specification; not bandwidth between one GPU pair.\n\n### Interfaces\n\nFacility services and jobs Input\n\nPower, cooling, network data, and scheduled work\n\nComputed results and heat Output\n\nModel outputs, exchanged tensors, and thermal load\n\n## Check your understanding\n\n### What makes the rack a usable AI system?\n\n1.  The coordinated compute, data, power, cooling, and software layers\n2.  Its cabinet color\n3.  GPU count alone\n\nAnswer and reasoning\n\nThe coordinated compute, data, power, cooling, and software layers\n\nAll layers must work together; arithmetic units cannot run useful jobs without data, power, cooling, and coordination.\n\n## Historical context\n\n### 2024 · GB200 NVL72 rack-scale architecture announced\n\nThe documented example integrates 72 GPUs through a large NVLink domain with purpose-built communication and liquid-cooling infrastructure. It illustrates system-level co-design rather than proving a model capability by itself.\n\n[NVIDIA — GB200 NVL72 Architecture, March 2024](https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/)\n\n## Evidence and scope\n\nThe terms require a common workload and time boundary.\n\nPropagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.\n\n## Sources\n\n-   [NVIDIA — GB200 NVL72 Product Specifications](https://www.nvidia.com/en-us/data-center/gb200-nvl72/)\n-   [NVIDIA — DGX GB Rack Scale Systems Hardware Guide](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html)\n-   [Berkeley Lab — Introduction to Roofline](https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/rack-d8e166de597e4f4d.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Compute baseboard and tray](https://semiaifoundry.com/learn/atlas/compute-baseboard/)\n-   [Rack power shelf and DC busbar](https://semiaifoundry.com/learn/atlas/power-shelf/)\n\n### Related lessons\n\n-   [GB200 compute tray](https://semiaifoundry.com/learn/atlas/compute-tray/)\n-   [NVLink switch tray and scale-up fabric](https://semiaifoundry.com/learn/atlas/nvlink-switch/)\n-   [NVLink copper cable cartridge](https://semiaifoundry.com/learn/atlas/cable-cartridge/)\n-   [Coolant distribution unit and separated loops](https://semiaifoundry.com/learn/atlas/cdu/)\n-   [Data, tensor, pipeline, and state parallelism](https://semiaifoundry.com/learn/atlas/parallelism/)"
}