{
  "schemaVersion": 1,
  "id": "atlas-lesson-nvlink-switch",
  "title": "NVLink switch tray and scale-up fabric",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Routes high-bandwidth GPU communication inside the rack-scale NVLink domain.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/nvlink-switch/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Rack integration",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The transfer bound uses decimal payload bytes and one specified direction. Values are synthetic; routing, competing flows, collective algorithms and measured effective bandwidth determine the applicable path.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-switch-9a4ae0fb9887a117.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#nvlink-switch",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "971c393ef286095c42737fd246cad34eec33def4cb5d54f0ddd2033945856631",
  "contentLength": 4969,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-nvlink-switch.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-nvlink-switch.md",
  "sections": [
    {
      "id": "section-1",
      "title": "NVLink switch tray and scale-up fabric",
      "start": 33,
      "end": 391
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 391,
      "end": 1375
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1375,
      "end": 1403
    },
    {
      "id": "section-4",
      "title": "Transfer-time floor",
      "start": 1403,
      "end": 2097
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2097,
      "end": 2346
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2346,
      "end": 2755
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 2755,
      "end": 3023
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3023,
      "end": 3157
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3157,
      "end": 3186
    },
    {
      "id": "section-10",
      "title": "What does the NVLink switch primarily provide?",
      "start": 3186,
      "end": 3504
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3504,
      "end": 3887
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 3887,
      "end": 4371
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4371,
      "end": 4394
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4394,
      "end": 4491
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 4491,
      "end": 4969
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#nvlink-switch"
    },
    {
      "title": "NVIDIA — DGX GB Rack Scale Systems Hardware Guide",
      "url": "https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"
    },
    {
      "title": "NVIDIA — GB200 NVL72 Architecture",
      "url": "https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-switch-9a4ae0fb9887a117.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "NVLink GPU interface",
      "url": "https://semiaifoundry.com/learn/atlas/nvlink-port/"
    },
    {
      "title": "Integrated accelerator rack: GB200 NVL72 example",
      "url": "https://semiaifoundry.com/learn/atlas/rack/"
    },
    {
      "title": "GB200 compute tray",
      "url": "https://semiaifoundry.com/learn/atlas/compute-tray/"
    },
    {
      "title": "NVLink copper cable cartridge",
      "url": "https://semiaifoundry.com/learn/atlas/cable-cartridge/"
    },
    {
      "title": "Collective communication library",
      "url": "https://semiaifoundry.com/learn/atlas/nccl/"
    },
    {
      "title": "Data, tensor, pipeline, and state parallelism",
      "url": "https://semiaifoundry.com/learn/atlas/parallelism/"
    }
  ],
  "recordHash": "e736f108789a2f09a8adf290ff083b807b11bb22d1a5e9b8866c82a9f7cdffd3",
  "contentFormat": "text/markdown",
  "content": "Rack integration · Atlas v1.3.0\n\n# NVLink switch tray and scale-up fabric\n\nRoutes high-bandwidth GPU communication inside the rack-scale NVLink domain.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#nvlink-switch)1U switch tray and rack-wide logical fabric\n\nLearning objectiveBound a specified transfer using its effective path bandwidth rather than the rack’s aggregate headline.\n\n## The mechanism\n\nAn NVLink switch connects GPU communication paths so a model partition can exchange tensors with peers beyond its own tray. In the GB200 NVL72 rack example, nine switch trays collectively connect the seventy-two GPUs; each switch tray contains two NVSwitch ASICs. This scale-up fabric differs from the Ethernet or InfiniBand network used to join larger groups of systems. The atlas separates endpoint links from switch internals and shows several simultaneous routes, conveying switching rather than a single central wire. A switch provides transport and supported communication capabilities, not a complete model execution plan. The software layer must choose which tensors move, when they move, and which peers participate in each operation.\n\n1.  Receive traffic from GPU endpoints through the rack cabling.\n2.  Select supported routes through the switch fabric.\n3.  Deliver each transfer to the intended peer while software coordinates the collective operation.\n\n## Science and assumptions\n\n### Transfer-time floor\n\n$$\nt_{\\mathrm{transfer}}\\ge\\frac{S}{B_{\\mathrm{effective}}}\n$$\n\nTransfer time is at least payload size divided by the sustained bandwidth available to that transfer.\n\nSymbols and units\n\nt\\_transfer\n\nTransfer completion time\n\ns\n\nS\n\nPayload bytes moved\n\nbytes\n\nB\\_effective\n\nSustained bandwidth usable by that transfer\n\nbytes/s\n\nMoving S bytes takes at least their size divided by sustained path bandwidth. Synchronization and protocol costs add latency, and traffic sharing can reduce effective bandwidth.\n\nAssumptions One specified transfer with bandwidth measured in bytes per second; aggregate endpoint or rack bandwidth is not automatically usable by one transfer.\n\n## Engineering example\n\nSynthetic transfer: sending 4 GB over a path sustaining 200 GB/s takes at least 4/200 = 0.02 s = 20 ms. Doubling the sum of unrelated rack ports leaves this bound unchanged if that transfer still has the same 200 GB/s path.\n\n## Distinctions and tradeoffs\n\nNVLink switching inside the rack and scale-out Ethernet or InfiniBand are different fabrics with different roles.\n\nA larger fast domain can reduce communication overhead. Switches, cabling, and endpoint limits still constrain traffic.\n\nMany model partitions exchange intermediate results frequently. Fast scale-up communication helps those partitions function as one workload.\n\n## Specifications and interfaces\n\nGB200 NVL72 trays\n\n9\n\nDocumented rack configuration.\n\nASICs per switch tray\n\n2 NVSwitches\n\nHardware guide example.\n\nFifth-generation GPU endpoint\n\n1.8 TB/s bidirectional\n\nPer-GPU aggregate NVLink throughput, not per individual link.\n\n### Interfaces\n\nGPU NVLink ports Input\n\nTensor and peer-communication traffic\n\nPeer GPU paths Output\n\nSwitched scale-up connectivity\n\n## Check your understanding\n\n### What does the NVLink switch primarily provide?\n\n1.  Permanent model-weight storage\n2.  Electrical power conversion\n3.  Fast routed GPU connectivity\n\nAnswer and reasoning\n\nFast routed GPU connectivity\n\nIt connects communication paths. Model placement and communication scheduling remain software responsibilities.\n\n## Evidence and scope\n\nThe transfer bound uses decimal payload bytes and one specified direction. Values are synthetic; routing, competing flows, collective algorithms and measured effective bandwidth determine the applicable path.\n\nVary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.\n\n## Sources\n\n-   [NVIDIA — DGX GB Rack Scale Systems Hardware Guide](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html)\n-   [NVIDIA — GB200 NVL72 Architecture](https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-switch-9a4ae0fb9887a117.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [NVLink GPU interface](https://semiaifoundry.com/learn/atlas/nvlink-port/)\n\n### Related lessons\n\n-   [Integrated accelerator rack: GB200 NVL72 example](https://semiaifoundry.com/learn/atlas/rack/)\n-   [GB200 compute tray](https://semiaifoundry.com/learn/atlas/compute-tray/)\n-   [NVLink copper cable cartridge](https://semiaifoundry.com/learn/atlas/cable-cartridge/)\n-   [Collective communication library](https://semiaifoundry.com/learn/atlas/nccl/)\n-   [Data, tensor, pipeline, and state parallelism](https://semiaifoundry.com/learn/atlas/parallelism/)"
}