{"id":"nvlink-switch","level":"rack","name":"NVLink switch tray and scale-up fabric","shortName":"NVLink switch","role":"Routes high-bandwidth GPU communication inside the rack-scale NVLink domain.","description":"An NVLink switch connects GPU communication paths so a model partition can exchange tensors with peers beyond its own tray. In the GB200 NVL72 rack example, nine switch trays collectively connect the seventy-two GPUs; each switch tray contains two NVSwitch ASICs. This scale-up fabric differs from the Ethernet or InfiniBand network used to join larger groups of systems. The atlas separates endpoint links from switch internals and shows several simultaneous routes, conveying switching rather than a single central wire. A switch provides transport and supported communication capabilities, not a complete model execution plan. The software layer must choose which tensors move, when they move, and which peers participate in each operation.","specs":[{"label":"GB200 NVL72 trays","value":"9","context":"Documented rack configuration."},{"label":"ASICs per switch tray","value":"2 NVSwitches","context":"Hardware guide example."},{"label":"Fifth-generation GPU endpoint","value":"1.8 TB/s bidirectional","context":"Per-GPU aggregate NVLink throughput, not per individual link."}],"interfaces":[{"name":"GPU NVLink ports","direction":"in","signal":"Tensor and peer-communication traffic"},{"name":"Peer GPU paths","direction":"out","signal":"Switched scale-up connectivity"}],"tradeoff":"A larger fast domain can reduce communication overhead. Switches, cabling, and endpoint limits still constrain traffic.","whyAI":"Many model partitions exchange intermediate results frequently. Fast scale-up communication helps those partitions function as one workload.","related":["rack","compute-tray","cable-cartridge","nccl","parallelism"],"sources":[{"title":"NVIDIA — DGX GB Rack Scale Systems Hardware Guide","url":"https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"},{"title":"NVIDIA — GB200 NVL72 Architecture","url":"https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/"}],"scale":"1U switch tray and rack-wide logical fabric","kind":"nvlink-switch","mechanism":["Receive traffic from GPU endpoints through the rack cabling.","Select supported routes through the switch fabric.","Deliver each transfer to the intended peer while software coordinates the collective operation."],"misconception":"NVLink switching inside the rack and scale-out Ethernet or InfiniBand are different fabrics with different roles.","check":{"question":"What does the NVLink switch primarily provide?","options":["Permanent model-weight storage","Electrical power conversion","Fast routed GPU connectivity"],"answer":2,"explanation":"It connects communication paths. Model placement and communication scheduling remain software responsibilities."},"branch":"Rack integration","subbranch":"Scale-up fabric","science":[{"title":"Transfer-time floor","equation":"t_transfer ≥ S / B_effective","explanation":"Moving S bytes takes at least their size divided by sustained path bandwidth. Synchronization and protocol costs add latency, and traffic sharing can reduce effective bandwidth.","assumptions":"One specified transfer with bandwidth measured in bytes per second; aggregate endpoint or rack bandwidth is not automatically usable by one transfer."}],"history":[],"prerequisites":["nvlink-port"],"learningObjective":"Bound a specified transfer using its effective path bandwidth rather than the rack’s aggregate headline.","engineeringExample":"Synthetic transfer: sending 4 GB over a path sustaining 200 GB/s takes at least 4/200 = 0.02 s = 20 ms. Doubling the sum of unrelated rack ports leaves this bound unchanged if that transfer still has the same 200 GB/s path.","evidenceNotes":"The transfer bound uses decimal payload bytes and one specified direction. Values are synthetic; routing, competing flows, collective algorithms and measured effective bandwidth determine the applicable path.","labId":"engineering","scope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.","deepLabId":"network","investigationId":"execution","labScope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model."}