The mechanism
An NVLink switch connects GPU communication paths so a model partition can exchange tensors with peers beyond its own tray. In the GB200 NVL72 rack example, nine switch trays collectively connect the seventy-two GPUs; each switch tray contains two NVSwitch ASICs. This scale-up fabric differs from the Ethernet or InfiniBand network used to join larger groups of systems. The atlas separates endpoint links from switch internals and shows several simultaneous routes, conveying switching rather than a single central wire. A switch provides transport and supported communication capabilities, not a complete model execution plan. The software layer must choose which tensors move, when they move, and which peers participate in each operation.
- Receive traffic from GPU endpoints through the rack cabling.
- Select supported routes through the switch fabric.
- Deliver each transfer to the intended peer while software coordinates the collective operation.
Science and assumptions
Transfer-time floor
Transfer time is at least payload size divided by the sustained bandwidth available to that transfer.
Symbols and units
- t_transfer
- Transfer completion time
s
- S
- Payload bytes moved
bytes
- B_effective
- Sustained bandwidth usable by that transfer
bytes/s
Moving S bytes takes at least their size divided by sustained path bandwidth. Synchronization and protocol costs add latency, and traffic sharing can reduce effective bandwidth.
Assumptions One specified transfer with bandwidth measured in bytes per second; aggregate endpoint or rack bandwidth is not automatically usable by one transfer.
Engineering example
Synthetic transfer: sending 4 GB over a path sustaining 200 GB/s takes at least 4/200 = 0.02 s = 20 ms. Doubling the sum of unrelated rack ports leaves this bound unchanged if that transfer still has the same 200 GB/s path.
Distinctions and tradeoffs
NVLink switching inside the rack and scale-out Ethernet or InfiniBand are different fabrics with different roles.
A larger fast domain can reduce communication overhead. Switches, cabling, and endpoint limits still constrain traffic.
Many model partitions exchange intermediate results frequently. Fast scale-up communication helps those partitions function as one workload.
Specifications and interfaces
- GB200 NVL72 trays
- 9
Documented rack configuration.
- ASICs per switch tray
- 2 NVSwitches
Hardware guide example.
- Fifth-generation GPU endpoint
- 1.8 TB/s bidirectional
Per-GPU aggregate NVLink throughput, not per individual link.
Interfaces
- GPU NVLink ports Input
- Tensor and peer-communication traffic
- Peer GPU paths Output
- Switched scale-up connectivity
Check your understanding
What does the NVLink switch primarily provide?
- Permanent model-weight storage
- Electrical power conversion
- Fast routed GPU connectivity
Answer and reasoning
Fast routed GPU connectivity
It connects communication paths. Model placement and communication scheduling remain software responsibilities.
Evidence and scope
The transfer bound uses decimal payload bytes and one specified direction. Values are synthetic; routing, competing flows, collective algorithms and measured effective bandwidth determine the applicable path.
Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.