{"id":"nvlink-port","level":"die","name":"NVLink GPU interface","shortName":"NVLink","role":"Moves tensor data between GPUs through a high-bandwidth scale-up interconnect.","description":"A model can be partitioned across GPUs because one device may not provide enough memory or enough compute. NVLink supplies a fast path for those devices to exchange tensor pieces and partial results. Physical links connect GPUs to peers or switches; software still chooses a parallel strategy and schedules the required communication. A connected group is not automatically one ordinary GPU with uniform access costs. Collective operations, message size, synchronization, and topology all affect useful performance. When a tensor is partitioned across devices, its local computations and any required reduction create separate arithmetic and communication costs. NVLink is distinct from the internal die-to-die interface and from the network that joins racks.","specs":[{"label":"Blackwell NVLink 5","value":"1.8 TB/s bidirectional per GPU","context":"Aggregate across 18 links; both directions summed."},{"label":"Link count","value":"18 per GPU","context":"100 GB/s bidirectional per link in NVIDIA’s specification."},{"label":"NVL72 aggregation","value":"About 130 TB/s","context":"Rack aggregate interface bandwidth; not the bandwidth of one transfer."}],"interfaces":[{"name":"Local tensors","direction":"in","signal":"GPU memory and collective-operation buffers"},{"name":"Remote transfers","direction":"out","signal":"Tensor partitions, reductions, and peer memory traffic"},{"name":"Remote results","direction":"in","signal":"Peer tensor data and synchronization"}],"tradeoff":"More communication capacity helps larger parallel models, but every synchronization remains work that must be scheduled. Partitioning a model too finely can make transfers dominate the saved compute time.","whyAI":"Tensor and expert parallelism depend on exchanging activations or partial sums between accelerators. Scale-up communication lets nearby GPUs cooperate more efficiently on one model.","related":["on-chip-network","gpu-die","grace-blackwell-superchip","network-adapter"],"sources":[{"title":"NVIDIA: NVLink and NVSwitch for LLM inference","url":"https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/"},{"title":"NVIDIA: Inside Blackwell Ultra, with Blackwell comparison","url":"https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/"},{"title":"NVIDIA NCCL tests: interpreting collective bandwidth","url":"https://github.com/NVIDIA/nccl-tests/blob/master/doc/PERFORMANCE.md"},{"title":"NVIDIA: Tesla P100 launch announcement","url":"https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437"}],"scale":"Die I/O blocks to centimeter/meter copper links; schematic signal paths","kind":"multi-gpu-reduction-links","mechanism":["Software partitions a workload and identifies the tensor values or partial results that must cross GPU boundaries.","NVLink-connected endpoints transfer those buffers through the supported peer or switched fabric.","Collective operations combine or distribute the results, then dependent computation proceeds when the required data are ready."],"misconception":"A 1.8 TB/s bidirectional rating sums traffic capacity in both directions. It does not promise one transfer moving at 1.8 TB/s in a single direction.","check":{"question":"Which work does NVLink most directly support?","options":["Refreshing a single DRAM capacitor.","Encoding a text token into its vocabulary ID.","Exchanging tensors among GPUs in a scale-up domain."],"answer":2,"explanation":"NVLink connects cooperating accelerators. The mathematical partition and communication schedule are still chosen by software."},"branch":"Connectivity","subbranch":"GPU scale-up communication","science":[{"title":"Communication has volume and startup costs","equation":"Tring ≈ 2(p − 1)α + [2(p − 1)/p] × M/β","explanation":"p is the number of ranks, M is each rank’s input size in bytes, α is startup time per ring step, and β is effective one-direction payload bandwidth per rank. The first term exposes why many small messages can be latency limited. The second describes the distributed data movement of a ring all-reduce, which reduces partial values and shares the combined result.","assumptions":"A balanced ring using reduce-scatter plus all-gather with equal links and no contention, reduction-compute bottleneck, or overlap. NCCL can choose other algorithms, including switch-assisted schemes; β must not be substituted with a summed bidirectional marketing rating."}],"history":[{"year":2016,"title":"Pascal P100 brings NVLink to NVIDIA accelerators","significance":"P100’s launch combined NVLink with a new accelerator and memory architecture, giving cooperating GPUs a communication path beyond the host PCIe interface.","source":{"title":"NVIDIA: Tesla P100 launch announcement","url":"https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437"}}],"learningObjective":"Explain why collective traffic grows with participating ranks.","engineeringExample":"A ring all-reduce sends approximately 2(p−1)M/p bytes per rank; startup latency can dominate small messages.","evidenceNotes":"The ring is an ideal topology model; implementations may use trees or hierarchies.","visualFamily":"signal-power","prerequisites":["on-chip-network"],"labId":"engineering","scope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.","deepLabId":"network","investigationId":"execution","labScope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model."}