# NVLink GPU interface Canonical: https://semiaifoundry.com/learn/atlas/nvlink-port/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Connectivity · Atlas v1.3.0 # NVLink GPU interface Moves tensor data between GPUs through a high-bandwidth scale-up interconnect. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#nvlink-port)Die I/O blocks to centimeter/meter copper links; schematic signal paths Learning objectiveExplain why collective traffic grows with participating ranks. ## The mechanism A model can be partitioned across GPUs because one device may not provide enough memory or enough compute. NVLink supplies a fast path for those devices to exchange tensor pieces and partial results. Physical links connect GPUs to peers or switches; software still chooses a parallel strategy and schedules the required communication. A connected group is not automatically one ordinary GPU with uniform access costs. Collective operations, message size, synchronization, and topology all affect useful performance. When a tensor is partitioned across devices, its local computations and any required reduction create separate arithmetic and communication costs. NVLink is distinct from the internal die-to-die interface and from the network that joins racks. 1. Software partitions a workload and identifies the tensor values or partial results that must cross GPU boundaries. 2. NVLink-connected endpoints transfer those buffers through the supported peer or switched fabric. 3. Collective operations combine or distribute the results, then dependent computation proceeds when the required data are ready. ## Science and assumptions ### Communication has volume and startup costs $$ T_{\mathrm{ring}}\approx2(p-1)\alpha+\frac{2(p-1)}{p}\frac{M}{\beta} $$ Ring time is approximately two times ranks minus one startup costs, plus two times ranks minus one divided by ranks times payload divided by bandwidth. Symbols and units T\_ring Modeled ring all-reduce time s p Participating ranks positive integer count α Startup latency per ring step s/step M Input payload per rank bytes β Effective one-direction payload bandwidth per rank bytes/s p is the number of ranks, M is each rank’s input size in bytes, α is startup time per ring step, and β is effective one-direction payload bandwidth per rank. The first term exposes why many small messages can be latency limited. The second describes the distributed data movement of a ring all-reduce, which reduces partial values and shares the combined result. Assumptions A balanced ring using reduce-scatter plus all-gather with equal links and no contention, reduction-compute bottleneck, or overlap. NCCL can choose other algorithms, including switch-assisted schemes; β must not be substituted with a summed bidirectional marketing rating. ## Engineering example A ring all-reduce sends approximately 2(p−1)M/p bytes per rank; startup latency can dominate small messages. ## Distinctions and tradeoffs A 1.8 TB/s bidirectional rating sums traffic capacity in both directions. It does not promise one transfer moving at 1.8 TB/s in a single direction. More communication capacity helps larger parallel models, but every synchronization remains work that must be scheduled. Partitioning a model too finely can make transfers dominate the saved compute time. Tensor and expert parallelism depend on exchanging activations or partial sums between accelerators. Scale-up communication lets nearby GPUs cooperate more efficiently on one model. ## Specifications and interfaces Blackwell NVLink 5 1.8 TB/s bidirectional per GPU Aggregate across 18 links; both directions summed. Link count 18 per GPU 100 GB/s bidirectional per link in NVIDIA’s specification. NVL72 aggregation About 130 TB/s Rack aggregate interface bandwidth; not the bandwidth of one transfer. ### Interfaces Local tensors Input GPU memory and collective-operation buffers Remote transfers Output Tensor partitions, reductions, and peer memory traffic Remote results Input Peer tensor data and synchronization ## Check your understanding ### Which work does NVLink most directly support? 1. Refreshing a single DRAM capacitor. 2. Encoding a text token into its vocabulary ID. 3. Exchanging tensors among GPUs in a scale-up domain. Answer and reasoning Exchanging tensors among GPUs in a scale-up domain. NVLink connects cooperating accelerators. The mathematical partition and communication schedule are still chosen by software. ## Historical context ### 2016 · Pascal P100 brings NVLink to NVIDIA accelerators P100’s launch combined NVLink with a new accelerator and memory architecture, giving cooperating GPUs a communication path beyond the host PCIe interface. [NVIDIA: Tesla P100 launch announcement](https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437) ## Evidence and scope The ring is an ideal topology model; implementations may use trees or hierarchies. Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model. ## Sources - [NVIDIA: NVLink and NVSwitch for LLM inference](https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/) - [NVIDIA: Inside Blackwell Ultra, with Blackwell comparison](https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/) - [NVIDIA NCCL tests: interpreting collective bandwidth](https://github.com/NVIDIA/nccl-tests/blob/master/doc/PERFORMANCE.md) - [NVIDIA: Tesla P100 launch announcement](https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-port-5a5e370e6edd4419.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [On-chip interconnect fabric](https://semiaifoundry.com/learn/atlas/on-chip-network/) ### Related lessons - [GPU logic dies](https://semiaifoundry.com/learn/atlas/gpu-die/) - [Grace Blackwell Superchip](https://semiaifoundry.com/learn/atlas/grace-blackwell-superchip/) - [Network interface adapter](https://semiaifoundry.com/learn/atlas/network-adapter/)