{
  "schemaVersion": 1,
  "id": "atlas-lesson-nvlink-port",
  "title": "NVLink GPU interface",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Moves tensor data between GPUs through a high-bandwidth scale-up interconnect.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/nvlink-port/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Connectivity",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The ring is an ideal topology model; implementations may use trees or hierarchies.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-port-5a5e370e6edd4419.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#nvlink-port",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "000f51c3a51a796d4315bb00decb22b8f1cecee9e1e53c9f41bd6fe406d1659e",
  "contentLength": 6339,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-nvlink-port.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-nvlink-port.md",
  "sections": [
    {
      "id": "section-1",
      "title": "NVLink GPU interface",
      "start": 29,
      "end": 355
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 355,
      "end": 1486
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1486,
      "end": 1514
    },
    {
      "id": "section-4",
      "title": "Communication has volume and startup costs",
      "start": 1514,
      "end": 2689
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2689,
      "end": 2823
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2823,
      "end": 3393
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3393,
      "end": 3722
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3722,
      "end": 3945
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3945,
      "end": 3974
    },
    {
      "id": "section-10",
      "title": "Which work does NVLink most directly support?",
      "start": 3974,
      "end": 4374
    },
    {
      "id": "section-11",
      "title": "Historical context",
      "start": 4374,
      "end": 4397
    },
    {
      "id": "section-12",
      "title": "2016 · Pascal P100 brings NVLink to NVIDIA accelerators",
      "start": 4397,
      "end": 4812
    },
    {
      "id": "section-13",
      "title": "Evidence and scope",
      "start": 4812,
      "end": 5069
    },
    {
      "id": "section-14",
      "title": "Sources",
      "start": 5069,
      "end": 5933
    },
    {
      "id": "section-15",
      "title": "Continue exploring",
      "start": 5933,
      "end": 5956
    },
    {
      "id": "section-16",
      "title": "Foundations",
      "start": 5956,
      "end": 6064
    },
    {
      "id": "section-17",
      "title": "Related lessons",
      "start": 6064,
      "end": 6339
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#nvlink-port"
    },
    {
      "title": "NVIDIA: Tesla P100 launch announcement",
      "url": "https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437"
    },
    {
      "title": "NVIDIA: NVLink and NVSwitch for LLM inference",
      "url": "https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/"
    },
    {
      "title": "NVIDIA: Inside Blackwell Ultra, with Blackwell comparison",
      "url": "https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/"
    },
    {
      "title": "NVIDIA NCCL tests: interpreting collective bandwidth",
      "url": "https://github.com/NVIDIA/nccl-tests/blob/master/doc/PERFORMANCE.md"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-port-5a5e370e6edd4419.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "On-chip interconnect fabric",
      "url": "https://semiaifoundry.com/learn/atlas/on-chip-network/"
    },
    {
      "title": "GPU logic dies",
      "url": "https://semiaifoundry.com/learn/atlas/gpu-die/"
    },
    {
      "title": "Grace Blackwell Superchip",
      "url": "https://semiaifoundry.com/learn/atlas/grace-blackwell-superchip/"
    },
    {
      "title": "Network interface adapter",
      "url": "https://semiaifoundry.com/learn/atlas/network-adapter/"
    }
  ],
  "recordHash": "74f49b061ceac1aa5c307f372f9ba5f6b36bff811e77f67f1f89397078cccd2f",
  "contentFormat": "text/markdown",
  "content": "Connectivity · Atlas v1.3.0\n\n# NVLink GPU interface\n\nMoves tensor data between GPUs through a high-bandwidth scale-up interconnect.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#nvlink-port)Die I/O blocks to centimeter/meter copper links; schematic signal paths\n\nLearning objectiveExplain why collective traffic grows with participating ranks.\n\n## The mechanism\n\nA model can be partitioned across GPUs because one device may not provide enough memory or enough compute. NVLink supplies a fast path for those devices to exchange tensor pieces and partial results. Physical links connect GPUs to peers or switches; software still chooses a parallel strategy and schedules the required communication. A connected group is not automatically one ordinary GPU with uniform access costs. Collective operations, message size, synchronization, and topology all affect useful performance. When a tensor is partitioned across devices, its local computations and any required reduction create separate arithmetic and communication costs. NVLink is distinct from the internal die-to-die interface and from the network that joins racks.\n\n1.  Software partitions a workload and identifies the tensor values or partial results that must cross GPU boundaries.\n2.  NVLink-connected endpoints transfer those buffers through the supported peer or switched fabric.\n3.  Collective operations combine or distribute the results, then dependent computation proceeds when the required data are ready.\n\n## Science and assumptions\n\n### Communication has volume and startup costs\n\n$$\nT_{\\mathrm{ring}}\\approx2(p-1)\\alpha+\\frac{2(p-1)}{p}\\frac{M}{\\beta}\n$$\n\nRing time is approximately two times ranks minus one startup costs, plus two times ranks minus one divided by ranks times payload divided by bandwidth.\n\nSymbols and units\n\nT\\_ring\n\nModeled ring all-reduce time\n\ns\n\np\n\nParticipating ranks\n\npositive integer count\n\nα\n\nStartup latency per ring step\n\ns/step\n\nM\n\nInput payload per rank\n\nbytes\n\nβ\n\nEffective one-direction payload bandwidth per rank\n\nbytes/s\n\np is the number of ranks, M is each rank’s input size in bytes, α is startup time per ring step, and β is effective one-direction payload bandwidth per rank. The first term exposes why many small messages can be latency limited. The second describes the distributed data movement of a ring all-reduce, which reduces partial values and shares the combined result.\n\nAssumptions A balanced ring using reduce-scatter plus all-gather with equal links and no contention, reduction-compute bottleneck, or overlap. NCCL can choose other algorithms, including switch-assisted schemes; β must not be substituted with a summed bidirectional marketing rating.\n\n## Engineering example\n\nA ring all-reduce sends approximately 2(p−1)M/p bytes per rank; startup latency can dominate small messages.\n\n## Distinctions and tradeoffs\n\nA 1.8 TB/s bidirectional rating sums traffic capacity in both directions. It does not promise one transfer moving at 1.8 TB/s in a single direction.\n\nMore communication capacity helps larger parallel models, but every synchronization remains work that must be scheduled. Partitioning a model too finely can make transfers dominate the saved compute time.\n\nTensor and expert parallelism depend on exchanging activations or partial sums between accelerators. Scale-up communication lets nearby GPUs cooperate more efficiently on one model.\n\n## Specifications and interfaces\n\nBlackwell NVLink 5\n\n1.8 TB/s bidirectional per GPU\n\nAggregate across 18 links; both directions summed.\n\nLink count\n\n18 per GPU\n\n100 GB/s bidirectional per link in NVIDIA’s specification.\n\nNVL72 aggregation\n\nAbout 130 TB/s\n\nRack aggregate interface bandwidth; not the bandwidth of one transfer.\n\n### Interfaces\n\nLocal tensors Input\n\nGPU memory and collective-operation buffers\n\nRemote transfers Output\n\nTensor partitions, reductions, and peer memory traffic\n\nRemote results Input\n\nPeer tensor data and synchronization\n\n## Check your understanding\n\n### Which work does NVLink most directly support?\n\n1.  Refreshing a single DRAM capacitor.\n2.  Encoding a text token into its vocabulary ID.\n3.  Exchanging tensors among GPUs in a scale-up domain.\n\nAnswer and reasoning\n\nExchanging tensors among GPUs in a scale-up domain.\n\nNVLink connects cooperating accelerators. The mathematical partition and communication schedule are still chosen by software.\n\n## Historical context\n\n### 2016 · Pascal P100 brings NVLink to NVIDIA accelerators\n\nP100’s launch combined NVLink with a new accelerator and memory architecture, giving cooperating GPUs a communication path beyond the host PCIe interface.\n\n[NVIDIA: Tesla P100 launch announcement](https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437)\n\n## Evidence and scope\n\nThe ring is an ideal topology model; implementations may use trees or hierarchies.\n\nVary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.\n\n## Sources\n\n-   [NVIDIA: NVLink and NVSwitch for LLM inference](https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/)\n-   [NVIDIA: Inside Blackwell Ultra, with Blackwell comparison](https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/)\n-   [NVIDIA NCCL tests: interpreting collective bandwidth](https://github.com/NVIDIA/nccl-tests/blob/master/doc/PERFORMANCE.md)\n-   [NVIDIA: Tesla P100 launch announcement](https://nvidianews.nvidia.com/news/nvidia-delivers-massive-performance-leap-for-deep-learning-hpc-applications-with-nvidia-tesla-p100-accelerators-6622437)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/nvlink-port-5a5e370e6edd4419.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [On-chip interconnect fabric](https://semiaifoundry.com/learn/atlas/on-chip-network/)\n\n### Related lessons\n\n-   [GPU logic dies](https://semiaifoundry.com/learn/atlas/gpu-die/)\n-   [Grace Blackwell Superchip](https://semiaifoundry.com/learn/atlas/grace-blackwell-superchip/)\n-   [Network interface adapter](https://semiaifoundry.com/learn/atlas/network-adapter/)"
}