{
  "schemaVersion": 1,
  "id": "atlas-lesson-fabric-collective-traffic",
  "title": "Collective traffic and congestion",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Coordinates distributed tensor exchanges while accounting for topology and contention.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/fabric-collective-traffic/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Chip-to-hall integration",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "This is a teaching algorithm, not a claim that NCCL always chooses a ring. Rank placement and software version belong with any measured result.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/fabric-collective-traffic-1028fa16b2d022a1.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#fabric-collective-traffic",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "64f409e81a26a1c840d807c550535a9962a97b4533ca55734c377fd46971182a",
  "contentLength": 4744,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-fabric-collective-traffic.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-fabric-collective-traffic.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Collective traffic and congestion",
      "start": 41,
      "end": 387
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 387,
      "end": 1517
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1517,
      "end": 1545
    },
    {
      "id": "section-4",
      "title": "Ideal ring all-reduce",
      "start": 1545,
      "end": 2223
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2223,
      "end": 2445
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2445,
      "end": 2917
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 2917,
      "end": 3166
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3166,
      "end": 3337
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3337,
      "end": 3366
    },
    {
      "id": "section-10",
      "title": "What is 400 Gbit/s in raw decimal bytes/s?",
      "start": 3366,
      "end": 3625
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3625,
      "end": 3943
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 3943,
      "end": 4287
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4287,
      "end": 4310
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4310,
      "end": 4503
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 4503,
      "end": 4744
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#fabric-collective-traffic"
    },
    {
      "title": "NCCL User Guide: Collective Operations",
      "url": "https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/fabric-collective-traffic-1028fa16b2d022a1.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Collective communication library",
      "url": "https://semiaifoundry.com/learn/atlas/nccl/"
    },
    {
      "title": "Leaf switch and endpoint network",
      "url": "https://semiaifoundry.com/learn/atlas/leaf-switch/"
    },
    {
      "title": "Data, tensor, pipeline, and state parallelism",
      "url": "https://semiaifoundry.com/learn/atlas/parallelism/"
    },
    {
      "title": "Scale-up, scale-out and management fabrics",
      "url": "https://semiaifoundry.com/learn/atlas/scaleup-scaleout-boundaries/"
    }
  ],
  "recordHash": "855d029115d577e24d86ad37ed13e14cfb09223019d2097a1f5d94057f1f6f87",
  "contentFormat": "text/markdown",
  "content": "Chip-to-hall integration · Atlas v1.3.0\n\n# Collective traffic and congestion\n\nCoordinates distributed tensor exchanges while accounting for topology and contention.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#fabric-collective-traffic)Package to rack and cluster; named reference geometry\n\nLearning objectiveConnect collective semantics to a bounded physical traffic model.\n\n## The mechanism\n\nA collective describes communication among a group of participants. All-reduce combines values and returns the result to all ranks; all-gather assembles distributed pieces; reduce-scatter combines values while leaving different output pieces on different ranks. Libraries such as NCCL choose implementations suited to message size and topology. The logical collective is not identical to one physical link transfer. Congestion, competing storage traffic and poorly placed ranks can reduce useful bandwidth, while many small exchanges expose latency. Correct diagnosis follows both the operation’s semantics and the actual traffic schedule, including any communication that overlaps computation.\n\n1.  Define the collective and the tensor bytes supplied by each rank.\n2.  Choose rank placement and an implementation suited to the topology.\n3.  Map the collective phases onto actual links and identify shared cuts.\n4.  Measure effective bandwidth, phase latency and exposed synchronization under competing traffic.\n5.  Change placement, scheduling or algorithm only after identifying the binding network behavior.\n\n## Science and assumptions\n\n### Ideal ring all-reduce\n\n$$\nT\\approx2(n-1)\\alpha+\\frac{2(n-1)S}{nB}\n$$\n\nIdeal ring duration is two times ranks minus one startup costs, plus two times ranks minus one divided by ranks times per-rank payload divided by bandwidth.\n\nSymbols and units\n\nT\n\nIdealized ring all-reduce duration\n\ns\n\nn\n\nParticipating ranks\n\npositive integer count\n\nα\n\nStartup latency per ring step\n\ns/step\n\nS\n\nInput payload per rank\n\nbytes\n\nB\n\nEffective one-direction payload bandwidth per rank\n\nbytes/s\n\nn is ranks, α seconds per step, S bytes/rank and B effective bytes/s.\n\nAssumptions A simplified ring model with equal links; topology, pipelining, overlap and congestion can change actual timing.\n\n## Engineering example\n\nSynthetic eight-rank ring with 1 GiB/rank and 50 GB/s effective bandwidth has about 37.6 ms of bandwidth time plus 14 latency terms. Using raw line rate as effective bandwidth would be optimistic.\n\n## Distinctions and tradeoffs\n\nA 400 Gbit/s port is not a measured 400 GB/s collective, and bidirectional rates cannot be silently used as one-way payload bandwidth.\n\nAlgorithms with fewer phases can help small messages, while larger transfers reward bandwidth-efficient routing; the best choice changes with topology.\n\nSynchronized training and tensor-parallel inference repeatedly wait for collective results, turning network behavior into model-step or token latency.\n\n## Specifications and interfaces\n\nMessage size\n\nBytes per participating rank\n\nDo not confuse one rank’s input with total wire traffic.\n\nEffective bandwidth\n\nUseful bytes/s under the chosen algorithm\n\nLine rate omits protocol and contention losses.\n\n### Interfaces\n\nRank-local tensors Input\n\nValues requiring collection, distribution or reduction\n\nCollective output Output\n\nCorrectly placed combined or gathered tensors\n\n## Check your understanding\n\n### What is 400 Gbit/s in raw decimal bytes/s?\n\n1.  50 GB/s before overhead\n2.  400 GB/s\n3.  3,200 GB/s\n\nAnswer and reasoning\n\n50 GB/s before overhead\n\nEight bits form one byte; useful payload rate is lower when coding, protocol and contention losses apply.\n\n## Evidence and scope\n\nThis is a teaching algorithm, not a claim that NCCL always chooses a ring. Rank placement and software version belong with any measured result.\n\nVary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.\n\n## Sources\n\n-   [NCCL User Guide: Collective Operations](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/fabric-collective-traffic-1028fa16b2d022a1.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Collective communication library](https://semiaifoundry.com/learn/atlas/nccl/)\n-   [Leaf switch and endpoint network](https://semiaifoundry.com/learn/atlas/leaf-switch/)\n\n### Related lessons\n\n-   [Data, tensor, pipeline, and state parallelism](https://semiaifoundry.com/learn/atlas/parallelism/)\n-   [Scale-up, scale-out and management fabrics](https://semiaifoundry.com/learn/atlas/scaleup-scaleout-boundaries/)"
}