{"id":"fabric-collective-traffic","level":"infrastructure","name":"Collective traffic and congestion","shortName":"Collective traffic and congestion","role":"Coordinates distributed tensor exchanges while accounting for topology and contention.","description":"A collective describes communication among a group of participants. All-reduce combines values and returns the result to all ranks; all-gather assembles distributed pieces; reduce-scatter combines values while leaving different output pieces on different ranks. Libraries such as NCCL choose implementations suited to message size and topology. The logical collective is not identical to one physical link transfer. Congestion, competing storage traffic and poorly placed ranks can reduce useful bandwidth, while many small exchanges expose latency. Correct diagnosis follows both the operation’s semantics and the actual traffic schedule, including any communication that overlaps computation.","specs":[{"label":"Message size","value":"Bytes per participating rank","context":"Do not confuse one rank’s input with total wire traffic."},{"label":"Effective bandwidth","value":"Useful bytes/s under the chosen algorithm","context":"Line rate omits protocol and contention losses."}],"interfaces":[{"name":"Rank-local tensors","direction":"in","signal":"Values requiring collection, distribution or reduction"},{"name":"Collective output","direction":"out","signal":"Correctly placed combined or gathered tensors"}],"tradeoff":"Algorithms with fewer phases can help small messages, while larger transfers reward bandwidth-efficient routing; the best choice changes with topology.","whyAI":"Synchronized training and tensor-parallel inference repeatedly wait for collective results, turning network behavior into model-step or token latency.","related":["nccl","parallelism","scaleup-scaleout-boundaries"],"sources":[{"title":"NCCL User Guide: Collective Operations","url":"https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html"}],"scale":"Package to rack and cluster; named reference geometry","kind":"architecture","branch":"Chip-to-hall integration","subbranch":"Rack anatomy and network fabrics","mechanism":["Define the collective and the tensor bytes supplied by each rank.","Choose rank placement and an implementation suited to the topology.","Map the collective phases onto actual links and identify shared cuts.","Measure effective bandwidth, phase latency and exposed synchronization under competing traffic.","Change placement, scheduling or algorithm only after identifying the binding network behavior."],"misconception":"A 400 Gbit/s port is not a measured 400 GB/s collective, and bidirectional rates cannot be silently used as one-way payload bandwidth.","check":{"question":"What is 400 Gbit/s in raw decimal bytes/s?","options":["50 GB/s before overhead","400 GB/s","3,200 GB/s"],"answer":0,"explanation":"Eight bits form one byte; useful payload rate is lower when coding, protocol and contention losses apply."},"science":[{"title":"Ideal ring all-reduce","equation":"T ≈ 2(n−1)α + 2(n−1)S/(nB)","explanation":"n is ranks, α seconds per step, S bytes/rank and B effective bytes/s.","assumptions":"A simplified ring model with equal links; topology, pipelining, overlap and congestion can change actual timing."}],"prerequisites":["nccl","leaf-switch"],"learningObjective":"Connect collective semantics to a bounded physical traffic model.","workstream":"W09","visualFamily":"rack-system","engineeringExample":"Synthetic eight-rank ring with 1 GiB/rank and 50 GB/s effective bandwidth has about 37.6 ms of bandwidth time plus 14 latency terms. Using raw line rate as effective bandwidth would be optimistic.","evidenceNotes":"This is a teaching algorithm, not a claim that NCCL always chooses a ring. Rank placement and software version belong with any measured result.","labId":"engineering","scope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model.","deepLabId":"network","investigationId":"execution","labScope":"Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model."}