The mechanism
Scale-up links connect tightly coupled accelerators within a selected domain, while scale-out networks join nodes or domains into larger clusters. Storage and management traffic may use separate fabrics even when cables or switches look similar. The distinction describes roles and locality, not a universal distance threshold or protocol monopoly. A collective can cross both domains, encountering different bandwidth, latency and contention. The NVIDIA rack documentation separates acceleration, cluster and management paths in a concrete implementation. A correct visualization follows the selected message across those boundaries and states whether any displayed bandwidth is one-way, bidirectional or an aggregate over many ports.
- Identify endpoints and the traffic’s purpose before choosing its route.
- Mark the local acceleration domain and the links that leave it.
- Trace every switch or network boundary required by the communication.
- Keep storage and management dependencies visible even if they are not on the main tensor path.
- Compare useful flow rates and congestion at the relevant cut rather than adding every advertised port rate.
Science and assumptions
Topology cut bound
Cut-crossing traffic requires at least its byte count divided by usable cut bandwidth.
Symbols and units
- T
- Time lower bound associated with the topology cut
s
- Q_cut
- Bytes crossing the defined cut
bytes
- B_cut
- Sustainable aggregate bandwidth actually usable across it
bytes/s
Qcut is bytes crossing a defined cut and Bcut sustainable bytes/s across it.
Assumptions A traffic bound before synchronization and queueing; parallel links count only if the routing can use them.
Engineering example
Synthetic topology: abundant rack-local bandwidth cannot hide a 10 GB transfer across a 100 GB/s inter-rack cut, which alone takes at least 100 ms.
Distinctions and tradeoffs
Scale-up and scale-out are roles in a system topology; neither label alone establishes coherence, latency or unlimited scaling.
Tightly connected local domains can accelerate frequent exchanges; spreading a job across domains may add capacity while exposing slower or contended cuts.
Model parallelism depends on where communication occurs, making topology placement as consequential as the number of accelerators allocated.
Specifications and interfaces
- Traffic roles
- Tensor exchange, storage and management
Roles may share equipment but require separate accounting.
- Bandwidth boundary
- Port, link, cut or fabric aggregate
A sum of ports is not a single flow’s achievable rate.
Interfaces
- Accelerator communication Input
- Partial tensors, gradients or routed activations
- Local and remote fabric Output
- Messages crossing the required topology boundaries
Check your understanding
What best predicts the limit for traffic moving between two rack groups?
- The sum of all ports inside both groups
- The count of management controllers
- The useful capacity of the cut those messages cross
Answer and reasoning
The useful capacity of the cut those messages cross
Every required cross-group byte must traverse the connecting cut, regardless of unused local links.
Evidence and scope
The example is synthetic. Protocol choice and placement depend on the actual system; no universal NVLink-versus-Ethernet performance ordering is implied.
Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.