# Scale-up, scale-out and management fabrics

Canonical: https://semiaifoundry.com/learn/atlas/scaleup-scaleout-boundaries/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Chip-to-hall integration · Atlas v1.3.0

# Scale-up, scale-out and management fabrics

Assigns each traffic class to its actual interconnect and resource boundary.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#scaleup-scaleout-boundaries)Package to rack and cluster; named reference geometry

Learning objectiveDistinguish traffic roles and bandwidth aggregation boundaries in a cluster.

## The mechanism

Scale-up links connect tightly coupled accelerators within a selected domain, while scale-out networks join nodes or domains into larger clusters. Storage and management traffic may use separate fabrics even when cables or switches look similar. The distinction describes roles and locality, not a universal distance threshold or protocol monopoly. A collective can cross both domains, encountering different bandwidth, latency and contention. The NVIDIA rack documentation separates acceleration, cluster and management paths in a concrete implementation. A correct visualization follows the selected message across those boundaries and states whether any displayed bandwidth is one-way, bidirectional or an aggregate over many ports.

1.  Identify endpoints and the traffic’s purpose before choosing its route.
2.  Mark the local acceleration domain and the links that leave it.
3.  Trace every switch or network boundary required by the communication.
4.  Keep storage and management dependencies visible even if they are not on the main tensor path.
5.  Compare useful flow rates and congestion at the relevant cut rather than adding every advertised port rate.

## Science and assumptions

### Topology cut bound

$$
T\ge\frac{Q_{\mathrm{cut}}}{B_{\mathrm{cut}}}
$$

Cut-crossing traffic requires at least its byte count divided by usable cut bandwidth.

Symbols and units

T

Time lower bound associated with the topology cut

s

Q\_cut

Bytes crossing the defined cut

bytes

B\_cut

Sustainable aggregate bandwidth actually usable across it

bytes/s

Qcut is bytes crossing a defined cut and Bcut sustainable bytes/s across it.

Assumptions A traffic bound before synchronization and queueing; parallel links count only if the routing can use them.

## Engineering example

Synthetic topology: abundant rack-local bandwidth cannot hide a 10 GB transfer across a 100 GB/s inter-rack cut, which alone takes at least 100 ms.

## Distinctions and tradeoffs

Scale-up and scale-out are roles in a system topology; neither label alone establishes coherence, latency or unlimited scaling.

Tightly connected local domains can accelerate frequent exchanges; spreading a job across domains may add capacity while exposing slower or contended cuts.

Model parallelism depends on where communication occurs, making topology placement as consequential as the number of accelerators allocated.

## Specifications and interfaces

Traffic roles

Tensor exchange, storage and management

Roles may share equipment but require separate accounting.

Bandwidth boundary

Port, link, cut or fabric aggregate

A sum of ports is not a single flow’s achievable rate.

### Interfaces

Accelerator communication Input

Partial tensors, gradients or routed activations

Local and remote fabric Output

Messages crossing the required topology boundaries

## Check your understanding

### What best predicts the limit for traffic moving between two rack groups?

1.  The sum of all ports inside both groups
2.  The count of management controllers
3.  The useful capacity of the cut those messages cross

Answer and reasoning

The useful capacity of the cut those messages cross

Every required cross-group byte must traverse the connecting cut, regardless of unused local links.

## Evidence and scope

The example is synthetic. Protocol choice and placement depend on the actual system; no universal NVLink-versus-Ethernet performance ordering is implied.

Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.

## Sources

-   [NVIDIA DGX GB Rack Scale Systems: Networking](https://docs.nvidia.com/dgx/dgxgb200-user-guide/networking.html)
-   [NVIDIA DGX GB Rack Scale Systems: Hardware](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/scaleup-scaleout-boundaries-bdd85897c5539126.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [Leaf switch and endpoint network](https://semiaifoundry.com/learn/atlas/leaf-switch/)
-   [NVLink switch tray and scale-up fabric](https://semiaifoundry.com/learn/atlas/nvlink-switch/)

### Related lessons

-   [NVLink GPU interface](https://semiaifoundry.com/learn/atlas/nvlink-port/)
-   [Spine switching and fabric topology](https://semiaifoundry.com/learn/atlas/spine-switch/)
-   [Collective traffic and congestion](https://semiaifoundry.com/learn/atlas/fabric-collective-traffic/)