# A documented rack as a system boundary

Canonical: https://semiaifoundry.com/learn/atlas/rack-reference-anatomy/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Chip-to-hall integration · Atlas v1.3.0

# A documented rack as a system boundary

Connects compute trays, switches, management, power and cooling in one named reference.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#rack-reference-anatomy)Package to rack and cluster; named reference geometry

Learning objectiveRead a rack as converging physical and operational subsystems.

## The mechanism

A rack-scale accelerator system contains more than GPUs. Compute trays hold CPUs, accelerators, local memory and peripheral devices; switches connect an acceleration domain; management controllers support configuration and recovery. Power shelves and busbars distribute electricity, while manifolds deliver coolant to selected components. NVIDIA’s DGX GB rack guide supplies a concrete NVL72 reference with distinct compute and switch trays. The reference is useful because its interfaces are documented, not because every AI installation uses this design. A tray remains the same assembly when considered through its network, electrical and thermal connections, but each view describes a different service and constraint.

1.  Identify the documented compute trays and their CPU/accelerator roles.
2.  Trace the switch and cable paths that form the rack’s acceleration domain.
3.  Locate separate management access for telemetry, recovery and configuration.
4.  Follow power and coolant distribution without confusing their connections with data cables.
5.  Verify the rack’s complete configuration and service dependencies before assigning workload capacity.

## Science and assumptions

### Component consistency

$$
N_{\mathrm{GPU}}=N_{\mathrm{tray}}N_{\mathrm{GPU/tray}}
$$

Total GPU count is the number of trays times GPUs per tray.

Symbols and units

N\_GPU

GPU count within the chosen assembly boundary

GPUs

N\_tray

Number of uniform compute trays

trays

N\_GPU/tray

GPU count per such tray

GPUs/tray

Counts are dimensionless hardware quantities; they do not establish tokens/s.

Assumptions A uniform documented tray configuration; spare, disabled or service-reserved resources require separate accounting.

## Engineering example

The documented 18 compute trays with four GPUs each account for 72 GPUs. This reconciles an inventory; it does not predict workload throughput, allocated capacity or electrical draw under every load.

## Distinctions and tradeoffs

A rack described as one large AI machine is not literally one GPU die or one unlimited coherent memory.

Rack integration increases local connectivity but concentrates service and facility requirements; maintenance may need coordinated management of many components.

Useful capacity depends on the entire rack operating correctly, including networking and peripheral components that do not perform matrix multiplication.

## Specifications and interfaces

Named boundary

DGX GB NVL72 rack reference

Preserve generation and system configuration in any numeric card.

Independent services

Compute, data, management, power and cooling

A working data link does not establish readiness of the other services.

### Interfaces

Facility services Input

Electrical supply, coolant, cluster and management networks

Rack workload resources Output

Connected compute with telemetry and cooling paths

## Check your understanding

### Why preserve a named reference configuration?

1.  A rack name determines model quality
2.  Its counts and interfaces have a defined evidence boundary
3.  Every future rack must match it

Answer and reasoning

Its counts and interfaces have a defined evidence boundary

A reference makes the anatomy auditable while leaving alternatives and later revisions distinct.

## Evidence and scope

The exact tray count is source-reported in the claims ledger. Mixed cooling and power details are specific to the reference, not universal rack standards.

Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.

## Sources

-   [NVIDIA DGX GB Rack Scale Systems: Hardware](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/rack-reference-anatomy-ea8cb65c5ed019e3.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [Integrated accelerator rack: GB200 NVL72 example](https://semiaifoundry.com/learn/atlas/rack/)
-   [GB200 compute tray](https://semiaifoundry.com/learn/atlas/compute-tray/)

### Related lessons

-   [NVLink switch tray and scale-up fabric](https://semiaifoundry.com/learn/atlas/nvlink-switch/)
-   [Rack power conversion and distribution](https://semiaifoundry.com/learn/atlas/rack-power-conversion/)