The mechanism
A rack-scale accelerator system contains more than GPUs. Compute trays hold CPUs, accelerators, local memory and peripheral devices; switches connect an acceleration domain; management controllers support configuration and recovery. Power shelves and busbars distribute electricity, while manifolds deliver coolant to selected components. NVIDIA’s DGX GB rack guide supplies a concrete NVL72 reference with distinct compute and switch trays. The reference is useful because its interfaces are documented, not because every AI installation uses this design. A tray remains the same assembly when considered through its network, electrical and thermal connections, but each view describes a different service and constraint.
- Identify the documented compute trays and their CPU/accelerator roles.
- Trace the switch and cable paths that form the rack’s acceleration domain.
- Locate separate management access for telemetry, recovery and configuration.
- Follow power and coolant distribution without confusing their connections with data cables.
- Verify the rack’s complete configuration and service dependencies before assigning workload capacity.
Science and assumptions
Component consistency
Total GPU count is the number of trays times GPUs per tray.
Symbols and units
- N_GPU
- GPU count within the chosen assembly boundary
GPUs
- N_tray
- Number of uniform compute trays
trays
- N_GPU/tray
- GPU count per such tray
GPUs/tray
Counts are dimensionless hardware quantities; they do not establish tokens/s.
Assumptions A uniform documented tray configuration; spare, disabled or service-reserved resources require separate accounting.
Engineering example
The documented 18 compute trays with four GPUs each account for 72 GPUs. This reconciles an inventory; it does not predict workload throughput, allocated capacity or electrical draw under every load.
Distinctions and tradeoffs
A rack described as one large AI machine is not literally one GPU die or one unlimited coherent memory.
Rack integration increases local connectivity but concentrates service and facility requirements; maintenance may need coordinated management of many components.
Useful capacity depends on the entire rack operating correctly, including networking and peripheral components that do not perform matrix multiplication.
Specifications and interfaces
- Named boundary
- DGX GB NVL72 rack reference
Preserve generation and system configuration in any numeric card.
- Independent services
- Compute, data, management, power and cooling
A working data link does not establish readiness of the other services.
Interfaces
- Facility services Input
- Electrical supply, coolant, cluster and management networks
- Rack workload resources Output
- Connected compute with telemetry and cooling paths
Check your understanding
Why preserve a named reference configuration?
- A rack name determines model quality
- Its counts and interfaces have a defined evidence boundary
- Every future rack must match it
Answer and reasoning
Its counts and interfaces have a defined evidence boundary
A reference makes the anatomy auditable while leaving alternatives and later revisions distinct.
Evidence and scope
The exact tray count is source-reported in the claims ledger. Mixed cooling and power details are specific to the reference, not universal rack standards.
Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.