# Dataset and checkpoint storage hierarchy

Canonical: https://semiaifoundry.com/learn/atlas/storage/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Data pipeline · Atlas v1.3.0

# Dataset and checkpoint storage hierarchy

Persists training data, model state, and results beyond accelerator working memory.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#storage)Device-to-cluster storage hierarchy; capacities are workload-dependent

Learning objectiveSeparate bytes read from storage from model working-set memory.

## The mechanism

AI storage provides the durable data behind repeated computation. Training reads samples and metadata, while checkpointing writes snapshots of model and optimizer state. These workloads stress different properties: many small operations, high sequential bandwidth, capacity, and recovery behavior may all matter. Local storage can stage data near a worker, while shared systems provide access across jobs and nodes. The atlas presents those layers separately from GPU HBM and CPU memory; storing a dataset does not mean every byte fits in accelerator memory at once. A well-supplied compute pipeline overlaps reading and preparation with execution. If storage cannot meet the workload's access pattern, expensive arithmetic hardware can wait idle.

1.  Persist datasets and model-state files in suitable storage tiers.
2.  Stage or stream the needed portions to active workers.
3.  Write results and recoverable checkpoints back to durable storage.

## Science and assumptions

### Bandwidth and I/O operations

$$
B\approx\mathrm{IOPS}\,S_{\mathrm{IO}}
$$

Average bandwidth is approximately I/O operations per second times mean bytes per operation.

Symbols and units

B

Average transferred byte rate

bytes/s

IOPS

Completed I/O rate over the selected workload

operations/s

S\_IO

Mean transferred size per counted operation

bytes/operation

For a workload with average transfer size $S_{IO}$, operations per second and transfer size together determine an approximate byte rate. Small random requests and large sequential transfers stress different limits.

Assumptions Average-rate relation over a defined workload; caching, metadata operations, concurrency, and latency alter observed performance.

## Engineering example

If 10 GB/s feeds a preprocessing stage for 60 seconds, 600 GB crosses that boundary; it need not reside in HBM simultaneously.

## Distinctions and tradeoffs

Storage capacity is not the same as storage throughput, and durable storage is not GPU working memory.

Caching and staging improve locality. Capacity, consistency, and recovery requirements add complexity.

Training needs a sustained supply of examples and recoverable state. Storage performance influences useful accelerator utilization.

## Specifications and interfaces

Durable contents

Datasets, checkpoints, outputs

Different access patterns may require different designs.

Performance dimensions

Bandwidth, IOPS, latency

Capacity alone does not specify performance.

Hierarchy

Local staging + shared persistence

A conceptual architecture, not one mandatory product.

### Interfaces

Data ingestion and checkpoints Input

Persistent records and model-state snapshots

Workers and recovery Output

Samples and restored state

## Check your understanding

### Why can a large storage system still starve GPUs?

1.  Its access latency or throughput may not match the workload
2.  Large capacity guarantees unlimited I/O
3.  GPUs cannot read data from any software pipeline

Answer and reasoning

Its access latency or throughput may not match the workload

Capacity describes how much fits. Sustained delivery depends on access patterns and the performance of the full data path.

## Evidence and scope

IOPS and sequential bandwidth have different workload meanings.

Use the manufacturing investigation to trace compatible inputs, assembly and operating constraints; no current standalone calculator models this lesson’s full mechanism.

## Sources

-   [NVIDIA — High-Performance Storage Architecture](https://docs.nvidia.com/dgx-superpod/reference-architecture/scalable-infrastructure-b300/latest/storage-architecture.html)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/storage-98e34d0e5e2a0a06.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [NVMe solid-state storage](https://semiaifoundry.com/learn/atlas/nvme-storage/)

### Related lessons

-   [Distributed checkpoint and recovery state](https://semiaifoundry.com/learn/atlas/checkpoint/)
-   [Leaf switch and endpoint network](https://semiaifoundry.com/learn/atlas/leaf-switch/)
-   [Cluster scheduler and resource allocation](https://semiaifoundry.com/learn/atlas/scheduler/)
-   [Inference serving and request scheduling](https://semiaifoundry.com/learn/atlas/serving/)
-   [Training data pipeline](https://semiaifoundry.com/learn/atlas/training-data/)