The mechanism
AI storage provides the durable data behind repeated computation. Training reads samples and metadata, while checkpointing writes snapshots of model and optimizer state. These workloads stress different properties: many small operations, high sequential bandwidth, capacity, and recovery behavior may all matter. Local storage can stage data near a worker, while shared systems provide access across jobs and nodes. The atlas presents those layers separately from GPU HBM and CPU memory; storing a dataset does not mean every byte fits in accelerator memory at once. A well-supplied compute pipeline overlaps reading and preparation with execution. If storage cannot meet the workload's access pattern, expensive arithmetic hardware can wait idle.
- Persist datasets and model-state files in suitable storage tiers.
- Stage or stream the needed portions to active workers.
- Write results and recoverable checkpoints back to durable storage.
Science and assumptions
Bandwidth and I/O operations
Average bandwidth is approximately I/O operations per second times mean bytes per operation.
Symbols and units
- B
- Average transferred byte rate
bytes/s
- IOPS
- Completed I/O rate over the selected workload
operations/s
- S_IO
- Mean transferred size per counted operation
bytes/operation
For a workload with average transfer size , operations per second and transfer size together determine an approximate byte rate. Small random requests and large sequential transfers stress different limits.
Assumptions Average-rate relation over a defined workload; caching, metadata operations, concurrency, and latency alter observed performance.
Engineering example
If 10 GB/s feeds a preprocessing stage for 60 seconds, 600 GB crosses that boundary; it need not reside in HBM simultaneously.
Distinctions and tradeoffs
Storage capacity is not the same as storage throughput, and durable storage is not GPU working memory.
Caching and staging improve locality. Capacity, consistency, and recovery requirements add complexity.
Training needs a sustained supply of examples and recoverable state. Storage performance influences useful accelerator utilization.
Specifications and interfaces
- Durable contents
- Datasets, checkpoints, outputs
Different access patterns may require different designs.
- Performance dimensions
- Bandwidth, IOPS, latency
Capacity alone does not specify performance.
- Hierarchy
- Local staging + shared persistence
A conceptual architecture, not one mandatory product.
Interfaces
- Data ingestion and checkpoints Input
- Persistent records and model-state snapshots
- Workers and recovery Output
- Samples and restored state
Check your understanding
Why can a large storage system still starve GPUs?
- Its access latency or throughput may not match the workload
- Large capacity guarantees unlimited I/O
- GPUs cannot read data from any software pipeline
Answer and reasoning
Its access latency or throughput may not match the workload
Capacity describes how much fits. Sustained delivery depends on access patterns and the performance of the full data path.
Evidence and scope
IOPS and sequential bandwidth have different workload meanings.
Use the manufacturing investigation to trace compatible inputs, assembly and operating constraints; no current standalone calculator models this lesson’s full mechanism.