The mechanism
An NVMe SSD combines nonvolatile NAND flash, a controller, firmware, and a fast host interface. It stores data persistently rather than providing the byte-addressable working memory used directly by arithmetic units. The controller maps logical blocks to flash locations, handles error management, and schedules reads and writes across internal parallelism. Large sequential transfers and small random accesses stress different parts of the device, so one speed number cannot describe every workload. Loading a checkpoint moves persistent data into working memory; repeated tensor computation then uses that working state, with different access and lifetime requirements. Supported direct-storage paths can reduce extra copies, but the storage device still has its own latency, bandwidth, and endurance limits.
- The host issues block requests through the NVMe protocol to the SSD controller.
- The controller maps those requests onto flash operations and uses internal parallelism and error handling to read or write data.
- Data are loaded into working memory for computation, while completed checkpoints can be committed back to persistent storage.
Science and assumptions
High IOPS requires enough outstanding work
Average queue depth is approximately completed operations per second times average operation latency.
Symbols and units
- QD_average
- Average operations in flight
operations
- IOPS
- Completed I/O operation rate
operations/s
- L_average
- Mean operation latency at the same boundary
s
Queue depth is the average number of operations in flight, IOPS is completed operations per second, and L is average latency in seconds. A drive capable of high throughput can remain underused if the application submits one request and waits before issuing the next. Increasing concurrency helps only until the storage, host link, or workload becomes the bottleneck.
Assumptions Little’s law applied to a stable workload with a consistent measurement boundary. Latency generally changes with load, so the relation does not promise constant latency at arbitrary queue depth; read size, write mix, and durability requirements matter.
Engineering example
Synthetic stable workload: 100,000 IOPS at 200 μs average latency implies 100,000 × 0.0002 = 20 average outstanding operations. Doubling submitted queue depth does not imply double IOPS, because latency and the limiting resource may also change.
Distinctions and tradeoffs
NVMe storage capacity is not interchangeable with HBM capacity. Storage persists without power but has a different access model and much higher access cost.
Storage offers durable capacity but is far slower than on-chip memory and HBM. Checkpoint frequency trades recovery effort against write traffic and interruption to useful work.
Training begins with stored data and produces checkpoints that must survive failures. Serving systems also need to load and update model artifacts reliably.
Specifications and interfaces
- Micron 9550 example
- Up to 14.0 GB/s sequential read
Vendor-rated maximum; capacity, host, and test conditions matter.
- Sequential write
- Up to 10.0 GB/s
Micron 9550 selected configurations; not all capacities reach this.
- Host protocol
- NVMe over PCIe Gen5
The protocol and physical interface are distinct layers.
Interfaces
- Read/write commands Input
- Logical block requests
- Persistent data Output
- Dataset batches, model files, and checkpoints
- Checkpoint writes Input
- Training state committed for recovery
Check your understanding
Why is a saved checkpoint useful after a node loses power?
- It remains in a powered-off GPU register.
- It changes the physical number of Tensor Cores.
- A nonvolatile storage copy survives when volatile working memory is lost.
Answer and reasoning
A nonvolatile storage copy survives when volatile working memory is lost.
Checkpoints persist the state needed for recovery. Their usefulness depends on what was saved and how recently it was committed.
Evidence and scope
This is a steady-state queueing identity applied to the storage boundary, as explained by the lesson. I/O size, read/write mix, persistence requirements and latency definition must match; no product performance is inferred.
Use the manufacturing investigation to trace compatible inputs, assembly and operating constraints; no current standalone calculator models this lesson’s full mechanism.