The mechanism
LPDDR5X is dynamic memory designed with strong attention to power efficiency. Grace uses it for CPU main memory, giving the host processor a local store distinct from the GPU’s HBM. Coherent CPU–GPU connectivity can simplify access to a shared address space, but it does not erase the physical difference between these memories or their paths. CPU memory and HBM are distinct reservoirs connected by an interconnect, so remote access still incurs data movement and path costs. Capacity options and rated bandwidth vary by module configuration. An early architecture estimate and a later product capacity describe different disclosures unless their configuration and conditions are explicitly reconciled. The useful question is which memory contains each part of the workload and how often that part must travel.
- Grace keeps CPU-resident program and data state in its local LPDDR5X memory subsystem.
- The CPU reads and updates that state through its memory controllers.
- When GPU work needs data located there, the coherent interconnect provides a path while preserving the distinction between physical memory locations.
Science and assumptions
Coherence does not remove path bottlenecks
Remote-access bandwidth cannot exceed the smallest effective bandwidth of the source memory, interconnect and destination.
Symbols and units
- B_remote
- Sustained remote-access payload rate
bytes/s
- B_source memory
- Effective source-memory rate in the selected direction
bytes/s
- B_interconnect
- Effective intervening-link rate in that direction
bytes/s
- B_destination
- Effective receiving-boundary rate
bytes/s
Each B is effective payload bandwidth at the relevant one-direction path boundary. Reading CPU-resident data from a GPU must traverse both the CPU memory subsystem and the interconnect, even when a coherent address space makes that access convenient. Capacity can be shared logically without turning every memory location into an equally fast physical resource.
Assumptions A serial-path upper bound for sustained transfers with adequate concurrency. Cache hits, compression, competing traffic, routing, and write/read direction change the applicable boundaries; do not substitute a summed bidirectional rating for one-way payload bandwidth.
Engineering example
Synthetic one-way path: effective source memory, link and destination limits of 300, 200 and 500 GB/s give a ceiling of min(300, 200, 500) = 200 GB/s. Moving 20 GB therefore takes at least 0.1 s before access startup or contention. Raising only destination bandwidth cannot improve that bound.
Distinctions and tradeoffs
A coherent address space does not merge CPU LPDDR and GPU HBM into one memory with identical performance. The physical route to each location still matters.
Closer memory integration can improve energy efficiency but offers different expansion and service options than socketed DIMMs. Coherent access is convenient, yet locality still affects performance.
Host memory supports preparation, orchestration, and data that do not reside permanently in HBM. Efficient placement avoids turning the CPU–GPU link into an unnecessary bottleneck.
Specifications and interfaces
- GB200 CPU memory
- Up to 480 GB LPDDR5X
Per Grace Blackwell Superchip product table; configurations vary.
- Published bandwidth
- Up to 512 GB/s
Product maximum; do not assume every capacity option achieves it.
- Storage role
- CPU main memory
Distinct from GPU HBM3E despite coherent access support.
Interfaces
- Grace requests Input
- CPU reads, writes, and memory management
- CPU data Output
- Host working-set values
- Coherent GPU access Output
- Data crossing the CPU–GPU link when requested
Check your understanding
Why distinguish CPU LPDDR from GPU HBM in a coherent system?
- Coherence means no bits ever move between components.
- They are the same physical memory chips under two names.
- They have different capacities, bandwidths, and access paths.
Answer and reasoning
They have different capacities, bandwidths, and access paths.
Coherence simplifies some programming tasks, but good placement still avoids unnecessary transfers between physically different memory systems.
Historical context
2022 · Grace CPU Superchip announces server LPDDR5X integration
The March 2022 Grace CPU Superchip announcement described an LPDDR5X memory subsystem with error correction in a data-center CPU platform. This is a Grace platform milestone, not the invention of LPDDR memory.
Evidence and scope
These rates are synthetic, not Grace specifications. The linked architecture documents establish distinct CPU memory and HBM paths; payload direction, configuration, cache effects and attained bandwidth must be specified.
Place read-only weights and mutable KV across specified tiers and inspect transfer paths; no HBF shipping-system benchmark or universal memory semantics are implied.