{"id":"lpddr-memory","level":"server","name":"Grace LPDDR5X memory","shortName":"LPDDR5X","role":"Provides the Grace CPU’s local main memory with short, energy-conscious package and board connections.","description":"LPDDR5X is dynamic memory designed with strong attention to power efficiency. Grace uses it for CPU main memory, giving the host processor a local store distinct from the GPU’s HBM. Coherent CPU–GPU connectivity can simplify access to a shared address space, but it does not erase the physical difference between these memories or their paths. CPU memory and HBM are distinct reservoirs connected by an interconnect, so remote access still incurs data movement and path costs. Capacity options and rated bandwidth vary by module configuration. An early architecture estimate and a later product capacity describe different disclosures unless their configuration and conditions are explicitly reconciled. The useful question is which memory contains each part of the workload and how often that part must travel.","specs":[{"label":"GB200 CPU memory","value":"Up to 480 GB LPDDR5X","context":"Per Grace Blackwell Superchip product table; configurations vary."},{"label":"Published bandwidth","value":"Up to 512 GB/s","context":"Product maximum; do not assume every capacity option achieves it."},{"label":"Storage role","value":"CPU main memory","context":"Distinct from GPU HBM3E despite coherent access support."}],"interfaces":[{"name":"Grace requests","direction":"in","signal":"CPU reads, writes, and memory management"},{"name":"CPU data","direction":"out","signal":"Host working-set values"},{"name":"Coherent GPU access","direction":"out","signal":"Data crossing the CPU–GPU link when requested"}],"tradeoff":"Closer memory integration can improve energy efficiency but offers different expansion and service options than socketed DIMMs. Coherent access is convenient, yet locality still affects performance.","whyAI":"Host memory supports preparation, orchestration, and data that do not reside permanently in HBM. Efficient placement avoids turning the CPU–GPU link into an unnecessary bottleneck.","related":["host-cpu","grace-blackwell-superchip","ddr-memory","hbm-stack"],"sources":[{"title":"NVIDIA GB200 NVL72 specifications","url":"https://www.nvidia.com/en-us/data-center/gb200-nvl72/"},{"title":"NVIDIA: Inside Grace CPU architecture","url":"https://developer.nvidia.com/blog/inside-nvidia-grace-cpu-nvidia-amps-up-superchip-engineering-for-hpc-and-ai/"},{"title":"NVIDIA: Grace CPU Superchip announcement","url":"https://nvidianews.nvidia.com/news/nvidia-introduces-grace-cpu-superchip"}],"scale":"Memory packages near the Grace CPU; exact module layout varies","kind":"two-memory-reservoirs","mechanism":["Grace keeps CPU-resident program and data state in its local LPDDR5X memory subsystem.","The CPU reads and updates that state through its memory controllers.","When GPU work needs data located there, the coherent interconnect provides a path while preserving the distinction between physical memory locations."],"misconception":"A coherent address space does not merge CPU LPDDR and GPU HBM into one memory with identical performance. The physical route to each location still matters.","check":{"question":"Why distinguish CPU LPDDR from GPU HBM in a coherent system?","options":["Coherence means no bits ever move between components.","They are the same physical memory chips under two names.","They have different capacities, bandwidths, and access paths."],"answer":2,"explanation":"Coherence simplifies some programming tasks, but good placement still avoids unnecessary transfers between physically different memory systems."},"branch":"Host platform","subbranch":"CPU main memory","science":[{"title":"Coherence does not remove path bottlenecks","equation":"Bremote ≤ min(Bsource-memory, Binterconnect, Bdestination)","explanation":"Each B is effective payload bandwidth at the relevant one-direction path boundary. Reading CPU-resident data from a GPU must traverse both the CPU memory subsystem and the interconnect, even when a coherent address space makes that access convenient. Capacity can be shared logically without turning every memory location into an equally fast physical resource.","assumptions":"A serial-path upper bound for sustained transfers with adequate concurrency. Cache hits, compression, competing traffic, routing, and write/read direction change the applicable boundaries; do not substitute a summed bidirectional rating for one-way payload bandwidth."}],"history":[{"year":2022,"title":"Grace CPU Superchip announces server LPDDR5X integration","significance":"The March 2022 Grace CPU Superchip announcement described an LPDDR5X memory subsystem with error correction in a data-center CPU platform. This is a Grace platform milestone, not the invention of LPDDR memory.","source":{"title":"NVIDIA: Grace CPU Superchip announcement","url":"https://nvidianews.nvidia.com/news/nvidia-introduces-grace-cpu-superchip"}}],"prerequisites":["ddr-memory","hbm-stack"],"learningObjective":"Identify the limiting bandwidth along a remote CPU-memory path instead of treating coherent capacity as local HBM.","engineeringExample":"Synthetic one-way path: effective source memory, link and destination limits of 300, 200 and 500 GB/s give a ceiling of min(300, 200, 500) = 200 GB/s. Moving 20 GB therefore takes at least 0.1 s before access startup or contention. Raising only destination bandwidth cannot improve that bound.","evidenceNotes":"These rates are synthetic, not Grace specifications. The linked architecture documents establish distinct CPU memory and HBM paths; payload direction, configuration, cache effects and attained bandwidth must be specified.","labId":"inference-placement","scope":"Place read-only weights and mutable KV across specified tiers and inspect transfer paths; no HBF shipping-system benchmark or universal memory semantics are implied.","investigationId":"execution","labScope":"Place read-only weights and mutable KV across specified tiers and inspect transfer paths; no HBF shipping-system benchmark or universal memory semantics are implied."}