{"id":"memory-interface-choice","level":"compute","name":"Choosing SRAM, HBM, DDR, LPDDR or GDDR","shortName":"Choosing SRAM, HBM, DDR, LPDDR or GDDR","role":"Matches memory technology and package integration to a workload’s access pattern.","description":"Memory alternatives solve different constraints. SRAM integrates near logic and supports fast local access, but has lower bit density than DRAM. HBM uses stacked DRAM and many short package connections. DDR commonly supplies expandable host memory; LPDDR emphasizes mobile-style power and signaling choices and also appears in some server and custom-accelerator designs. GDDR uses high-rate interfaces in discrete memory packages, providing another accelerator memory route. Capacity, sustained bandwidth, power, board area, package complexity and supply all matter. A valid comparison fixes a generation and configuration instead of treating one family as universally faster or cheaper.","specs":[{"label":"Interface bandwidth","value":"Data pins × bit rate / 8","context":"Bytes/s before utilization losses; effective data rate already accounts for signaling transfers."},{"label":"Technology choice","value":"Memory device plus integration route","context":"Controller/PHY, package, board and cooling must support the chosen interface."}],"interfaces":[{"name":"Workload memory demand","direction":"in","signal":"Resident bytes, reuse, burst size and access concurrency"},{"name":"Memory subsystem","direction":"out","signal":"Delivered data under capacity, power and integration constraints"}],"tradeoff":"HBM’s wide nearby interface adds advanced package dependencies; discrete DRAM choices move more routing onto a board, while additional SRAM competes with compute area.","whyAI":"The right memory balance differs for a small reusable recommendation working set, a large model’s weights and rapidly growing request KV state.","related":["hbm-stack","ddr-memory","lpddr-memory","inference-custom-silicon"],"sources":[{"title":"Graphics memory","url":"https://www.micron.com/products/memory/graphics-memory"},{"title":"MTIA v1: Meta’s first-generation AI inference accelerator","url":"https://ai.meta.com/blog/meta-training-inference-accelerator-AI-MTIA/"}],"scale":"Register to storage hierarchy and product lifecycle","kind":"architecture","branch":"Custom silicon ecosystem","subbranch":"Memory choices and silicon economics","mechanism":["Determine which state must stay resident and which can be streamed without breaking the service target.","Estimate reuse and concurrent accesses so the required bandwidth is tied to useful data movement.","Evaluate local SRAM together with external DRAM rather than choosing one memory technology in isolation.","Budget controller, PHY, package and board resources for the chosen width and signaling rate.","Compare qualified configurations under the same workload while labeling price, supply and undocumented implementation details as unknown."],"misconception":"HBM is not simply DDR stacked higher, and high pin rate alone does not determine aggregate usable bandwidth.","check":{"question":"Which change can raise aggregate bandwidth without raising per-pin rate?","options":["Increasing the number of simultaneously active data pins","Increasing only stored capacity","Renaming memory as cache"],"answer":0,"explanation":"A wider interface transfers more bits in parallel, provided controllers, channels and traffic can use that width."},"science":[{"title":"Raw interface rate","equation":"Braw = w r / 8","explanation":"w is active data pins and r is effective bits/s per pin; Braw is bytes/s. Multiply by a measured efficiency to estimate usable bandwidth.","assumptions":"Do not double an already effective DDR data rate; command overhead, refresh and bank conflicts reduce realized throughput."}],"prerequisites":["memory-hierarchy-route","hbm-stack"],"learningObjective":"Compare memory families using a workload and complete interface boundary.","workstream":"W07","visualFamily":"memory-route","engineeringExample":"Synthetic interfaces: 256 data pins at 16 Gbit/s and 1,024 pins at 4 Gbit/s both yield 512 GB/s raw. Their package requirements, power and latency can still be very different.","evidenceNotes":"Pin counts and rates in the example are synthetic. No contemporary contract price, universal HBM advantage or interchangeable memory-controller compatibility is claimed.","labId":"inference-placement","scope":"Place read-only weights and mutable KV across specified tiers and inspect transfer paths; no HBF shipping-system benchmark or universal memory semantics are implied.","investigationId":"execution","labScope":"Place read-only weights and mutable KV across specified tiers and inspect transfer paths; no HBF shipping-system benchmark or universal memory semantics are implied."}