{
  "schemaVersion": 1,
  "id": "atlas-lesson-hbm-controller",
  "title": "HBM controller and PHY",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Schedules DRAM commands and translates on-chip requests into the electrical signals of the memory interface.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/hbm-controller/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Memory hierarchy",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "It is a dimensional estimate, not a product channel count.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/hbm-controller-bf096ea73b5c5207.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#hbm-controller",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "0039466590a5343bff29e7cea62b9fd7c814a122e79f3950c33c828425a18230",
  "contentLength": 6158,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-hbm-controller.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-hbm-controller.md",
  "sections": [
    {
      "id": "section-1",
      "title": "HBM controller and PHY",
      "start": 33,
      "end": 394
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 394,
      "end": 1626
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1626,
      "end": 1654
    },
    {
      "id": "section-4",
      "title": "Concurrent requests cover memory latency",
      "start": 1654,
      "end": 2732
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2732,
      "end": 2899
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2899,
      "end": 3452
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3452,
      "end": 3864
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3864,
      "end": 4044
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4044,
      "end": 4073
    },
    {
      "id": "section-10",
      "title": "What is the difference between the controller and the PHY?",
      "start": 4073,
      "end": 4577
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4577,
      "end": 4813
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4813,
      "end": 5586
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5586,
      "end": 5609
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5609,
      "end": 5799
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5799,
      "end": 6158
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#hbm-controller"
    },
    {
      "title": "Micron HBM3E product brief",
      "url": "https://assets.micron.com/adobe/assets/urn%3Aaaid%3Aaem%3Ab710d8f2-7f66-44c1-a234-456e2b986347/renditions/original/as/hbm3e-product-brief.pdf"
    },
    {
      "title": "NVIDIA GB200 NVL72 specifications",
      "url": "https://www.nvidia.com/en-us/data-center/gb200-nvl72/"
    },
    {
      "title": "Intel: Memory performance, latency, and bandwidth",
      "url": "https://www.intel.com/content/www/us/en/developer/articles/technical/memory-performance-in-a-nutshell.html"
    },
    {
      "title": "John D. C. Little (1961): A Proof for the Queuing Formula L = λW",
      "url": "https://pubsonline.informs.org/doi/abs/10.1287/opre.9.3.383"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/hbm-controller-bf096ea73b5c5207.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "DRAM die and memory banks",
      "url": "https://semiaifoundry.com/learn/atlas/dram-die/"
    },
    {
      "title": "On-chip metal and via hierarchy",
      "url": "https://semiaifoundry.com/learn/atlas/interconnect/"
    },
    {
      "title": "High-bandwidth memory stack",
      "url": "https://semiaifoundry.com/learn/atlas/hbm-stack/"
    },
    {
      "title": "Interposer and redistribution layers",
      "url": "https://semiaifoundry.com/learn/atlas/interposer/"
    },
    {
      "title": "Shared L2 cache",
      "url": "https://semiaifoundry.com/learn/atlas/l2-cache/"
    },
    {
      "title": "On-chip interconnect fabric",
      "url": "https://semiaifoundry.com/learn/atlas/on-chip-network/"
    }
  ],
  "recordHash": "3b5bd23a50ceaa519d38794dc228b4cb127141729eb0fb32f61f28991eeb2f09",
  "contentFormat": "text/markdown",
  "content": "Memory hierarchy · Atlas v1.3.0\n\n# HBM controller and PHY\n\nSchedules DRAM commands and translates on-chip requests into the electrical signals of the memory interface.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#hbm-controller)Die-edge interface blocks and millimeter package links; schematic layout\n\nLearning objectiveTranslate a bandwidth target into a required interface count.\n\n## The mechanism\n\nA memory controller turns requests for addresses into commands that the DRAM organization can execute. It coordinates reads, writes, timing constraints, and maintenance such as refresh. The PHY is the physical interface circuitry that drives and receives signals across package wires. These are distinct jobs: the controller manages the protocol and schedule, while the PHY makes the bits electrically reliable. Many parallel wires provide aggregate HBM bandwidth without demanding that each wire run at the highest possible rate. A request passes through controller queues and bank selection before its data crosses the memory interface and package wiring. Read responses return through that path to the requesting work. Published stack bandwidth is an upper-bound interface capability, not the sustained bandwidth of every access pattern.\n\n1.  The controller accepts memory requests, maps addresses to memory resources, and schedules legal operations around timing constraints.\n2.  The PHY drives or receives many electrical signals across the package interface to the selected HBM stack.\n3.  Read data return to the cache and fabric, while refresh and other maintenance continue alongside application traffic.\n\n## Science and assumptions\n\n### Concurrent requests cover memory latency\n\n$$\nN_{\\mathrm{needed}}\\approx\\frac{B_{\\mathrm{target}}L}{s}\n$$\n\nRequired request concurrency is approximately target bandwidth times latency divided by useful bytes per request.\n\nSymbols and units\n\nN\\_needed\n\nAverage independent requests needed in flight\n\nrequests\n\nB\\_target\n\nTarget useful byte rate\n\nbytes/s\n\nL\n\nMean request latency\n\ns\n\ns\n\nUseful bytes per completed request\n\nbytes/request\n\nBtarget is the desired useful bandwidth in bytes per second, L is average request latency in seconds, and s is useful bytes per completed request. The result estimates how many independent requests must be outstanding. Sequentially dependent loads may fail to expose that concurrency even when the memory stack advertises ample bandwidth; controller queues cannot create independence absent from the program.\n\nAssumptions Steady-state Little’s-law estimate with a consistent request size and measured latency. Bank timing, refresh, arbitration, and protocol overhead determine the achievable B and L; no vendor queue capacity is assumed.\n\n## Engineering example\n\nFor a target byte rate B and per-link latency L, $N_{\\mathrm{needed}}≈BL/s$ is a sizing relation; channel scheduling and PHY overhead remain.\n\n## Distinctions and tradeoffs\n\nRated aggregate HBM bandwidth is not the speed of every individual access. Bank conflicts, transfer patterns, and protocol timing affect sustained throughput.\n\nWider interfaces increase bandwidth but require dense package routing and more connections. Poorly localized or conflicting accesses can achieve much less than the aggregate peak.\n\nDecoding a token often streams many model weights, so memory delivery can dominate latency. HBM controllers connect large model state to the much smaller arithmetic neighborhoods.\n\n## Specifications and interfaces\n\nHBM3E interface width\n\n1,024 data bits per stack\n\nHBM3E wide-interface organization; excludes command, clock, power, and ground connections.\n\nReference GPU memory bandwidth\n\n8 TB/s per GPU\n\nDerived from the GB200 product table’s 16 TB/s per two-GPU Superchip.\n\nMemory access\n\nRead/write commands plus refresh\n\nProtocol scheduling and physical signaling are separate functions.\n\n### Interfaces\n\nMemory requests Input\n\nAddresses and data from L2/fabric\n\nDRAM commands Output\n\nBank/row/column operations and writes\n\nRead data Input\n\nParallel data from HBM PHY\n\n## Check your understanding\n\n### What is the difference between the controller and the PHY?\n\n1.  The controller schedules memory operations; the PHY implements electrical signaling.\n2.  The controller stores all weights; the PHY performs attention.\n3.  They are two names for an NVMe drive.\n\nAnswer and reasoning\n\nThe controller schedules memory operations; the PHY implements electrical signaling.\n\nProtocol scheduling and physical bit transmission are distinct functions, even when integrated into one memory-interface subsystem.\n\n## Evidence and scope\n\nIt is a dimensional estimate, not a product channel count.\n\nUse the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.\n\n## Sources\n\n-   [Micron HBM3E product brief](https://assets.micron.com/adobe/assets/urn%3Aaaid%3Aaem%3Ab710d8f2-7f66-44c1-a234-456e2b986347/renditions/original/as/hbm3e-product-brief.pdf)\n-   [NVIDIA GB200 NVL72 specifications](https://www.nvidia.com/en-us/data-center/gb200-nvl72/)\n-   [Intel: Memory performance, latency, and bandwidth](https://www.intel.com/content/www/us/en/developer/articles/technical/memory-performance-in-a-nutshell.html)\n-   [John D. C. Little (1961): A Proof for the Queuing Formula L = λW](https://pubsonline.informs.org/doi/abs/10.1287/opre.9.3.383)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/hbm-controller-bf096ea73b5c5207.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [DRAM die and memory banks](https://semiaifoundry.com/learn/atlas/dram-die/)\n-   [On-chip metal and via hierarchy](https://semiaifoundry.com/learn/atlas/interconnect/)\n\n### Related lessons\n\n-   [High-bandwidth memory stack](https://semiaifoundry.com/learn/atlas/hbm-stack/)\n-   [Interposer and redistribution layers](https://semiaifoundry.com/learn/atlas/interposer/)\n-   [Shared L2 cache](https://semiaifoundry.com/learn/atlas/l2-cache/)\n-   [On-chip interconnect fabric](https://semiaifoundry.com/learn/atlas/on-chip-network/)"
}