{
  "schemaVersion": 1,
  "id": "atlas-lesson-memory-to-hall-propagation",
  "title": "From memory placement to network, power and cooling",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Trace how a tensor-placement decision changes link demand and useful service at the hall boundary.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/memory-to-hall-propagation/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Infrastructure",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "All numerical values are scenario inputs. No relationship between a megawatt and tokens is assumed without a named workload. CRG provides an attributed way to organize these realization boundaries; the arithmetic is a resource-accounting model.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/memory-to-hall-propagation-1502229faf4cfc16.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#memory-to-hall-propagation",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "1521593380dabb522766151d28118624e915648e7e02773a036506fb0c2d13ee",
  "contentLength": 6802,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-memory-to-hall-propagation.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-memory-to-hall-propagation.md",
  "sections": [
    {
      "id": "section-1",
      "title": "From memory placement to network, power and cooling",
      "start": 31,
      "end": 458
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 458,
      "end": 1679
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1679,
      "end": 1707
    },
    {
      "id": "section-4",
      "title": "Resource-normalized service bound",
      "start": 1707,
      "end": 2488
    },
    {
      "id": "section-5",
      "title": "Energy per accepted work",
      "start": 2488,
      "end": 3123
    },
    {
      "id": "section-6",
      "title": "Engineering example",
      "start": 3123,
      "end": 3422
    },
    {
      "id": "section-7",
      "title": "Distinctions and tradeoffs",
      "start": 3422,
      "end": 3828
    },
    {
      "id": "section-8",
      "title": "Specifications and interfaces",
      "start": 3828,
      "end": 4060
    },
    {
      "id": "section-9",
      "title": "Interfaces",
      "start": 4060,
      "end": 4246
    },
    {
      "id": "section-10",
      "title": "Check your understanding",
      "start": 4246,
      "end": 4275
    },
    {
      "id": "section-11",
      "title": "A placement change lowers HBM traffic but doubles remote transfers. What should be compared?",
      "start": 4275,
      "end": 4746
    },
    {
      "id": "section-12",
      "title": "Evidence and scope",
      "start": 4746,
      "end": 5183
    },
    {
      "id": "section-13",
      "title": "Sources",
      "start": 5183,
      "end": 5943
    },
    {
      "id": "section-14",
      "title": "Continue exploring",
      "start": 5943,
      "end": 5966
    },
    {
      "id": "section-15",
      "title": "Foundations",
      "start": 5966,
      "end": 6328
    },
    {
      "id": "section-16",
      "title": "Related lessons",
      "start": 6328,
      "end": 6802
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#memory-to-hall-propagation"
    },
    {
      "title": "NVIDIA — DGX GB rack hardware guide",
      "url": "https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"
    },
    {
      "title": "MIT OpenCourseWare — Introduction to Heat Transfer",
      "url": "https://ocw.mit.edu/courses/2-051-introduction-to-heat-transfer-fall-2015/"
    },
    {
      "title": "NVIDIA — Mastering LLM Techniques: Inference Optimization",
      "url": "https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/"
    },
    {
      "title": "Semi AI Foundry — Communication Theory of Computation Realization",
      "url": "https://semiaifoundry.com/updates/communication-theory-of-computation-realization/"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/memory-to-hall-propagation-1502229faf4cfc16.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Place the working set, not just the model",
      "url": "https://semiaifoundry.com/learn/atlas/memory-working-set-placement/"
    },
    {
      "title": "Collective traffic and congestion",
      "url": "https://semiaifoundry.com/learn/atlas/fabric-collective-traffic/"
    },
    {
      "title": "Cold plates, manifolds and separated coolant loops",
      "url": "https://semiaifoundry.com/learn/atlas/coolant-loop-separation/"
    },
    {
      "title": "Qualify the memory system and retain useful work",
      "url": "https://semiaifoundry.com/learn/atlas/memory-system-qualification/"
    },
    {
      "title": "Package and system preserve the contract",
      "url": "https://semiaifoundry.com/learn/atlas/crg-package-system-closure/"
    },
    {
      "title": "Verification closes the realization loop",
      "url": "https://semiaifoundry.com/learn/atlas/crg-verification-loop/"
    },
    {
      "title": "Commissioning a usable AI cluster",
      "url": "https://semiaifoundry.com/learn/atlas/cluster-commissioning-gates/"
    }
  ],
  "recordHash": "d81444484bd500edc50e2d2675591d7761b71fdcd31be6200a09fb28fd86f6fa",
  "contentFormat": "text/markdown",
  "content": "Infrastructure · Atlas v1.3.0\n\n# From memory placement to network, power and cooling\n\nTrace how a tensor-placement decision changes link demand and useful service at the hall boundary.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#memory-to-hall-propagation)Package/link topology → rack services → facility accounting; schematic\n\nLearning objectiveTrace how a tensor-placement decision changes link demand and useful service at the hall boundary.\n\n## The mechanism\n\nMoving state out of HBM can free capacity while creating traffic on package, CPU–GPU, PCIe, storage or network paths. Every path has its own direction, contention domain and service overhead. A larger batch may amortize weight reads while increasing KV and queueing; disaggregated serving may improve resource matching while adding KV transfers. The resulting workload changes electrical demand and heat distribution, but fixed conversion losses and idle power remain. A credible comparison closes memory, movement, compute, facility services and response-quality requirements on the same time interval.\n\n1.  Start with the working-set plan and trace every moved byte across named physical interfaces.\n2.  Aggregate traffic by link and direction, separating expert-weight reads, token routing, KV movement and checkpoints.\n3.  Find a workload-rate bound from compute, memory and network demand; use a schedule to expose dependencies and overlap.\n4.  Apply measured or explicitly assumed power at the chosen IT boundary and the facility overhead at the same interval.\n5.  Report accepted outputs meeting quality and latency targets, and identify whether the constraint moved after the placement change.\n\n## Science and assumptions\n\n### Resource-normalized service bound\n\n$$\nR\\leq\\min\\left(\\frac{F}{f},\\frac{B_{\\mathrm{mem}}}{v_{\\mathrm{mem}}},\\frac{B_{\\mathrm{net}}}{v_{\\mathrm{net}}}\\right)\n$$\n\nR ≤ min(F/f, Bmem/vmem, Bnet/vnet)\n\nSymbols and units\n\nR\n\nCompleted defined work units per second\n\nwork/s\n\nF\n\nUsable compute rate\n\noperations/s\n\nf\n\nOperations per work unit\n\noperations/work\n\nBmem\n\nUsable memory bandwidth\n\nbytes/s\n\nvmem\n\nMemory bytes per work unit\n\nbytes/work\n\nBnet\n\nUsable network bandwidth\n\nbytes/s\n\nvnet\n\nNetwork bytes per work unit\n\nbytes/work\n\nNormalize each resource to the same defined unit of work before taking the minimum.\n\nAssumptions Steady aggregate upper bound, independent capacities and positive demand; omit an unused resource. Serial phases, latency and skew can lower performance.\n\n### Energy per accepted work\n\n$$\ne=\\frac{P_{\\mathrm{IT}}\\,\\mathrm{PUE}}{R_{\\mathrm{accepted}}}\n$$\n\ne = PIT PUE / Raccepted\n\nSymbols and units\n\ne\n\nFacility energy per accepted work unit\n\nJ/work\n\nPIT\n\nMean IT equipment power\n\nW\n\nPUE\n\nFacility/IT energy ratio for that interval\n\ndimensionless\n\nRaccepted\n\nAccepted work rate\n\nwork/s\n\nUse average IT power and accepted-work rate over the same measured interval; PUE supplies the facility boundary conversion.\n\nAssumptions Stable or interval-averaged boundaries; Raccepted must be positive and enforce a defined quality/latency criterion. This is not a universal joules-per-token constant.\n\n## Engineering example\n\nSynthetic service: compute supports 300 work/s, memory 200 work/s and network 100 work/s; the simple bound is 100 work/s. Halving memory traffic alone leaves that bound unchanged. At 12 kW IT, PUE 1.2 and 90 accepted work/s, facility energy is 160 J per accepted work unit.\n\n## Distinctions and tradeoffs\n\nA local improvement can move the bottleneck to another interface or increase work elsewhere in the system.\n\nCapacity offload can reduce device count but increase transfer stalls or energy elsewhere; no component metric alone predicts the net result.\n\nThis closes the wafer → package → memory → hall → model journey through actual work rather than installed hardware totals.\n\n## Specifications and interfaces\n\nSame work unit\n\nToken, request or training update\n\nDefine model, precision, quality and latency target.\n\nBoundary\n\nLink → rack → IT → facility\n\nDo not mix package power with facility-wide outputs.\n\n### Interfaces\n\nMemory/runtime policy Input\n\nResident state, misses, transfer volume and schedule\n\nHall service result Output\n\nAccepted work, power, cooling and deployment requirements\n\n## Check your understanding\n\n### A placement change lowers HBM traffic but doubles remote transfers. What should be compared?\n\n1.  The complete workload at consistent quality, latency and energy boundaries\n2.  Only HBM bandwidth utilization\n3.  Only the capacity printed on the memory package\n\nAnswer and reasoning\n\nThe complete workload at consistent quality, latency and energy boundaries\n\nA local improvement can move the bottleneck to another interface or increase work elsewhere in the system.\n\n## Evidence and scope\n\nAll numerical values are scenario inputs. No relationship between a megawatt and tokens is assumed without a named workload. CRG provides an attributed way to organize these realization boundaries; the arithmetic is a resource-accounting model.\n\nPropagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.\n\n## Sources\n\n-   [NVIDIA — DGX GB rack hardware guide](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html)\n-   [MIT OpenCourseWare — Introduction to Heat Transfer](https://ocw.mit.edu/courses/2-051-introduction-to-heat-transfer-fall-2015/)\n-   [NVIDIA — Mastering LLM Techniques: Inference Optimization](https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/)\n-   [Semi AI Foundry — Communication Theory of Computation Realization](https://semiaifoundry.com/updates/communication-theory-of-computation-realization/)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/memory-to-hall-propagation-1502229faf4cfc16.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Place the working set, not just the model](https://semiaifoundry.com/learn/atlas/memory-working-set-placement/)\n-   [Collective traffic and congestion](https://semiaifoundry.com/learn/atlas/fabric-collective-traffic/)\n-   [Cold plates, manifolds and separated coolant loops](https://semiaifoundry.com/learn/atlas/coolant-loop-separation/)\n\n### Related lessons\n\n-   [Qualify the memory system and retain useful work](https://semiaifoundry.com/learn/atlas/memory-system-qualification/)\n-   [Package and system preserve the contract](https://semiaifoundry.com/learn/atlas/crg-package-system-closure/)\n-   [Verification closes the realization loop](https://semiaifoundry.com/learn/atlas/crg-verification-loop/)\n-   [Commissioning a usable AI cluster](https://semiaifoundry.com/learn/atlas/cluster-commissioning-gates/)"
}