{"id":"memory-to-hall-propagation","level":"infrastructure","name":"From memory placement to network, power and cooling","shortName":"From memory placement to network, power and cooling","role":"Trace how a tensor-placement decision changes link demand and useful service at the hall boundary.","description":"Moving state out of HBM can free capacity while creating traffic on package, CPU–GPU, PCIe, storage or network paths. Every path has its own direction, contention domain and service overhead. A larger batch may amortize weight reads while increasing KV and queueing; disaggregated serving may improve resource matching while adding KV transfers. The resulting workload changes electrical demand and heat distribution, but fixed conversion losses and idle power remain. A credible comparison closes memory, movement, compute, facility services and response-quality requirements on the same time interval.","mechanism":["Start with the working-set plan and trace every moved byte across named physical interfaces.","Aggregate traffic by link and direction, separating expert-weight reads, token routing, KV movement and checkpoints.","Find a workload-rate bound from compute, memory and network demand; use a schedule to expose dependencies and overlap.","Apply measured or explicitly assumed power at the chosen IT boundary and the facility overhead at the same interval.","Report accepted outputs meeting quality and latency targets, and identify whether the constraint moved after the placement change."],"science":[{"title":"Resource-normalized service bound","equation":"R ≤ min(F/f, Bmem/vmem, Bnet/vnet)","explanation":"Normalize each resource to the same defined unit of work before taking the minimum.","assumptions":"Steady aggregate upper bound, independent capacities and positive demand; omit an unused resource. Serial phases, latency and skew can lower performance."},{"title":"Energy per accepted work","equation":"e = PIT PUE / Raccepted","explanation":"Use average IT power and accepted-work rate over the same measured interval; PUE supplies the facility boundary conversion.","assumptions":"Stable or interval-averaged boundaries; Raccepted must be positive and enforce a defined quality/latency criterion. This is not a universal joules-per-token constant."}],"specs":[{"label":"Same work unit","value":"Token, request or training update","context":"Define model, precision, quality and latency target."},{"label":"Boundary","value":"Link → rack → IT → facility","context":"Do not mix package power with facility-wide outputs."}],"interfaces":[{"name":"Memory/runtime policy","direction":"in","signal":"Resident state, misses, transfer volume and schedule"},{"name":"Hall service result","direction":"out","signal":"Accepted work, power, cooling and deployment requirements"}],"tradeoff":"Capacity offload can reduce device count but increase transfer stalls or energy elsewhere; no component metric alone predicts the net result.","whyAI":"This closes the wafer → package → memory → hall → model journey through actual work rather than installed hardware totals.","prerequisites":["memory-working-set-placement","fabric-collective-traffic","coolant-loop-separation"],"related":["memory-system-qualification","crg-package-system-closure","crg-verification-loop","cluster-commissioning-gates"],"misconception":"A local improvement can move the bottleneck to another interface or increase work elsewhere in the system.","check":{"question":"A placement change lowers HBM traffic but doubles remote transfers. What should be compared?","options":["The complete workload at consistent quality, latency and energy boundaries","Only HBM bandwidth utilization","Only the capacity printed on the memory package"],"answer":0,"explanation":"A local improvement can move the bottleneck to another interface or increase work elsewhere in the system."},"engineeringExample":"Synthetic service: compute supports 300 work/s, memory 200 work/s and network 100 work/s; the simple bound is 100 work/s. Halving memory traffic alone leaves that bound unchanged. At 12 kW IT, PUE 1.2 and 90 accepted work/s, facility energy is 160 J per accepted work unit.","evidenceNotes":"All numerical values are scenario inputs. No relationship between a megawatt and tokens is assumed without a named workload. CRG provides an attributed way to organize these realization boundaries; the arithmetic is a resource-accounting model.","sources":[{"title":"NVIDIA — DGX GB rack hardware guide","url":"https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html"},{"title":"MIT OpenCourseWare — Introduction to Heat Transfer","url":"https://ocw.mit.edu/courses/2-051-introduction-to-heat-transfer-fall-2015/"},{"title":"NVIDIA — Mastering LLM Techniques: Inference Optimization","url":"https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/"},{"title":"Semi AI Foundry — Communication Theory of Computation Realization","url":"https://semiaifoundry.com/updates/communication-theory-of-computation-realization/"}],"learningObjective":"Trace how a tensor-placement decision changes link demand and useful service at the hall boundary.","workstream":"W09","visualFamily":"hall-system","scale":"Package/link topology → rack services → facility accounting; schematic","kind":"system","labId":"engineering","branch":"Infrastructure","subbranch":"Memory choices to commissioned useful work","scope":"Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included.","deepLabId":"hall","investigationId":"manufacturing","labScope":"Propagate a stated facility power budget, PUE, readiness and utilization to illustrative work; detailed conversion, battery, cooling and grid models are not included."}