# On-chip interconnect fabric Canonical: https://semiaifoundry.com/learn/atlas/on-chip-network/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Connectivity · Atlas v1.3.0 # On-chip interconnect fabric Routes data and control between compute units, cache slices, memory controllers, and I/O. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#on-chip-network)Millimeter-to-centimeter internal routes; illustrative topology Learning objectiveRelate in-flight data to rate and latency. ## The mechanism The on-chip fabric is the communication system within the silicon. Requests carry addresses, operation types, and data between blocks; arbitration decides which request uses a contested path. Wires have capacitance and resistance, so longer journeys consume time and energy even before leaving the package. A GPU can expose a unified address space while still having internal locality domains. An illustrative routing graph is distinct from an undisclosed physical topology: a tidy mesh is a teaching model, not evidence that Blackwell uses that exact network. Shared routes can create contention, and local versus remote cache access can use different paths; the chip’s communication behavior matters alongside its arithmetic. 1. A compute, cache, or I/O block injects an addressed transaction into the chip’s communication fabric. 2. Routing and arbitration carry that transaction toward its destination while managing competition and backpressure. 3. The response returns through the fabric; repeated or long-distance traffic consumes bandwidth, energy, and time. ## Science and assumptions ### Enough requests must be in flight $$ N_{\mathrm{inflight}}=\lambda L,\qquad M_{\mathrm{inflight}}\approx BL $$ Average in-flight requests equal request rate times latency. The corresponding in-flight byte amount is approximately byte throughput times latency. Symbols and units N\_inflight Average outstanding request count requests λ Completed request rate requests/s L Mean request time within the same boundary s M\_inflight Mean in-flight data amount under consistent payload accounting bytes B Sustained payload byte rate bytes/s Little’s law relates average in-flight request count N to completed requests per second λ and average request latency L. Multiplying byte throughput B by latency gives the corresponding amount of in-flight data. A fabric can have wide links yet remain underused if the workload or queues cannot sustain enough independent transactions to cover the round trip. Assumptions Stable steady-state averages with the same system boundary for arrivals, completions, and latency. The byte form assumes consistent payload accounting. It supplies a concurrency requirement, not a claim about Blackwell’s undisclosed queue depths. ## Engineering example At 2 TB/s and 100 ns of round-trip latency, the bandwidth-delay product is 200 kB of in-flight data before protocol overhead. ## Distinctions and tradeoffs The atlas’s routing graph is a teaching diagram, not a verified Blackwell mesh layout. A physical topology should not be inferred from the illustrative geometry. Wider routes and more buffering improve communication capacity but consume area and power. Placing related work and data near one another can reduce movement before any hardware is added. Thousands of concurrent tensor operations must obtain operands and exchange results. Internal communication determines whether the advertised arithmetic units remain busy. ## Specifications and interfaces Endpoints SMs, cache, memory, I/O Logical component classes; exact proprietary topology is not asserted. Address view Unified GPU memory space Physical locality can still affect movement cost. Constraint Bandwidth, latency, and energy No unsupported numerical internal-fabric rating is assigned. ### Interfaces Request injection Input Addressed transactions from compute or I/O Routing Output Packets or transactions to selected internal endpoints Backpressure Input Flow-control information from occupied resources ## Check your understanding ### Why does locality still matter inside one GPU? 1. Only data outside the rack require energy to move. 2. Internal wires and shared routes still incur delay, energy, and contention. 3. One GPU contains only one possible data route. Answer and reasoning Internal wires and shared routes still incur delay, energy, and contention. Even on one chip, communication has physical costs. Keeping related data and work nearby can reduce unnecessary traffic. ## Evidence and scope The relation assumes a stable stream and excludes topology-specific backpressure. Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism. ## Sources - [NVIDIA: GPU locality domains and L2 fabric](https://developer.nvidia.com/blog/accelerating-data-processing-with-nvidia-multi-instance-gpu-and-numa-node-localization/) - [Intel: Memory performance, latency, and bandwidth](https://www.intel.com/content/www/us/en/developer/articles/technical/memory-performance-in-a-nutshell.html) - [John D. C. Little (1961): A Proof for the Queuing Formula L = λW](https://pubsonline.informs.org/doi/abs/10.1287/opre.9.3.383) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/on-chip-network-2fef2e4eae57f640.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [On-chip metal and via hierarchy](https://semiaifoundry.com/learn/atlas/interconnect/) - [Shared L2 cache](https://semiaifoundry.com/learn/atlas/l2-cache/) ### Related lessons - [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/) - [HBM controller and PHY](https://semiaifoundry.com/learn/atlas/hbm-controller/) - [NVLink GPU interface](https://semiaifoundry.com/learn/atlas/nvlink-port/) - [GPU logic dies](https://semiaifoundry.com/learn/atlas/gpu-die/)