{
  "schemaVersion": 1,
  "id": "atlas-lesson-on-chip-network",
  "title": "On-chip interconnect fabric",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Routes data and control between compute units, cache slices, memory controllers, and I/O.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/on-chip-network/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Connectivity",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The relation assumes a stable stream and excludes topology-specific backpressure.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/on-chip-network-2fef2e4eae57f640.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#on-chip-network",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "5aeec86310e3cb22d7495f32e87389d6424b9cf675362bb3dd1bf1585a0797be",
  "contentLength": 5893,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-on-chip-network.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-on-chip-network.md",
  "sections": [
    {
      "id": "section-1",
      "title": "On-chip interconnect fabric",
      "start": 29,
      "end": 349
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 349,
      "end": 1439
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1439,
      "end": 1467
    },
    {
      "id": "section-4",
      "title": "Enough requests must be in flight",
      "start": 1467,
      "end": 2646
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2646,
      "end": 2797
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2797,
      "end": 3353
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3353,
      "end": 3692
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3692,
      "end": 3919
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3919,
      "end": 3948
    },
    {
      "id": "section-10",
      "title": "Why does locality still matter inside one GPU?",
      "start": 3948,
      "end": 4408
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4408,
      "end": 4667
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4667,
      "end": 5342
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5342,
      "end": 5365
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5365,
      "end": 5545
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5545,
      "end": 5893
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#on-chip-network"
    },
    {
      "title": "NVIDIA: GPU locality domains and L2 fabric",
      "url": "https://developer.nvidia.com/blog/accelerating-data-processing-with-nvidia-multi-instance-gpu-and-numa-node-localization/"
    },
    {
      "title": "Intel: Memory performance, latency, and bandwidth",
      "url": "https://www.intel.com/content/www/us/en/developer/articles/technical/memory-performance-in-a-nutshell.html"
    },
    {
      "title": "John D. C. Little (1961): A Proof for the Queuing Formula L = λW",
      "url": "https://pubsonline.informs.org/doi/abs/10.1287/opre.9.3.383"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/on-chip-network-2fef2e4eae57f640.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "On-chip metal and via hierarchy",
      "url": "https://semiaifoundry.com/learn/atlas/interconnect/"
    },
    {
      "title": "Shared L2 cache",
      "url": "https://semiaifoundry.com/learn/atlas/l2-cache/"
    },
    {
      "title": "Streaming multiprocessor",
      "url": "https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/"
    },
    {
      "title": "HBM controller and PHY",
      "url": "https://semiaifoundry.com/learn/atlas/hbm-controller/"
    },
    {
      "title": "NVLink GPU interface",
      "url": "https://semiaifoundry.com/learn/atlas/nvlink-port/"
    },
    {
      "title": "GPU logic dies",
      "url": "https://semiaifoundry.com/learn/atlas/gpu-die/"
    }
  ],
  "recordHash": "198ce3b829665171165f64b79ea9d909a1be2310c039eb71ac9b9494521b3314",
  "contentFormat": "text/markdown",
  "content": "Connectivity · Atlas v1.3.0\n\n# On-chip interconnect fabric\n\nRoutes data and control between compute units, cache slices, memory controllers, and I/O.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#on-chip-network)Millimeter-to-centimeter internal routes; illustrative topology\n\nLearning objectiveRelate in-flight data to rate and latency.\n\n## The mechanism\n\nThe on-chip fabric is the communication system within the silicon. Requests carry addresses, operation types, and data between blocks; arbitration decides which request uses a contested path. Wires have capacitance and resistance, so longer journeys consume time and energy even before leaving the package. A GPU can expose a unified address space while still having internal locality domains. An illustrative routing graph is distinct from an undisclosed physical topology: a tidy mesh is a teaching model, not evidence that Blackwell uses that exact network. Shared routes can create contention, and local versus remote cache access can use different paths; the chip’s communication behavior matters alongside its arithmetic.\n\n1.  A compute, cache, or I/O block injects an addressed transaction into the chip’s communication fabric.\n2.  Routing and arbitration carry that transaction toward its destination while managing competition and backpressure.\n3.  The response returns through the fabric; repeated or long-distance traffic consumes bandwidth, energy, and time.\n\n## Science and assumptions\n\n### Enough requests must be in flight\n\n$$\nN_{\\mathrm{inflight}}=\\lambda L,\\qquad M_{\\mathrm{inflight}}\\approx BL\n$$\n\nAverage in-flight requests equal request rate times latency. The corresponding in-flight byte amount is approximately byte throughput times latency.\n\nSymbols and units\n\nN\\_inflight\n\nAverage outstanding request count\n\nrequests\n\nλ\n\nCompleted request rate\n\nrequests/s\n\nL\n\nMean request time within the same boundary\n\ns\n\nM\\_inflight\n\nMean in-flight data amount under consistent payload accounting\n\nbytes\n\nB\n\nSustained payload byte rate\n\nbytes/s\n\nLittle’s law relates average in-flight request count N to completed requests per second λ and average request latency L. Multiplying byte throughput B by latency gives the corresponding amount of in-flight data. A fabric can have wide links yet remain underused if the workload or queues cannot sustain enough independent transactions to cover the round trip.\n\nAssumptions Stable steady-state averages with the same system boundary for arrivals, completions, and latency. The byte form assumes consistent payload accounting. It supplies a concurrency requirement, not a claim about Blackwell’s undisclosed queue depths.\n\n## Engineering example\n\nAt 2 TB/s and 100 ns of round-trip latency, the bandwidth-delay product is 200 kB of in-flight data before protocol overhead.\n\n## Distinctions and tradeoffs\n\nThe atlas’s routing graph is a teaching diagram, not a verified Blackwell mesh layout. A physical topology should not be inferred from the illustrative geometry.\n\nWider routes and more buffering improve communication capacity but consume area and power. Placing related work and data near one another can reduce movement before any hardware is added.\n\nThousands of concurrent tensor operations must obtain operands and exchange results. Internal communication determines whether the advertised arithmetic units remain busy.\n\n## Specifications and interfaces\n\nEndpoints\n\nSMs, cache, memory, I/O\n\nLogical component classes; exact proprietary topology is not asserted.\n\nAddress view\n\nUnified GPU memory space\n\nPhysical locality can still affect movement cost.\n\nConstraint\n\nBandwidth, latency, and energy\n\nNo unsupported numerical internal-fabric rating is assigned.\n\n### Interfaces\n\nRequest injection Input\n\nAddressed transactions from compute or I/O\n\nRouting Output\n\nPackets or transactions to selected internal endpoints\n\nBackpressure Input\n\nFlow-control information from occupied resources\n\n## Check your understanding\n\n### Why does locality still matter inside one GPU?\n\n1.  Only data outside the rack require energy to move.\n2.  Internal wires and shared routes still incur delay, energy, and contention.\n3.  One GPU contains only one possible data route.\n\nAnswer and reasoning\n\nInternal wires and shared routes still incur delay, energy, and contention.\n\nEven on one chip, communication has physical costs. Keeping related data and work nearby can reduce unnecessary traffic.\n\n## Evidence and scope\n\nThe relation assumes a stable stream and excludes topology-specific backpressure.\n\nUse the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.\n\n## Sources\n\n-   [NVIDIA: GPU locality domains and L2 fabric](https://developer.nvidia.com/blog/accelerating-data-processing-with-nvidia-multi-instance-gpu-and-numa-node-localization/)\n-   [Intel: Memory performance, latency, and bandwidth](https://www.intel.com/content/www/us/en/developer/articles/technical/memory-performance-in-a-nutshell.html)\n-   [John D. C. Little (1961): A Proof for the Queuing Formula L = λW](https://pubsonline.informs.org/doi/abs/10.1287/opre.9.3.383)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/on-chip-network-2fef2e4eae57f640.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [On-chip metal and via hierarchy](https://semiaifoundry.com/learn/atlas/interconnect/)\n-   [Shared L2 cache](https://semiaifoundry.com/learn/atlas/l2-cache/)\n\n### Related lessons\n\n-   [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/)\n-   [HBM controller and PHY](https://semiaifoundry.com/learn/atlas/hbm-controller/)\n-   [NVLink GPU interface](https://semiaifoundry.com/learn/atlas/nvlink-port/)\n-   [GPU logic dies](https://semiaifoundry.com/learn/atlas/gpu-die/)"
}