The mechanism
The on-chip fabric is the communication system within the silicon. Requests carry addresses, operation types, and data between blocks; arbitration decides which request uses a contested path. Wires have capacitance and resistance, so longer journeys consume time and energy even before leaving the package. A GPU can expose a unified address space while still having internal locality domains. An illustrative routing graph is distinct from an undisclosed physical topology: a tidy mesh is a teaching model, not evidence that Blackwell uses that exact network. Shared routes can create contention, and local versus remote cache access can use different paths; the chip’s communication behavior matters alongside its arithmetic.
- A compute, cache, or I/O block injects an addressed transaction into the chip’s communication fabric.
- Routing and arbitration carry that transaction toward its destination while managing competition and backpressure.
- The response returns through the fabric; repeated or long-distance traffic consumes bandwidth, energy, and time.
Science and assumptions
Enough requests must be in flight
Average in-flight requests equal request rate times latency. The corresponding in-flight byte amount is approximately byte throughput times latency.
Symbols and units
- N_inflight
- Average outstanding request count
requests
- λ
- Completed request rate
requests/s
- L
- Mean request time within the same boundary
s
- M_inflight
- Mean in-flight data amount under consistent payload accounting
bytes
- B
- Sustained payload byte rate
bytes/s
Little’s law relates average in-flight request count N to completed requests per second λ and average request latency L. Multiplying byte throughput B by latency gives the corresponding amount of in-flight data. A fabric can have wide links yet remain underused if the workload or queues cannot sustain enough independent transactions to cover the round trip.
Assumptions Stable steady-state averages with the same system boundary for arrivals, completions, and latency. The byte form assumes consistent payload accounting. It supplies a concurrency requirement, not a claim about Blackwell’s undisclosed queue depths.
Engineering example
At 2 TB/s and 100 ns of round-trip latency, the bandwidth-delay product is 200 kB of in-flight data before protocol overhead.
Distinctions and tradeoffs
The atlas’s routing graph is a teaching diagram, not a verified Blackwell mesh layout. A physical topology should not be inferred from the illustrative geometry.
Wider routes and more buffering improve communication capacity but consume area and power. Placing related work and data near one another can reduce movement before any hardware is added.
Thousands of concurrent tensor operations must obtain operands and exchange results. Internal communication determines whether the advertised arithmetic units remain busy.
Specifications and interfaces
- Endpoints
- SMs, cache, memory, I/O
Logical component classes; exact proprietary topology is not asserted.
- Address view
- Unified GPU memory space
Physical locality can still affect movement cost.
- Constraint
- Bandwidth, latency, and energy
No unsupported numerical internal-fabric rating is assigned.
Interfaces
- Request injection Input
- Addressed transactions from compute or I/O
- Routing Output
- Packets or transactions to selected internal endpoints
- Backpressure Input
- Flow-control information from occupied resources
Check your understanding
Why does locality still matter inside one GPU?
- Only data outside the rack require energy to move.
- Internal wires and shared routes still incur delay, energy, and contention.
- One GPU contains only one possible data route.
Answer and reasoning
Internal wires and shared routes still incur delay, energy, and contention.
Even on one chip, communication has physical costs. Keeping related data and work nearby can reduce unnecessary traffic.
Evidence and scope
The relation assumes a stable stream and excludes topology-specific backpressure.
Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.