{"id":"wafer-scale-routing","level":"compute","name":"Wafer-scale compute and distributed SRAM","shortName":"Wafer-scale compute and distributed SRAM","role":"Keeps a very large compute fabric connected across a wafer-scale implementation.","description":"A wafer-scale processor uses a connected area much larger than a conventional reticle-sized logic die. Cerebras WSE-3 provides a public example with many compute elements and distributed on-chip SRAM, integrated into a purpose-built system. The attraction is fine-grained local communication and a large aggregate memory fabric. It also demands mechanisms for defect tolerance, routing around unavailable resources, power delivery and heat removal. Wafer-scale integration still has finite storage, routing capacity and fault-tolerance resources; a larger connected surface does not provide unlimited memory. Large models may still need external state movement or system-level streaming, so aggregate local SRAM bandwidth cannot be substituted for external memory bandwidth.","specs":[{"label":"Physical boundary","value":"Wafer-scale engine versus complete system","context":"System memory, power and cooling are outside the compute-fabric outline."},{"label":"Bandwidth scope","value":"Aggregate local versus external","context":"Many simultaneous local links cannot all serve one external transfer."}],"interfaces":[{"name":"Distributed local memory","direction":"in","signal":"Weights, activations and routed messages"},{"name":"System interfaces","direction":"out","signal":"Model state exchange and completed tensor results"}],"tradeoff":"Keeping communication local can reduce repeated off-chip movement, but the system must handle defects, thermal density and workloads that exceed local state capacity.","whyAI":"Model partitions can exploit a large connected fabric if the compiler balances compute, SRAM and routing rather than merely filling every processing element.","related":["sram","on-chip-network","parallelism-communication-budget"],"sources":[{"title":"Cerebras announces third-generation Wafer Scale Engine","url":"https://www.cerebras.ai/press-release/cerebras-announces-third-generation-wafer-scale-engine"}],"scale":"Network endpoint to wafer; electrical and optical views are conceptual","kind":"architecture","branch":"Processing architectures","subbranch":"Offload and research architectures","mechanism":["Partition the model’s operations and state across many local compute-memory regions.","Use redundant resources and a supported routing map so manufacturing defects do not require every region to be perfect.","Route intermediate activations through the fabric while keeping reusable state close to its consumer.","Schedule external transfers for state that cannot remain in distributed local SRAM.","Close power, cooling and system I/O budgets before treating the connected wafer as deployable capacity."],"misconception":"A wafer-scale chip is not one uniformly accessible giant register, and aggregate on-chip bandwidth is not equivalent to HBM bandwidth at a package boundary.","check":{"question":"Why is adding every local-link rate misleading for a single external data stream?","options":["The stream must cross its actual interface and topology cuts","All local links form one external port","Bandwidth and capacity are the same unit"],"answer":0,"explanation":"Aggregate rates describe parallel local transfers; a particular source-to-destination path is limited by the links and cuts it actually crosses."},"science":[{"title":"Distributed storage budget","equation":"Mlocal = Σi Mi; Ttransfer ≥ Qexternal / Bexternal","explanation":"Mi is local memory in bytes; Qexternal is external bytes moved and Bexternal is sustainable bytes/s.","assumptions":"Summed capacity assumes usable placement; fragmentation and replication can lower usable model capacity."}],"prerequisites":["sram","on-chip-network"],"learningObjective":"Separate local aggregate capability from external and full-system limits of wafer-scale computing.","workstream":"W07","visualFamily":"architecture-array","engineeringExample":"Synthetic layout: four independent regions each move 1 TB/s locally, but a 0.2 TB/s external port still takes at least 5 ms to supply 1 GB. Summing local links cannot remove that boundary.","evidenceNotes":"WSE-3 announcement facts are vendor-reported. No proprietary defect map, yield, routing algorithm or production price is inferred.","labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}