The mechanism
A custom-chip project begins by defining the work it must perform and the conditions under which success will be measured. A workload contract records operator shapes, numerical quality, latency targets, throughput, state capacity and expected model changes. Architecture exploration then asks how much arithmetic, memory, communication and control are needed together. Existing IP can supply processors, interfaces or other blocks, but its licensing and integration constraints matter. Meta’s MTIA co-design account is one public example of workload-driven development. A chip that excels on one benchmark can still fail the contract if the production graph falls back to a slow unsupported path.
- Collect production-like traces, including unusual sequence lengths, operator shapes and tail-latency cases.
- Define acceptable numerical quality and measurement conditions so the performance target has a stable denominator.
- Build resource models for arithmetic, memory traffic, resident state and inter-device communication.
- Prototype the compiler and critical kernels against architectural alternatives before fixing the hardware interfaces.
- Record unsupported paths and change scenarios, then use these as explicit risks in the architecture decision.
Science and assumptions
Workload-weighted demand
Aggregate resource demand sums each workload class’s request rate times its per-request demand.
Choose one resource and unit before summing; bytes and operations cannot be added together.
Symbols and units
- D_resource
- Aggregate resource demand rate
bytes/s or operations/s
- λₖ
- Arrival rate of workload class k
requests/s
- qₖ
- Resource demand per class-k request
bytes/request or operations/request
- k
- Workload-class index
integer index
λk is requests/s for workload class k and qk is bytes or operations per request. Dresource is bytes/s or operations/s.
Assumptions Linear aggregation before queueing and reuse effects; correlated bursts require an additional capacity margin.
Engineering example
Synthetic mix: 80 requests/s needing 1 GB each and 20 needing 4 GB each creates 160 GB/s of un-reused traffic. Designing from the common 1 GB request alone underestimates the mix by a factor of two.
Distinctions and tradeoffs
A chip specification is not just a target peak FLOPS number; it is a contract for complete workloads and their operating conditions.
Specialization improves the frequent path while narrowing flexibility; programmable fallbacks and memory headroom cost area but can extend useful life.
AI models evolve on a different schedule from silicon, so a workload contract must expose assumptions that may fail before a product ships.
Specifications and interfaces
- Acceptance unit
- Useful workload at fixed quality
For example accepted requests/s under a stated latency target.
- Requirements horizon
- Present mix plus explicit change scenarios
Future operator needs are assumptions, not known measurements.
Interfaces
- Service and model teams Input
- Representative traces and acceptance criteria
- Architecture and software teams Output
- Resource budgets, supported operations and executable model prototypes
Check your understanding
Which requirement most directly prevents a misleading inference performance target?
- A quality threshold and latency condition attached to throughput
- The largest possible transistor count
- A product name shared with a benchmark winner
Answer and reasoning
A quality threshold and latency condition attached to throughput
Throughput is only useful when the outputs satisfy the task and arrive under the required service conditions.
Evidence and scope
Workload mix values are synthetic. Public co-design accounts do not reveal a provider’s complete internal traces, IP contracts or acceptance thresholds.
Use the manufacturing investigation to trace compatible inputs, assembly and operating constraints; no current standalone calculator models this lesson’s full mechanism.