{
  "schemaVersion": 1,
  "id": "atlas-lesson-inference-custom-silicon",
  "title": "Custom inference silicon: Maia and MTIA",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Allocates compute and memory around a provider’s measured inference needs.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/inference-custom-silicon/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Processing architectures",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Maia specifications are provider-reported. The inaccessible next-generation Meta blog was replaced by accessible first-generation and engineering sources; no unsupported current MTIA peak is claimed.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/inference-custom-silicon-5d522fce85efc4d7.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#inference-custom-silicon",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "bfd540602796a9b3954114011c9f9f82fdcd84564b8689744a54645f10634fa5",
  "contentLength": 5877,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-inference-custom-silicon.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-inference-custom-silicon.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Custom inference silicon: Maia and MTIA",
      "start": 41,
      "end": 432
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 432,
      "end": 1721
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1721,
      "end": 1749
    },
    {
      "id": "section-4",
      "title": "Weighted service cost",
      "start": 1749,
      "end": 2659
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2659,
      "end": 2922
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2922,
      "end": 3379
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3379,
      "end": 3682
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3682,
      "end": 3870
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3870,
      "end": 3899
    },
    {
      "id": "section-10",
      "title": "What evidence best motivates more on-chip SRAM for an inference design?",
      "start": 3899,
      "end": 4325
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4325,
      "end": 4693
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4693,
      "end": 5326
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5326,
      "end": 5349
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5349,
      "end": 5537
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5537,
      "end": 5877
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#inference-custom-silicon"
    },
    {
      "title": "Maia 200: The AI accelerator built for inference",
      "url": "https://blogs.microsoft.com/blog/2026/01/26/maia-200-the-ai-accelerator-built-for-inference/"
    },
    {
      "title": "MTIA v1: Meta’s first-generation AI inference accelerator",
      "url": "https://ai.meta.com/blog/meta-training-inference-accelerator-AI-MTIA/"
    },
    {
      "title": "Inside the hardware and co-design of MTIA",
      "url": "https://engineering.fb.com/2024/08/22/ml-applications/meta-mtia-hardware-co-design/"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/inference-custom-silicon-5d522fce85efc4d7.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Full memory hierarchy and input path",
      "url": "https://semiaifoundry.com/learn/atlas/memory-hierarchy-route/"
    },
    {
      "title": "Host CPU",
      "url": "https://semiaifoundry.com/learn/atlas/host-cpu/"
    },
    {
      "title": "Workload contract and architecture selection",
      "url": "https://semiaifoundry.com/learn/atlas/chip-workload-contract/"
    },
    {
      "title": "Choosing SRAM, HBM, DDR, LPDDR or GDDR",
      "url": "https://semiaifoundry.com/learn/atlas/memory-interface-choice/"
    },
    {
      "title": "Custom-chip volume economics",
      "url": "https://semiaifoundry.com/learn/atlas/chip-volume-economics/"
    }
  ],
  "recordHash": "39694085170190eccb4ea25154006846c1092ce81722ccaa4e557c4e22bbb144",
  "contentFormat": "text/markdown",
  "content": "Processing architectures · Atlas v1.3.0\n\n# Custom inference silicon: Maia and MTIA\n\nAllocates compute and memory around a provider’s measured inference needs.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#inference-custom-silicon)Processing element to accelerator package; array tiles exaggerated\n\nLearning objectiveExplain custom inference design through workload and software requirements rather than brand categories.\n\n## The mechanism\n\nCustom inference chips express a workload choice, not a single architecture category. Microsoft’s Maia 200 announcement describes an inference-focused accelerator with a memory hierarchy and system connections designed together. Meta’s historical MTIA v1 disclosure centers ranking and recommendation inference and uses substantial SRAM with LPDDR5; its second-generation co-design discussion is a separate specimen. Those examples show why every custom accelerator need not reproduce a GPU’s balance of matrix arithmetic, cache and HBM. A recommendation lookup and a long-context language-model decode have different working sets and reuse, so comparing them solely by nominal operations per second hides the design rationale.\n\n1.  Measure the actual operator mix and service objectives instead of assuming every AI workload resembles dense training.\n2.  Estimate which working sets can remain in SRAM and which must cross a DRAM interface.\n3.  Choose arithmetic, memory and networking resources that jointly address the measured bottleneck.\n4.  Co-develop the compiler and kernels so the intended data placement is achievable by production models.\n5.  Separate announcement, disclosed architecture and measured deployment evidence when judging the resulting platform.\n\n## Science and assumptions\n\n### Weighted service cost\n\n$$\nC_{\\mathrm{result}}=\\frac{C_{\\mathrm{time}}}{R_{\\mathrm{useful}}a}\n$$\n\nCost per accepted result is the charged cost rate divided by the running accepted-result rate times operational availability.\n\na explicitly names the source availability term. Cost and result rates must use the same accounting interval and service target.\n\nSymbols and units\n\nC\\_result\n\nTime-based service cost per accepted result\n\ncurrency/result\n\nC\\_time\n\nCharged service cost rate over the full accounting interval\n\ncurrency/s\n\nR\\_useful\n\nAccepted-result rate while running\n\nresults/s\n\na\n\nOperational availability fraction over that interval\n\nfraction\n\nCtime is currency/s, Ruseful is accepted results/s while running, and availability is a dimensionless operational fraction.\n\nAssumptions Consistent service quality and accounting boundary; model mix and fixed development costs need separate treatment.\n\n## Engineering example\n\nSynthetic comparison: a lookup set with high reuse that fits local SRAM can avoid repeated external reads; doubling arithmetic alone leaves that traffic unchanged. A different model whose weights exceed SRAM requires a new memory budget.\n\n## Distinctions and tradeoffs\n\nCustom means designed for a chosen workload and organization; it does not automatically mean faster, cheaper or appropriate for every model.\n\nA provider can optimize frequent operations aggressively, but must fund the compiler, kernels and adaptation needed when models change.\n\nLarge service volumes can justify specialization only when the resulting platform delivers acceptable quality, reliability and useful throughput.\n\n## Specifications and interfaces\n\nMaia specimen\n\nMaia 200, January 2026 announcement\n\nProvider-reported inference design and deployment statements.\n\nMTIA specimen\n\nMTIA v1 and separate 2024 co-design discussion\n\nHistorical ranking/recommendation context, not a claim about all future MTIA generations.\n\n### Interfaces\n\nProduction workload traces Input\n\nOperator shapes, memory reuse and latency targets\n\nSpecialized platform Output\n\nSupported inference graph under a defined service target\n\n## Check your understanding\n\n### What evidence best motivates more on-chip SRAM for an inference design?\n\n1.  A useful working set is repeatedly accessed and can fit there\n2.  A competitor advertises more peak operations\n3.  The chip name contains AI\n\nAnswer and reasoning\n\nA useful working set is repeatedly accessed and can fit there\n\nMeasured reuse and a fitting working set provide a causal link between SRAM allocation and reduced external traffic.\n\n## Evidence and scope\n\nMaia specifications are provider-reported. The inaccessible next-generation Meta blog was replaced by accessible first-generation and engineering sources; no unsupported current MTIA peak is claimed.\n\nNo current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.\n\n## Sources\n\n-   [Maia 200: The AI accelerator built for inference](https://blogs.microsoft.com/blog/2026/01/26/maia-200-the-ai-accelerator-built-for-inference/)\n-   [MTIA v1: Meta’s first-generation AI inference accelerator](https://ai.meta.com/blog/meta-training-inference-accelerator-AI-MTIA/)\n-   [Inside the hardware and co-design of MTIA](https://engineering.fb.com/2024/08/22/ml-applications/meta-mtia-hardware-co-design/)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/inference-custom-silicon-5d522fce85efc4d7.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Full memory hierarchy and input path](https://semiaifoundry.com/learn/atlas/memory-hierarchy-route/)\n-   [Host CPU](https://semiaifoundry.com/learn/atlas/host-cpu/)\n\n### Related lessons\n\n-   [Workload contract and architecture selection](https://semiaifoundry.com/learn/atlas/chip-workload-contract/)\n-   [Choosing SRAM, HBM, DDR, LPDDR or GDDR](https://semiaifoundry.com/learn/atlas/memory-interface-choice/)\n-   [Custom-chip volume economics](https://semiaifoundry.com/learn/atlas/chip-volume-economics/)"
}