{
  "schemaVersion": 1,
  "id": "atlas-lesson-aws-neuron-dataflow",
  "title": "Trainium and Inferentia execution paths",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Combines specialized engines, local buffers and a compiler-managed accelerator runtime.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/aws-neuron-dataflow/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Processing architectures",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Published memory values are retained in evidence claims. The example is synthetic and does not imply equal features or performance across Trainium2 and Inferentia2.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/aws-neuron-dataflow-a76c9fd8110a4cf1.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#aws-neuron-dataflow",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "986e8099757ff52ce94a6dfae389010bc22a3fe6404af40c2b746dc20df7cbff",
  "contentLength": 5280,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-aws-neuron-dataflow.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-aws-neuron-dataflow.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Trainium and Inferentia execution paths",
      "start": 41,
      "end": 417
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 417,
      "end": 1661
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1661,
      "end": 1689
    },
    {
      "id": "section-4",
      "title": "Engine schedule bound",
      "start": 1689,
      "end": 2355
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2355,
      "end": 2565
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2565,
      "end": 3067
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3067,
      "end": 3327
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3327,
      "end": 3497
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3497,
      "end": 3526
    },
    {
      "id": "section-10",
      "title": "Why must an Inf2 instance memory total be distinguished from an Inferentia2 chip value?",
      "start": 3526,
      "end": 3964
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3964,
      "end": 4297
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4297,
      "end": 4784
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4784,
      "end": 4807
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4807,
      "end": 5037
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5037,
      "end": 5280
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#aws-neuron-dataflow"
    },
    {
      "title": "Trainium2 Architecture",
      "url": "https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/arch/neuron-hardware/trainium2.html"
    },
    {
      "title": "Amazon EC2 Inf2 Architecture",
      "url": "https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/arch/neuron-hardware/inf2-arch.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/aws-neuron-dataflow-a76c9fd8110a4cf1.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Systolic arrays and tensor dataflow",
      "url": "https://semiaifoundry.com/learn/atlas/systolic-array-dataflow/"
    },
    {
      "title": "Compiler, intermediate code, and device executable",
      "url": "https://semiaifoundry.com/learn/atlas/compiler/"
    },
    {
      "title": "Compiler layout, lowering and buffer planning",
      "url": "https://semiaifoundry.com/learn/atlas/compiler-layout-lowering/"
    },
    {
      "title": "Runtime, driver and device dispatch",
      "url": "https://semiaifoundry.com/learn/atlas/runtime-device-dispatch/"
    }
  ],
  "recordHash": "2d898c13c8deb742c201d5031078ba23eb6a384a2924d0a423ad4d49686f59a3",
  "contentFormat": "text/markdown",
  "content": "Processing architectures · Atlas v1.3.0\n\n# Trainium and Inferentia execution paths\n\nCombines specialized engines, local buffers and a compiler-managed accelerator runtime.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#aws-neuron-dataflow)Processing element to accelerator package; array tiles exaggerated\n\nLearning objectiveIdentify chip, instance and logical-core boundaries in an accelerator disclosure.\n\n## The mechanism\n\nAn AWS Neuron program targets a machine with multiple kinds of execution and data movement resources. Matrix-heavy work, elementwise work and transfers need compatible schedules and buffer layouts. Trainium2 and Inferentia2 are named chip generations, while Trn2 and Inf2 describe instance families that assemble chips with host and interconnect resources. AWS also distinguishes physical and logical NeuronCore configurations, making a core count meaningful only with its definition. The compiler and runtime are part of the deployment path: supported operators, executable shapes and collective placement determine whether an application can use the disclosed hardware effectively.\n\n1.  Choose a chip and instance boundary before counting cores, memory or interconnect resources.\n2.  Compile the model into operations supported by the selected Neuron software and hardware generation.\n3.  Place live tiles in on-chip buffers and arrange transfers from HBM to feed the relevant compute engines.\n4.  Schedule collective transfers and independent arithmetic where dependencies permit useful overlap.\n5.  Profile the complete application and record software versions, logical-core mode and shape configuration with any result.\n\n## Science and assumptions\n\n### Engine schedule bound\n\n$$\nT_{\\mathrm{step}}\\ge\\max(T_{\\mathrm{tensor}},T_{\\mathrm{vector}},T_{\\mathrm{transfer}})\n$$\n\nStep time is at least the largest resource-time requirement among tensor, vector and transfer engines.\n\nSymbols and units\n\nT\\_step\n\nStep duration lower bound\n\ns\n\nT\\_tensor\n\nTime required on tensor engines\n\ns\n\nT\\_vector\n\nTime required on vector engines\n\ns\n\nT\\_transfer\n\nTime required on transfer resources\n\ns\n\nEach term is the time required on one resource class in seconds. Parallel engines only overlap work allowed by dependencies.\n\nAssumptions A lower bound for sufficiently buffered execution; engine contention and serial edges add time.\n\n## Engineering example\n\nSynthetic graph: 5 ms of matrix work, 2 ms of vector work and 4 ms of transfers could approach a 5 ms bound if fully overlapped. A serial dependency ordering can instead require 11 ms.\n\n## Distinctions and tradeoffs\n\nTrainium and Inferentia names do not mean training and inference are physically exclusive operations; support and performance depend on the generation and software path.\n\nExplicit scheduling can improve reuse and overlap, but unusual shapes or unsupported operators can require recompilation, custom kernels or fallback execution.\n\nTraining and serving need a complete supported model path; a high matrix peak does not establish end-to-end operator coverage or latency.\n\n## Specifications and interfaces\n\nSpecimen\n\nTrainium2 and Inferentia2\n\nHistorical disclosed generations, not interchangeable chip specifications.\n\nMemory units\n\nGiB capacity; TB/s link rate\n\nPreserve the source’s binary capacity and decimal bandwidth labels.\n\n### Interfaces\n\nNeuron executable Input\n\nOperations, shapes, buffer assignments and placements\n\nSpecialized engines Output\n\nTensor results and collective data transfers\n\n## Check your understanding\n\n### Why must an Inf2 instance memory total be distinguished from an Inferentia2 chip value?\n\n1.  The instance can contain several accelerator chips\n2.  Instances contain no host resources\n3.  Memory capacity is always bandwidth\n\nAnswer and reasoning\n\nThe instance can contain several accelerator chips\n\nAn instance aggregates a defined configuration of chips and hosts; comparing its total against a single chip changes the denominator.\n\n## Evidence and scope\n\nPublished memory values are retained in evidence claims. The example is synthetic and does not imply equal features or performance across Trainium2 and Inferentia2.\n\nNo current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.\n\n## Sources\n\n-   [Trainium2 Architecture](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/arch/neuron-hardware/trainium2.html)\n-   [Amazon EC2 Inf2 Architecture](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/arch/neuron-hardware/inf2-arch.html)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/aws-neuron-dataflow-a76c9fd8110a4cf1.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Systolic arrays and tensor dataflow](https://semiaifoundry.com/learn/atlas/systolic-array-dataflow/)\n-   [Compiler, intermediate code, and device executable](https://semiaifoundry.com/learn/atlas/compiler/)\n\n### Related lessons\n\n-   [Compiler layout, lowering and buffer planning](https://semiaifoundry.com/learn/atlas/compiler-layout-lowering/)\n-   [Runtime, driver and device dispatch](https://semiaifoundry.com/learn/atlas/runtime-device-dispatch/)"
}