{
  "schemaVersion": 1,
  "id": "atlas-lesson-kernel",
  "title": "GPU kernel and thread hierarchy",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Maps an operation onto many coordinated logical threads or tiles.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/kernel/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Execution software",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "Numbers are synthetic and share a precision and byte boundary. The roofline is an upper bound on rate, not a measured runtime; dependencies, occupancy, launch overhead and cache behavior can reduce performance.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/kernel-4d14f06c973afea4.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#kernel",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "4017f9b0ea32f49df4e2567504ed0cb53c38b875b4f6efb4e26a27e4456b6cf8",
  "contentLength": 5329,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-kernel.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-kernel.md",
  "sections": [
    {
      "id": "section-1",
      "title": "GPU kernel and thread hierarchy",
      "start": 35,
      "end": 357
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 357,
      "end": 1315
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1315,
      "end": 1343
    },
    {
      "id": "section-4",
      "title": "Roofline model",
      "start": 1343,
      "end": 2297
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2297,
      "end": 2557
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 2557,
      "end": 2955
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 2955,
      "end": 3255
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 3255,
      "end": 3410
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 3410,
      "end": 3439
    },
    {
      "id": "section-10",
      "title": "Can a normal kernel assume all grid blocks run simultaneously?",
      "start": 3439,
      "end": 3869
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 3869,
      "end": 4222
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 4222,
      "end": 4659
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 4659,
      "end": 4682
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 4682,
      "end": 4945
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 4945,
      "end": 5329
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#kernel"
    },
    {
      "title": "NVIDIA CUDA — Programming Model",
      "url": "https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html"
    },
    {
      "title": "Berkeley Lab — Introduction to Roofline",
      "url": "https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/kernel-4d14f06c973afea4.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "CUDA programming and runtime layer",
      "url": "https://semiaifoundry.com/learn/atlas/cuda/"
    },
    {
      "title": "Tensor Core matrix unit",
      "url": "https://semiaifoundry.com/learn/atlas/tensor-core/"
    },
    {
      "title": "Shared memory SRAM",
      "url": "https://semiaifoundry.com/learn/atlas/shared-sram/"
    },
    {
      "title": "Compiler, intermediate code, and device executable",
      "url": "https://semiaifoundry.com/learn/atlas/compiler/"
    },
    {
      "title": "Data, tensor, pipeline, and state parallelism",
      "url": "https://semiaifoundry.com/learn/atlas/parallelism/"
    },
    {
      "title": "Register file",
      "url": "https://semiaifoundry.com/learn/atlas/register-file/"
    },
    {
      "title": "Feed-forward network",
      "url": "https://semiaifoundry.com/learn/atlas/feed-forward/"
    }
  ],
  "recordHash": "c23cf225eb6f4c4f908d9e3946ee72810f5c93f542b8337b52d4fc3f6703900d",
  "contentFormat": "text/markdown",
  "content": "Execution software · Atlas v1.3.0\n\n# GPU kernel and thread hierarchy\n\nMaps an operation onto many coordinated logical threads or tiles.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#kernel)Logical work grid mapped over physical GPU resources\n\nLearning objectiveUse arithmetic intensity to decide which roofline bound controls a specified kernel.\n\n## The mechanism\n\nA GPU kernel is a unit of device code launched with an execution configuration. In CUDA's thread model, threads are grouped into blocks, and blocks form a grid. A block executes on a streaming multiprocessor and can use block-level shared memory and synchronization. The grid may contain far more blocks than the device can host simultaneously, so the logical grid is not a physical map of dedicated cores. The atlas animates blocks being assigned to available processors while preserving the distinction between work items and hardware. Efficient kernels arrange data access and reuse carefully. Arbitrary cross-block waiting is unsafe in a normal launch because blocks are not guaranteed to run in a particular order.\n\n1.  Divide the operation into indexed work items or tiles.\n2.  Launch blocks that the device assigns to available execution resources.\n3.  Reuse and synchronize data within supported boundaries, then write the result.\n\n## Science and assumptions\n\n### Roofline model\n\n$$\nI=\\frac{F}{M},\\qquad R\\le\\min(P_{\\mathrm{peak}},B_{\\mathrm{memory}}I)\n$$\n\nArithmetic intensity is floating-point work divided by bytes moved. Throughput is bounded by the smaller of arithmetic peak and memory bandwidth times intensity.\n\nSymbols and units\n\nI\n\nArithmetic intensity at the selected memory boundary\n\nFLOP/byte\n\nF\n\nWork in the declared floating-point convention\n\nFLOP\n\nM\n\nBytes moved at that boundary\n\nbytes\n\nR\n\nAttainable arithmetic throughput\n\nFLOP/s\n\nP\\_peak\n\nCompatible arithmetic peak\n\nFLOP/s\n\nB\\_memory\n\nUseful memory bandwidth\n\nbytes/s\n\nArithmetic intensity connects the operation count to traffic at a selected memory boundary. Tiling can raise intensity by reusing fetched data, shifting a kernel toward its arithmetic ceiling.\n\nAssumptions Upper-bound model for a specified numeric format and memory level; measured performance is also limited by launch, occupancy, dependencies, and implementation details.\n\n## Engineering example\n\nSynthetic kernel does 2 GFLOP and moves 1 GB at the selected memory boundary, so intensity is 2 FLOP/byte. A 10 TFLOP/s arithmetic peak and 1 TB/s memory rate yield min(10, 2) = 2 TFLOP/s; the optimistic execution lower bound is 1 ms.\n\n## Distinctions and tradeoffs\n\nOne launched thread is not one permanently dedicated physical GPU core; logical work is scheduled onto shared hardware.\n\nMore parallel work can hide latency. Register use, shared memory, divergence, and data access constrain useful occupancy.\n\nKernels implement the actual tensor operations on GPUs. Tiling and reuse can matter as much as raw arithmetic capability.\n\n## Specifications and interfaces\n\nThread-model hierarchy\n\nThread → block → grid\n\nLogical launch organization.\n\nWarp size\n\n32 CUDA threads\n\nExecution grouping in the documented programming model.\n\nNormal block ordering\n\nNot guaranteed\n\nSpecial supported cooperative mechanisms have additional rules.\n\n### Interfaces\n\nInput tensors and launch shape Input\n\nData pointers and parallel work configuration\n\nOutput tensors Output\n\nResults written by the kernel\n\n## Check your understanding\n\n### Can a normal kernel assume all grid blocks run simultaneously?\n\n1.  Yes, every block has its own permanent core\n2.  Yes, if the grid is colored uniformly\n3.  No; block scheduling and order are not generally guaranteed\n\nAnswer and reasoning\n\nNo; block scheduling and order are not generally guaranteed\n\nA grid can exceed the device’s resident capacity, so ordinary cross-block dependencies must respect the programming model.\n\n## Evidence and scope\n\nNumbers are synthetic and share a precision and byte boundary. The roofline is an upper bound on rate, not a measured runtime; dependencies, occupancy, launch overhead and cache behavior can reduce performance.\n\nCompare arithmetic and memory-bandwidth ceilings; occupancy and instruction dependencies remain omitted constraints.\n\n## Sources\n\n-   [NVIDIA CUDA — Programming Model](https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html)\n-   [Berkeley Lab — Introduction to Roofline](https://crd.lbl.gov/assets/Uploads/ECP19-Roofline-1-intro.pdf)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/kernel-4d14f06c973afea4.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [CUDA programming and runtime layer](https://semiaifoundry.com/learn/atlas/cuda/)\n-   [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/)\n-   [Shared memory SRAM](https://semiaifoundry.com/learn/atlas/shared-sram/)\n\n### Related lessons\n\n-   [Compiler, intermediate code, and device executable](https://semiaifoundry.com/learn/atlas/compiler/)\n-   [Data, tensor, pipeline, and state parallelism](https://semiaifoundry.com/learn/atlas/parallelism/)\n-   [Register file](https://semiaifoundry.com/learn/atlas/register-file/)\n-   [Feed-forward network](https://semiaifoundry.com/learn/atlas/feed-forward/)"
}