# Register file Canonical: https://semiaifoundry.com/learn/atlas/register-file/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Memory hierarchy · Atlas v1.3.0 # Register file Keeps each active thread’s immediate values close to the arithmetic pipelines. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#register-file)On-die storage banks; physical bank dimensions are schematic Learning objectiveExplain why registers can limit occupancy. ## The mechanism Registers hold the addresses, counters, partial results, and scalar operands that threads need immediately. The compiler assigns program values to a finite physical pool. A value in a register does not need a fresh trip to off-chip memory for every instruction. If a kernel asks for too many registers, fewer blocks may fit at once; some values may also spill into local memory, which is backed by device memory and cached. Registers are private to a thread in the programming model, although explicit warp shuffle instructions can exchange values between participating threads. Local register reuse keeps operands near arithmetic lanes; a spill adds memory traffic and can lengthen the path to the same result. 1. The compiler maps frequently used thread values to registers, subject to the available register pool and allocation rules. 2. Execution pipelines repeatedly read and update those nearby values without fetching them anew from HBM. 3. If register demands exceed a useful allocation, parallel residency can fall or values can spill to device-backed local memory. ## Science and assumptions ### Register allocation limits resident work $$ B_{\mathrm{resident}}\le\left\lfloor\frac{R_{\mathrm{SM}}}{r_{\mathrm{thread}}T_{\mathrm{block}}}\right\rfloor $$ Resident blocks cannot exceed the floor of available registers divided by registers per thread times threads per block. Symbols and units B\_resident Blocks permitted by the register-only budget blocks/SM R\_SM Available 32-bit register slots per SM registers/SM r\_thread Register slots requested by one thread registers/thread T\_block Threads in one block threads/block RSM is the number of available 32-bit registers per SM, rthread is registers requested per thread, and Tblock is threads per block. Their product estimates the register footprint of one block. Doubling the per-thread requirement can reduce the number of blocks that fit, even though it may save loads by retaining more intermediate results. Assumptions A register-only upper bound, assuming uniform allocation. Hardware allocation granularity, reserved resources, warp/block limits, and shared-memory use can lower residency further. It is not a complete CUDA occupancy calculator. ## Engineering example If each thread consumes more registers, floor($R_{SM}$/($r_{\mathrm{thread}}$ $T_{\mathrm{block}}$)) decreases even if arithmetic units remain available. ## Distinctions and tradeoffs CUDA local memory does not mean physically nearest to a thread. Register spills to local memory are backed by device memory and may be cached. Keeping more intermediates in registers can reduce traffic, but large allocations limit parallel residency. Kernel fusion therefore needs a balance between reuse and register pressure. Matrix kernels, normalization, and attention all reuse small values many times. Register allocation can change their speed even when the mathematical model remains identical. ## Specifications and interfaces Pool per SM 64K × 32-bit registers Blackwell architectural register-file count. Per-thread limit 255 registers Architectural maximum; allocation granularity and occupancy also matter. Warp exchange Shuffle instructions Explicit register-value exchange among participating warp threads. ### Interfaces Load or computation Input Thread-local values Execution operands Output Addresses and arithmetic inputs Spill path Output Values moved to device-backed local memory ## Check your understanding ### What can happen when each thread uses many more registers? 1. The GPU gains more physical register capacity automatically. 2. The model becomes permanently stored on the die. 3. Fewer blocks fit concurrently, or some values spill. Answer and reasoning Fewer blocks fit concurrently, or some values spill. Registers are a finite resource shared among resident work. More per-thread state can trade local reuse for lower concurrency or extra memory traffic. ## Evidence and scope The model ignores allocation granularity, spilling and architecture-specific limits. Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism. ## Sources - [NVIDIA CUDA Blackwell Tuning Guide](https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html) - [NVIDIA: Register Cache for Warp-Centric CUDA Programs](https://developer.nvidia.com/blog/register-cache-warp-cuda/) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/register-file-a316c3c8ecd82c48.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [Edge-triggered D flip-flop](https://semiaifoundry.com/learn/atlas/flipflop/) - [Six-transistor static memory cell](https://semiaifoundry.com/learn/atlas/sram/) ### Related lessons - [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/) - [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/) - [L1 and texture cache](https://semiaifoundry.com/learn/atlas/l1-cache/) - [GPU kernel and thread hierarchy](https://semiaifoundry.com/learn/atlas/kernel/)