# Shared memory SRAM Canonical: https://semiaifoundry.com/learn/atlas/shared-sram/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Memory hierarchy · Atlas v1.3.0 # Shared memory SRAM Lets cooperating threads explicitly stage and reuse a tile near the compute units. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#shared-sram)On-die SRAM banks; logical address map rather than transistor layout Learning objectiveRelate scratchpad capacity to tile reuse. ## The mechanism Shared memory is a software-managed scratchpad allocated to a thread block. Threads cooperate to load a tile, synchronize when required, and reuse it for several operations before replacing it. Its underlying storage is SRAM, which retains state while powered without the periodic refresh needed by DRAM. Shared memory is banked, so the access pattern matters: requests to different words in the same bank can serialize. Padding or rearranging a tile can remove those conflicts. The capacity is small compared with HBM, but a small tile can be reused many times. Requests that land on the same memory bank can serialize; changing a tile’s layout, for example by padding a matrix transpose, can reduce conflicts without changing its mathematical result. 1. Cooperating threads or a copy engine bring a reusable tile from device memory into a block’s shared-memory allocation. 2. A synchronization point establishes readiness; threads then read the tile repeatedly while computing several outputs. 3. The block reuses the scratchpad for another tile, choosing an address layout that avoids conflicting accesses to the same bank. ## Science and assumptions ### Address layout determines bank conflicts $$ \operatorname{bank}(a)=\left\lfloor\frac{a}{4\,\mathrm{bytes}}\right\rfloor\bmod32 $$ Divide the byte address by four bytes, take the floor, then take the remainder modulo 32 to obtain the bank index. The explicit four-byte divisor makes the original address-unit convention visible; this mapping is configuration-specific. Symbols and units a Byte-address offset within the mapped shared-memory address space bytes bank(a) Bank index for the addressed word integer 0–31 4 bytes Word width used by this mapping bytes/word 32 Number of shared-memory banks count a is a byte address for a 32-bit word in the documented shared-memory organization. Adjacent words land in different banks, while addresses separated by 128 bytes return to the same bank. A warp accessing different words in one bank may serialize. Padding a two-dimensional array changes its row stride and can transform a column access from conflicting to distributed. Assumptions 32 banks and 4-byte word mapping in the referenced CUDA programming model. Wider instructions may be split into transactions; broadcast of the same read address is a special case and is not equivalent to different-word conflicts. ## Engineering example A tile that exceeds the available scratchpad must be split or reloaded, adding traffic that can erase arithmetic reuse. ## Distinctions and tradeoffs Shared memory and L1 cache are not interchangeable programming concepts. Shared memory is explicitly managed by software; cache retention is automatic. Larger tiles improve reuse but consume more scratchpad and may reduce active blocks. Bank conflicts can waste this fast storage unless the layout matches the access pattern. Attention kernels can avoid materializing large intermediate matrices in HBM by combining local tiles with online reductions. The savings come from a better dataflow, not extra model parameters. ## Specifications and interfaces Address banks 32 banks Successive 32-bit words map to successive banks in CUDA shared memory. Capacity per SM Up to 228 KB Blackwell compute capability 10.0 shared-memory carveout. Per-block maximum 227 KB One KB reserved by CUDA; dynamic opt-in required above static limits. ### Interfaces Tile staging Input Coalesced loads or asynchronous copies Tile reuse Output Operands consumed by cooperating threads Synchronization Input Barriers establishing when staged data are ready ## Check your understanding ### Why might adding one padding column to a shared-memory tile improve performance? 1. It increases HBM capacity. 2. It can redirect accesses across different banks and avoid serialization. 3. It adds more Tensor Cores to the SM. Answer and reasoning It can redirect accesses across different banks and avoid serialization. Padding changes address-to-bank mapping. The arithmetic can stay identical while the memory requests stop competing for the same bank. ## Evidence and scope The bank and capacity behavior depends on a specific implementation. Compare unblocked and tiled matrix traffic for fixed arithmetic; architecture-specific instructions, fusion and attention algorithms are context rather than simulated implementations. ## Sources - [NVIDIA CUDA: Writing SIMT Kernels](https://docs.nvidia.com/cuda/archive/13.2.0/cuda-programming-guide/02-basics/writing-cuda-kernels.html) - [NVIDIA CUDA Blackwell Tuning Guide](https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/shared-sram-a41c0da718170ae4.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [Six-transistor static memory cell](https://semiaifoundry.com/learn/atlas/sram/) - [Register file](https://semiaifoundry.com/learn/atlas/register-file/) ### Related lessons - [Streaming multiprocessor](https://semiaifoundry.com/learn/atlas/streaming-multiprocessor/) - [Tensor Core matrix unit](https://semiaifoundry.com/learn/atlas/tensor-core/) - [L1 and texture cache](https://semiaifoundry.com/learn/atlas/l1-cache/) - [Tensor Memory Accelerator](https://semiaifoundry.com/learn/atlas/tensor-memory-accelerator/)