{"id":"shared-sram","level":"die","name":"Shared memory SRAM","shortName":"Shared SRAM","role":"Lets cooperating threads explicitly stage and reuse a tile near the compute units.","description":"Shared memory is a software-managed scratchpad allocated to a thread block. Threads cooperate to load a tile, synchronize when required, and reuse it for several operations before replacing it. Its underlying storage is SRAM, which retains state while powered without the periodic refresh needed by DRAM. Shared memory is banked, so the access pattern matters: requests to different words in the same bank can serialize. Padding or rearranging a tile can remove those conflicts. The capacity is small compared with HBM, but a small tile can be reused many times. Requests that land on the same memory bank can serialize; changing a tile’s layout, for example by padding a matrix transpose, can reduce conflicts without changing its mathematical result.","specs":[{"label":"Address banks","value":"32 banks","context":"Successive 32-bit words map to successive banks in CUDA shared memory."},{"label":"Capacity per SM","value":"Up to 228 KB","context":"Blackwell compute capability 10.0 shared-memory carveout."},{"label":"Per-block maximum","value":"227 KB","context":"One KB reserved by CUDA; dynamic opt-in required above static limits."}],"interfaces":[{"name":"Tile staging","direction":"in","signal":"Coalesced loads or asynchronous copies"},{"name":"Tile reuse","direction":"out","signal":"Operands consumed by cooperating threads"},{"name":"Synchronization","direction":"in","signal":"Barriers establishing when staged data are ready"}],"tradeoff":"Larger tiles improve reuse but consume more scratchpad and may reduce active blocks. Bank conflicts can waste this fast storage unless the layout matches the access pattern.","whyAI":"Attention kernels can avoid materializing large intermediate matrices in HBM by combining local tiles with online reductions. The savings come from a better dataflow, not extra model parameters.","related":["streaming-multiprocessor","tensor-core","l1-cache","tensor-memory-accelerator"],"sources":[{"title":"NVIDIA CUDA: Writing SIMT Kernels","url":"https://docs.nvidia.com/cuda/archive/13.2.0/cuda-programming-guide/02-basics/writing-cuda-kernels.html"},{"title":"NVIDIA CUDA Blackwell Tuning Guide","url":"https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html"}],"scale":"On-die SRAM banks; logical address map rather than transistor layout","kind":"banked-scratchpad","mechanism":["Cooperating threads or a copy engine bring a reusable tile from device memory into a block’s shared-memory allocation.","A synchronization point establishes readiness; threads then read the tile repeatedly while computing several outputs.","The block reuses the scratchpad for another tile, choosing an address layout that avoids conflicting accesses to the same bank."],"misconception":"Shared memory and L1 cache are not interchangeable programming concepts. Shared memory is explicitly managed by software; cache retention is automatic.","check":{"question":"Why might adding one padding column to a shared-memory tile improve performance?","options":["It increases HBM capacity.","It can redirect accesses across different banks and avoid serialization.","It adds more Tensor Cores to the SM."],"answer":1,"explanation":"Padding changes address-to-bank mapping. The arithmetic can stay identical while the memory requests stop competing for the same bank."},"branch":"Memory hierarchy","subbranch":"On-chip storage","science":[{"title":"Address layout determines bank conflicts","equation":"bank(a) = floor(a / 4) mod 32","explanation":"a is a byte address for a 32-bit word in the documented shared-memory organization. Adjacent words land in different banks, while addresses separated by 128 bytes return to the same bank. A warp accessing different words in one bank may serialize. Padding a two-dimensional array changes its row stride and can transform a column access from conflicting to distributed.","assumptions":"32 banks and 4-byte word mapping in the referenced CUDA programming model. Wider instructions may be split into transactions; broadcast of the same read address is a special case and is not equivalent to different-word conflicts."}],"history":[],"learningObjective":"Relate scratchpad capacity to tile reuse.","engineeringExample":"A tile that exceeds the available scratchpad must be split or reloaded, adding traffic that can erase arithmetic reuse.","evidenceNotes":"The bank and capacity behavior depends on a specific implementation.","visualFamily":"memory-route","prerequisites":["sram","register-file"],"labId":"engineering","scope":"Compare unblocked and tiled matrix traffic for fixed arithmetic; architecture-specific instructions, fusion and attention algorithms are context rather than simulated implementations.","deepLabId":"dataflow","investigationId":"execution","labScope":"Compare unblocked and tiled matrix traffic for fixed arithmetic; architecture-specific instructions, fusion and attention algorithms are context rather than simulated implementations."}