{"id":"l1-cache","level":"die","name":"L1 and texture cache","shortName":"L1 cache","role":"Automatically retains recently accessed data and combines nearby thread requests near an SM.","description":"The L1 cache is the first automatic cache close to an SM’s execution units. When threads request memory, hardware checks whether useful data are already nearby. Hits can avoid a longer journey; misses continue toward lower levels of the hierarchy. Consecutive thread addresses can be coalesced into fewer memory transactions, while scattered addresses may move many unused bytes. Blackwell shares a physical capacity budget among L1, texture functionality, and the shared-memory carveout. This differs from the explicit ownership and lifetime of software-managed shared memory. A cache hit serves a load locally, while a miss requests data farther down the hierarchy. Reshaping a warp’s addresses can change transferred bytes even when it computes the same values.","specs":[{"label":"Combined capacity","value":"256 KB per SM","context":"Blackwell B200 L1/texture/shared-memory aggregate; not 256 KB of L1 plus shared memory."},{"label":"Allocation","value":"Configurable carveout","context":"A larger shared-memory reservation leaves a different cache allocation."},{"label":"Coalescing example","value":"32-byte transaction segments","context":"CUDA programming guidance; actual request handling depends on instruction and access pattern."}],"interfaces":[{"name":"Memory request","direction":"in","signal":"Warp load/store addresses"},{"name":"Cache hit","direction":"out","signal":"Locally available data"},{"name":"Cache miss","direction":"out","signal":"Request toward L2 and device memory"}],"tradeoff":"A cache helps when nearby or recent data will be reused, but unpredictable access can defeat locality. Increasing software-managed shared memory can improve explicit reuse while reducing space available to automatic caching.","whyAI":"Embedding lookups, normalization, and intermediate tensor access can become memory limited. Data layout determines whether the hierarchy moves useful numbers or mostly unused bytes.","related":["streaming-multiprocessor","shared-sram","l2-cache","register-file"],"sources":[{"title":"NVIDIA CUDA Blackwell Tuning Guide","url":"https://docs.nvidia.com/cuda/blackwell-tuning-guide/index.html"},{"title":"NVIDIA CUDA C++ Best Practices Guide","url":"https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html"}],"scale":"On-die memory near SMs; illustrative cache lines","kind":"cache-hit-miss","mechanism":["Threads issue addresses, and the memory system combines compatible nearby requests into transactions.","A cache lookup supplies data that are already present near the SM; a miss continues toward the next memory level.","Returned data may populate the cache, making later local or nearby accesses less expensive."],"misconception":"A 256 KB combined L1/texture/shared-memory capacity does not mean 256 KB of L1 plus another 256 KB of shared memory. They share a capacity budget.","check":{"question":"Which access pattern generally makes better use of memory transactions?","options":["Nearby threads requesting neighboring useful values.","Every thread requesting an unrelated distant address.","Every load deliberately bypassing all reuse."],"answer":0,"explanation":"Coalescing can serve neighboring addresses with fewer transactions, reducing the number of unused bytes moved for the same computation."},"branch":"Memory hierarchy","subbranch":"On-chip storage","science":[{"title":"Useful bytes versus bytes actually moved","equation":"ηuseful = Museful / Mtransferred; Buseful = ηuseful × Btransferred","explanation":"Museful counts bytes the computation needs; Mtransferred counts all bytes fetched in the memory transactions serving those needs. A scattered access pattern can move many unused bytes, reducing effective bandwidth even if the physical interface is busy. Coalescing neighboring thread requests and reusing cache lines improve the fraction of movement that performs useful work.","assumptions":"Accounting over the same interval and memory boundary. Count unique or repeated demand consistently, and treat compression, cache hits, write traffic, and replayed transactions separately when profiling."}],"history":[],"learningObjective":"Define useful bytes separately from transferred bytes.","engineeringExample":"If only half of transferred cache-line bytes feed useful arithmetic, η_useful is 0.5 even though the interface moved every byte.","evidenceNotes":"The ratio is workload- and access-pattern-specific.","visualFamily":"memory-route","prerequisites":["sram"],"labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}