{"id":"cuda","level":"software","name":"CUDA programming and runtime layer","shortName":"CUDA","role":"Provides a software model and interfaces for executing parallel work on NVIDIA GPUs.","description":"CUDA connects host-side programs to GPU execution. A program prepares data, invokes device work, and coordinates memory and synchronization through runtime or driver interfaces and libraries. The programming model gives developers a way to describe many related operations without manually addressing every physical arithmetic lane. Frameworks commonly use CUDA indirectly through optimized libraries or generated kernels. The atlas represents this layer as a translation boundary between tensor operations and device launches, not as a physical chip component. CUDA alone is not an AI model, a complete training framework, or a promise of optimal performance. Data movement, launch structure, numerical formats, and kernel behavior still determine how well the application uses the hardware.","specs":[{"label":"Execution roles","value":"Host + device","context":"Cooperating CPU-side control and GPU-side work."},{"label":"Work expression","value":"Kernels and libraries","context":"Frameworks may call these indirectly."},{"label":"Scope","value":"NVIDIA GPU platform","context":"Other accelerator ecosystems have different interfaces."}],"interfaces":[{"name":"Application operations","direction":"in","signal":"Code, tensors, and launch requests"},{"name":"GPU execution","direction":"out","signal":"Device work and synchronized results"}],"tradeoff":"High-level interfaces improve programmability. Efficient execution still requires suitable algorithms and memory behavior.","whyAI":"CUDA is one major bridge from AI frameworks to accelerators. It helps software turn tensor operations into executable device work.","related":["kernel","compiler","nccl","serving"],"sources":[{"title":"NVIDIA — CUDA FAQ","url":"https://developer.nvidia.com/cuda/faq"},{"title":"Hill and Marty — Amdahl’s Law in the Multicore Era","url":"https://research.cs.wisc.edu/multifacet/amdahl/"}],"scale":"Software abstraction; no physical size","kind":"cuda","mechanism":["Prepare operations and data from the host-side application.","Invoke GPU kernels or accelerated library routines.","Coordinate completion and use the produced results in later work."],"misconception":"CUDA is a programming platform, not a neural network and not a transistor-level execution diagram.","check":{"question":"Where can an AI framework use CUDA?","options":["Only during wafer fabrication","Through runtime interfaces, libraries, or generated GPU kernels","Only after a model reaches AGI"],"answer":1,"explanation":"Framework operations can dispatch work through several CUDA-layer mechanisms, often without users writing kernels directly."},"branch":"Execution software","subbranch":"Programming interfaces","science":[{"title":"Amdahl speedup bound","equation":"S(N) = 1 / [(1 − p) + p/N]","explanation":"If fraction p of baseline time can be accelerated N-fold, the remaining fraction still limits total speedup. Host work and transfers can therefore dominate after a GPU kernel becomes very fast.","assumptions":"Fixed workload, ideal acceleration of the selected fraction, and no added overhead; real GPU applications may have additional communication and scaling effects."}],"history":[],"prerequisites":["host-cpu","streaming-multiprocessor"],"learningObjective":"Apply Amdahl’s bound to distinguish acceleration of device work from whole-program speedup.","engineeringExample":"Synthetic program spends 90% of its original time in work accelerated by 10×, with the other 10% unchanged. Ideal speedup is 1/(0.1 + 0.9/10) ≈ 5.26×. Even unlimited acceleration of that 90% would cap the same fixed workload at 10×.","evidenceNotes":"This applies the lesson’s fixed-workload model, not a guaranteed CUDA speedup. Transfers, launches, changed algorithms, problem size and CPU contention can change both the parallel fraction and overhead.","labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}