Atlas lessons.

Advanced packaging

• GPU logic dies

  Integrate transistor circuits into the compute, storage, and communication structures of one accelerator.

• Interposer and redistribution layers

  Connects logic dies and memory through dense package-level wiring.

• Microbumps and bonding joints

  Join matching electrical terminals at the surfaces of dies and package interconnect layers.

• Package substrate

  Fans dense chip connections out to a mechanically supported interface with the circuit board.

• Through-silicon via

  Provides a vertical electrical connection through silicon between stacked layers.

Agent systems

• GPT-6 Astra

  Connect documented model capabilities to a larger system of context, tools, and evaluation.

• Tools, memory & agent loop

  Connect model inference to actions and evidence.

Arithmetic circuits

• Full adder and carry network

  Combines binary operands and propagates the carry needed for exact integer addition.

• Multiply–accumulate and fused arithmetic

  Combines products into the accumulated sums used by dot products and matrices.

• Binary multiplier and partial-product reduction

  Builds a product by generating and combining weighted partial products.

Atomic foundations

• Diamond-cubic silicon lattice

  Organizes atoms into a repeatable semiconductor crystal.

• Donor and acceptor doping

  Changes carrier populations by introducing selected impurity atoms.

• Energy bands and carriers

  Explains how a semiconductor can be electrically controlled.

• Silicon atom

  Supplies the electronic states from which semiconductor behavior is engineered.

Authority, context and execution boundaries

• Context, retrieval and durable memory

  Separate context window, retrieved passages, workflow state, durable user data and model parameters.

• A tool protocol is a boundary

  Version-pinned MCP host/client/server roles, schema discovery, invocation and errors.

• Execution boundaries for code and tools

  Filesystem, process, network and secret boundaries; authorization is checked at actual execution points.

• User authority and service identity

  User authorization, workload authentication, token audience, scope and resource policy are separate checks.

• Retrieved text is data with a trust boundary

  Track untrusted retrieval provenance and enforce tool authority outside model output.

Chip-to-hall integration

• Die-to-die links and UCIe compatibility

  Defines what adjacent chiplets must agree on to exchange useful data.

• Collective traffic and congestion

  Coordinates distributed tensor exchanges while accounting for topology and contention.

• Signal integrity, timing and the eye diagram

  Tests whether a receiver can distinguish transmitted symbols under actual channel conditions.

• Optical transceivers and co-packaged optics

  Changes where electrical signals become optical while retaining a complete link budget.

• PCIe and CXL: transport versus memory semantics

  Distinguishes peripheral data transfer from supported coherent memory operations.

• Power integrity and transient voltage margin

  Keeps the voltage at a chip within its operating window during changing current demand.

• A documented rack as a system boundary

  Connects compute trays, switches, management, power and cooling in one named reference.

• Scale-up, scale-out and management fabrics

  Assigns each traffic class to its actual interconnect and resource boundary.

Compute architecture

• Streaming multiprocessor

  Schedules many threads and supplies the execution units and nearby storage that make GPU parallelism useful.

• Tensor Core matrix unit

  Multiplies small matrix tiles and accumulates their products with supported numerical formats.

• Tensor Memory Accelerator

  Moves multidimensional tensor tiles asynchronously so compute threads can spend less effort copying data.

Connectivity

• Network interface adapter

  Moves data between the server and the cluster network, often with direct-memory and protocol offloads.

• NVLink GPU interface

  Moves tensor data between GPUs through a high-bandwidth scale-up interconnect.

• On-chip interconnect fabric

  Routes data and control between compute units, cache slices, memory controllers, and I/O.

• PCI Express interconnect

  Connects host processors, accelerators, storage, and network adapters through standardized serial links.

Contrasting computational structures

• Audio sampling, windows and spectra

  Recover a finite frequency representation and diagnose sampling ambiguity.

• Recurrent state and convolution

  Carry history in a state; recover the same output by an explicitly unrolled kernel.

• Local filters and patch attention

  Compare fixed local receptive fields with content-weighted patch aggregation.

Custom silicon ecosystem

• PDKs, synthesis and physical closure

  Converts logical behavior into a manufacturable layout under electrical constraints.

• RTL and functional verification

  Checks that a cycle-based design obeys its specification before manufacture.

• Tapeout, bring-up and productization

  Coordinates design release, manufacturing partners and software into a usable product.

• Custom-chip volume economics

  Compares lifecycle costs per useful result under explicit volume and performance assumptions.

• Workload contract and architecture selection

  Turns desired service behavior into measurable hardware and software requirements.

• Full memory hierarchy and input path

  Tracks data through storage, host memory, accelerator memory, caches and operand storage.

• Choosing SRAM, HBM, DDR, LPDDR or GDDR

  Matches memory technology and package integration to a workload’s access pattern.

• Historical memory-wall evidence

  Separates historical fitted compute, bandwidth and capacity trends from workload diagnosis.

Data pipeline

• Dataset and checkpoint storage hierarchy

  Persists training data, model state, and results beyond accelerator working memory.

Data-center fabric

• Leaf switch and endpoint network

  Connects compute or service endpoints to the broader data-center fabric.

• Optical transceiver and fiber link

  Converts electrical communication into light for transport through fiber.

• Spine switching and fabric topology

  Provides multiple high-capacity paths between leaf switches.

Data-center operations

• Checkpointing, failure and retained useful work

  Balances checkpoint overhead against lost progress and recovery delay.

• Storage paths for datasets and checkpoints

  Coordinates persistent data, local caching and durable state transfer without confusing their roles.

• Job admission, topology and resource fragmentation

  Places complete jobs onto compatible resources while balancing utilization and service goals.

Data-center physical systems

• Commissioning a usable AI cluster

  Requires accepted electrical, thermal, network, software and operational behavior together.

• Compatible supply and converging bottlenecks

  Converts independently manufactured inputs into a common finished-system basis.

• Cold plates, manifolds and separated coolant loops

  Transports heat through controlled fluid circuits without treating circulation as consumption.

• Facility heat rejection and cooling tradeoffs

  Transfers collected heat to the outdoor environment under weather-dependent limits.

• Grid interconnection and deliverable electrical capacity

  Connects facility demand to location-specific network, equipment and operating readiness.

• Production ramps, inventory and allocation

  Tracks time-dependent output and distinguishes qualified supply from a customer’s accessible share.

• Dynamic load, reserve and power control

  Accounts for changes in electrical demand over timescales from chip activity to facility operation.

• Rack power conversion and distribution

  Moves electrical energy from facility supply through conversion stages to low-voltage chip rails.

Delegated tasks and coordination

• Delegated tasks, messages and artifacts

  Task lifecycle, artifact ownership, input-required states, cancellation and authenticated service boundaries.

• Coordination can add work

  Parallel critical path, duplicated effort, state merge, correlated error and budgets.

Device architectures

• Fin field-effect transistor

  Raises the channel into a fin so the gate controls multiple surfaces.

• Gate-all-around nanosheet transistor

  Surrounds thin semiconductor sheets with a gate for stronger channel control.

Device physics

• Controllable semiconductor channel

  Forms the field-controlled conduction path inside the transistor.

• Metal gate and high-k dielectric

  Couples a control voltage into the channel while suppressing gate leakage.

• Metal–oxide–semiconductor field-effect transistor

  Uses a gate voltage to control current between source and drain.

• P–n junction and depletion region

  Creates an internal electric field between differently doped regions.

• Source and drain regions

  Connects the controlled channel to reservoirs that supply and collect carriers.

• Gate sidewall and inner spacers

  Separates the gate from neighboring terminal structures and limits parasitic coupling.

Devices and logic

• Loaded CMOS transfer, noise and transition delay

  Compare the same cell’s DC voltage-transfer contract with its explicitly loaded charging and discharging behavior.

• Oxide variation to electrical cell acceptance

  Carry a named material specimen into electrical parameters and separate acceptance gates.

Digital circuits

• Clock tree and gated timing domains

  Distributes capture timing to sequential circuits while managing switching activity.

• Edge-triggered D flip-flop

  Captures a bit at a clock event and retains it for the next computation stage.

• Complementary CMOS inverter

  Turns one binary voltage state into its complement.

• Two-input CMOS NAND gate

  Implements a universal Boolean operation with complementary transistor networks.

• Six-transistor static memory cell

  Retains a local bit using powered feedback and controlled access devices.

Distributed software

• Collective communication library

  Coordinates group communication operations across participating GPUs.

• Data, tensor, pipeline, and state parallelism

  Splits model work and state across devices to fit memory and exploit concurrent execution.

Electrical infrastructure

• Rack power shelf and DC busbar

  Converts incoming power and distributes controlled DC supply throughout the rack.

• Uninterruptible power supply and backup chain

  Maintains power continuity across disturbances and transitions to other supply sources.

Evidence and frontiers

• Evaluation & the AGI frontier

  Measure breadth, reliability, and practical competence.

Executable software

• Completion, reset and execution epochs

  Separate physical work, accepted completion and useful output while a device is stopped and reinitialized.

• Uneven batches and the global gradient

  Preserve the intended training objective when ranks contribute different numbers of valid examples or tokens.

• DMA visibility, ordering and buffer lifetime

  Separate coherent values, ordered descriptor publication and ownership of in-flight storage.

• Firmware handoff and device readiness

  Separate device compatibility, driver discovery, resource ownership and queue readiness before any operand is consumed.

• Floating-point reductions and compiler permissions

  Distinguish algebraic equivalence, numerical error and bitwise reproducibility when compilation or collectives regroup arithmetic.

• IOMMU permission and tensor identity

  Distinguish protection validity from the semantic identity of a matrix operand.

• Process, physical and device address spaces

  Trace the same allocated bytes through separate CPU virtual and device-visible mappings.

• Sharded matrix multiplication and result ownership

  Derive the communication operation from the partitioned matrix axis and the next consumer’s required result layout.

• Speculative decoding: proposals and verified output

  Account for target-distributed output tokens while separating draft work, verification and actual elapsed time.

Execution software

• Compiler, intermediate code, and device executable

  Transforms program descriptions into executable instructions and runtime-compatible artifacts.

• CUDA programming and runtime layer

  Provides a software model and interfaces for executing parallel work on NVIDIA GPUs.

• GPU kernel and thread hierarchy

  Maps an operation onto many coordinated logical threads or tiles.

Fabrication

• Chemical mechanical planarization

  Restores controlled surface planarity between fabrication stages.

• Lithography and pattern transfer

  Turns a circuit pattern into spatially selected regions on a wafer.

• Wafer and fabrication hierarchy

  Provides the shared substrate on which many integrated circuits are fabricated.

Fabrication / BEOL and co-design

• Backside power and design–technology co-optimization

  Changes physical power routing and evaluates the device, layout and system consequences together.

• BEOL layers, damascene and vias

  Builds a multilayer conductor network above completed device terminals.

• Wire resistance, low-k dielectrics and reliability

  Connects shrinking wire geometry to delay, coupling and current-density limits.

Fabrication / Control, test and ramp

• Metrology, inspection and process windows

  Measures geometry and defects at the stage where a manufacturing decision can still be made.

• Cycle time, ramp and effective capacity

  Connects tool throughput, queues, qualification and yield learning over time.

• DFT, probe, binning and known-good die

  Uses designed test access and measurements to screen material before costly assembly.

• Wafer maps and good-die yield

  Separates geometrical die count, defects, electrical limits and screened output.

Fabrication / FEOL and MOL

• Cleaning and CMP as interface preparation

  Removes residues and topography while controlling loss, shape and defects.

• How a FinFET fin is formed and integrated

  Turns a three-dimensional silicon ridge into a controlled transistor channel.

• Implantation, activation and diffusion

  Introduces dopants and uses heat to repair damage and establish electrical activity.

• Nanosheet release and replacement gates

  Creates channels that can be surrounded by a gate after sacrificial materials are removed.

• Source/drain, spacers, contacts and MOL

  Connects a gated channel to useful local wiring without bypassing gate control.

• Wells and shallow trench isolation

  Separates neighboring devices electrically before their active structures are completed.

Fabrication / Factory systems

• Cleanroom and subfab utilities

  Supplies the environmental and support systems required by process tools.

• MES, dispatch and process release

  Coordinates material routes, equipment eligibility, holds and measurement decisions.

• A process chamber as a controlled reactor

  Transforms a wafer through coupled gas, energy, surface and measurement systems.

• Wafer handling, carriers and genealogy

  Moves fragile material while retaining identity, orientation and process history.

Fabrication / Materials

• Contamination and the state of a surface

  Tracks particles, unwanted chemistry and surface changes across process boundaries.

• Electronic-grade silicon and crystal growth

  Converts purified feedstock into an oriented single-crystal substrate.

• Epitaxy and engineered starting layers

  Grows a crystalline layer whose composition and doping serve the later device.

• Wafer finishing, orientation and geometry

  Turns an ingot slice into a clean, geometrically controlled starting surface.

Fabrication / Repeated operations

• ALD and self-limiting surface reactions

  Builds a film through separated chemical exposures and controlled surface saturation.

• CVD and PVD: two ways to supply a film

  Contrasts chemical surface growth with physical material transport.

• DUV, EUV and high-NA imaging

  Explains the coupled resolution and focus consequences of wavelength and numerical aperture.

• Masks, OPC and pattern-aware design

  Pre-distorts a mask so the process prints a desired wafer pattern.

• Plasma and wet etch: selectivity and profile

  Removes selected material while preserving the feature and surrounding stack.

• Coat, bake, expose and develop

  Creates a temporary resist pattern that guides the following material operation.

• Thermal oxidation and the silicon interface

  Converts surface silicon into oxide while moving a buried interface.

From request to durable action

• Durable actions and the ambiguous timeout

  Persist intent, execute with idempotency key, store receipt and recover without duplicate business effects.

• Events, queues and backpressure

  Distinguish event occurrence, delivery, acknowledgement, deduplication, retries and admission.

• Human review, interruption and cancellation

  Approve a concrete versioned action; stale approval invalidates on change; distinguish request, acknowledgement and committed effects.

• Traces connect cost, latency and results

  Parent/child span identity, events, retries, token/byte counts and independent result evaluation.

• Tool schemas and semantic validation

  Syntax, types, domain constraints, resource identity, output schema and result errors.

• Workflow graphs and explicit state

  Compare a fixed DAG/state machine, model-assisted node and model-directed tool choice on one task.

Frontier infrastructure

• Orbital computing evidence ladder

  Separates onboard processing, flight demonstrations, laboratory tests and proposed services.

• Orbital lifecycle economics and workload fit

  Compares complete useful service under conditional launch and support assumptions.

• Orbital optical links and contact budgets

  Separates inter-satellite exchange from ground-data delivery and sustained useful service.

• Radiation effects, detection and recovery

  Connects mission-specific particle exposure to device behavior and retained workload correctness.

• Radiators, view factors and orbital heat rejection

  Rejects spacecraft waste heat through radiation while accounting for absorbed environmental heat.

• Launch, replacement and end-of-life mass

  Accounts for a computing fleet’s support mass and continuing lifecycle logistics.

• Orbital solar generation and eclipse storage

  Balances payload and spacecraft energy through sunlight, eclipse and degradation.

Generation

• Key–value cache

  Reuse earlier attention state during generation.

• Output head & decoding

  Convert the final representation into the next token.

Global AI infrastructure

• Facility phases and geographical evidence

  Keeps physical sites, organizational names and development stages distinct before counting capacity.

• Energy forecasts, geography and measurement boundaries

  Distinguishes estimated electricity use from conditional demand projections.

• Siting, regional infrastructure and access to compute

  Connects location choices to service paths, resources and who can actually use capacity.

• Water withdrawal, consumption and cooling circulation

  Accounts for local water effects with a defined physical and energy denominator.

Host platform

• DDR5 server memory

  Provides expandable main memory for conventional host CPUs and their software working sets.

• Host CPU

  Runs the operating system and coordinates the workloads, data preparation, and I/O surrounding accelerated computation.

• Grace LPDDR5X memory

  Provides the Grace CPU’s local main memory with short, energy-conscious package and board connections.

• NVMe solid-state storage

  Persists datasets, model checkpoints, and software beyond power cycles.

Infrastructure

• Qualify the memory system and retain useful work

  Separate component screening, integration qualification, commissioning and sustained service evidence.

• From memory placement to network, power and cooling

  Trace how a tensor-placement decision changes link demand and useful service at the hall boundary.

• When completed work survives a restart

  Distinguish completed, restart-durable, unfinished and lost work under a declared failure schedule.

Learning

• Loss, gradients & optimizer

  Turn prediction errors into parameter updates.

• Post-training & feedback

  Shape a pretrained model into an assistant.

• Scaling laws & resource allocation

  Understand how data, parameters, and compute interact.

• Training data pipeline

  Convert a source collection into useful learning examples.

Materials and wiring

• Device contact and interface resistance

  Bridges semiconductor terminals to local metallic wiring.

• Copper conductor and barrier

  Carries electrical signals and current through much of the chip wiring stack.

• Insulators and low-k films

  Separates conductors while shaping capacitance and electric fields.

• On-chip metal and via hierarchy

  Connects cells through a multilayer electrical network.

Memory

• HBF latency, transfer granularity and prefetch

  Explain why matching read bandwidth does not guarantee matching request latency or useful-byte efficiency.

• HBF: a read-oriented tier and its evidence boundary

  Distinguish an announced HBF interface/capacity specification from validated workload behavior.

• HBM layer count and physical height

  Separate memory-die count, package thickness, stored bits and interface bandwidth.

• HBM heat, assembly yield and test access

  Explain how added layers affect the heat path, screening strategy and delivered good stacks.

Memory / HBM fabrication and assembly

• DRAM cells, arrays and peripheral circuits

  Creates charge-storage cells and the circuitry needed to address, sense and restore them.

• HBM stack assembly, repair and staged test

  Combines screened memory dies into a delivered stack using a specified joining route.

• TSV formation, keep-out and backside reveal

  Builds insulated vertical conductors and exposes their opposite ends for stacking.

Memory / HBM system constraints

• Base die, PHY and memory controller

  Separates memory-stack logic from host scheduling and the electrical interface.

• HBM generations, channels, refresh and ECC

  Compares capacity and interface organization without confusing physical layers with logical channels.

• HBM silicon intensity and allocation

  Estimates wafer demand per delivered bit without double-counting embedded yield losses.

Memory hierarchy

• DRAM die and memory banks

  Stores dense arrays of charge-based bits and organizes them into banks for read, write, and refresh.

• HBM controller and PHY

  Schedules DRAM commands and translates on-chip requests into the electrical signals of the memory interface.

• High-bandwidth memory stack

  Stacks DRAM vertically beside the processor to provide large capacity through a very wide local interface.

• L1 and texture cache

  Automatically retains recently accessed data and combines nearby thread requests near an SM.

• Shared L2 cache

  Provides a larger on-chip reuse layer between SM traffic and HBM.

• Register file

  Keeps each active thread’s immediate values close to the arithmetic pipelines.

• Shared memory SRAM

  Lets cooperating threads explicitly stage and reuse a tile near the compute units.

• Tensor Memory

  Provides dedicated on-chip storage for fifth-generation Tensor Core operands and accumulators.

Model deployment

• Quantization and numerical representation

  Represents model values with reduced precision to lower storage and arithmetic costs.

• Inference serving and request scheduling

  Turns incoming model requests into scheduled prefill and generation work.

Models become evaluated applications

• A code patch needs execution evidence

  Repository context, patch application, sandboxed commands, hidden tests and result artifacts.

• Document fields become validated records

  Document layout, transcription/extraction, schema normalization, confidence and cross-field constraints.

• Model artifacts, rollout and drift

  Version weights, normalization, feature schema and thresholds; separate deployed pointer, observations and labels.

• Ranking changes what is observed

  Candidate retrieval, sparse embeddings, ranking, exposure and feedback bias.

• Retrieval from source document to answer

  Chunking, lexical versus dense retrieval, provenance, reranking and answer validation are distinct stages.

• A learned field still obeys an evidence boundary

  Train on a bounded parameter family; compare prediction with a numerical reference, boundary conditions and conservation residuals.

• Speech is a timed interaction

  Audio frames, transcription revisions, endpointing, interruption and tool confirmation.

• Forecasts are tested forward in time

  Horizon, rolling origin, seasonal baseline, probabilistic intervals and leakage.

Model–machine co-design

• IO-aware attention and online softmax

  Computes attention while reducing large intermediate transfers to outer memory.

• Compiler layout, lowering and buffer planning

  Chooses memory organization and hardware operations that implement a tensor graph.

• Autoregressive decode and weight traffic

  Produces dependent new tokens while repeatedly accessing weights and growing KV state.

• Prefill/decode disaggregation and useful serving rate

  Separates workload phases when independent scheduling benefits exceed the state-transfer cost.

• MoE routing, active parameters and resident state

  Balances conditional expert computation against placement, communication and capacity.

• Model graph and tensor intermediate representation

  Preserves operation semantics before committing to a device instruction schedule.

• Kernel tiling, fusion and operand reuse

  Keeps intermediate results close to their next consumer where resource limits allow.

• Parallelism as a memory and communication budget

  Chooses tensor, pipeline, data and state partitions under topology and dependency constraints.

• Prefill and prompt processing

  Processes many known prompt tokens with substantial potential for weight reuse.

• Runtime, driver and device dispatch

  Turns compiled work into ordered execution with real buffer lifetimes and completion events.

• Training state and activation rematerialization

  Budgets weights, gradients, optimizer state and saved activations across a training step.

Observations become physical actions

• Learned action policies need embodiment evidence

  Observation/action encoding, training trajectory distribution, policy rate and downstream control constraints.

• Feedback closes the action loop

  Sampled controller, actuator limits, delayed sensing and disturbance.

• Goals, plans and actions are different objects

  Goal constraints, collision-aware planning, timed execution, feedback and cancellation.

• Robot observations have coordinates and age

  Frame transforms, timestamps, measurement uncertainty and stale-state consequences.

Operations and reliability

• Baseboard management controller

  Monitors and controls hardware through a management path separate from the main workload.

Packaging

• Selecting a bonding route under constraints

  Compare qualified output, height, thermal behavior, inspection and cost on the same product boundary.

• Hybrid bonding: surface-to-interface closure

  Follow CMP, cleaning, alignment, contact and anneal as one surface-controlled integration problem.

• Mass reflow and molded underfill

  Separate stack placement, metallic reflow and molding in an MR-MUF assembly route.

• Thermocompression with non-conductive film

  Trace a bump-based TC-NCF route and separate polymer behavior from electrical contact formation.

Packaging / Assembly and reliability

• Microbumps, reflow and thermocompression

  Forms electrical joints while controlling placement, local contact and material transformation.

• Hybrid bonding: surface state and alignment

  Joins dielectric surfaces and metal connections across a carefully prepared interface.

• Underfill, warpage, thermal paths and qualification

  Connects package materials and stress histories to measurable failure mechanisms.

• Temporary bonding, thinning and singulation

  Supports fragile wafers or dies while exposing backside structures and separating units.

Packaging / Interposers, bridges and stacking

• CoWoS-R and CoWoS-L: RDL and local silicon

  Distinguishes polymer/copper redistribution from RDL combined with dense local silicon links.

• CoWoS-S and the silicon interposer route

  Integrates logic and memory over a shared silicon routing structure.

• EMIB and localized die-to-die bridges

  Places fine-routing silicon near adjacent die edges inside a substrate.

• SoIC and Foveros: vertical integration families

  Compares named die-stacking routes while preserving their base-die and bonding differences.

Packaging / Materials and assembly

• Flip-chip assembly and organic substrates

  Connects face-down dies to a multilayer package substrate with a board-facing interface.

• Redistribution layers and fan-out packaging

  Routes die connections through added layers that can extend beyond the die footprint.

Physical design

• Standard-cell library and physical design

  Packages characterized transistor circuits into reusable building blocks for chip layout.

Power delivery

• Power supply and power shelf

  Converts facility power into the regulated DC distribution used by the compute rack.

• Voltage regulator module

  Converts a distribution voltage into tightly controlled low-voltage rails for processors and memory.

Processing architectures

• Analog and photonic arithmetic research

  Explores physical multiply accumulation with explicit conversion and accuracy limits.

• Trainium and Inferentia execution paths

  Combines specialized engines, local buffers and a compiler-managed accelerator runtime.

• Cache coherence and NUMA placement

  Keeps shared cache lines consistent while preserving awareness of physical memory location.

• Out-of-order CPU pipeline

  Finds ready independent instructions while preserving architectural results.

• Scalar, vector and CPU matrix units

  Executes different shapes of arithmetic within a general-purpose processor.

• DPU and infrastructure offload

  Moves selected networking and infrastructure work away from the host CPU.

• FPGA fabrics, DSP blocks and adaptive pipelines

  Configures a spatial datapath around a stream and its timing constraints.

• GPU chiplets and memory locality

  Partitions compute and I/O physically while presenting a programmable accelerator.

• Occupancy and asynchronous tile pipelines

  Keeps work ready while controlling registers, scratchpad capacity and synchronization.

• Custom inference silicon: Maia and MTIA

  Allocates compute and memory around a provider’s measured inference needs.

• SIMT execution and divergent branches

  Shares instruction issue across threads while tracking which lanes participate.

• Systolic arrays and tensor dataflow

  Reuses operands through a regular network of multiply–accumulate elements.

• Wafer-scale compute and distributed SRAM

  Keeps a very large compute fabric connected across a wafer-scale implementation.

Rack integration

• NVLink copper cable cartridge

  Provides dense physical signal connections between compute and switch trays.

• GB200 compute tray

  Packages CPU–GPU compute and local support hardware into a serviceable rack unit.

• NVLink switch tray and scale-up fabric

  Routes high-bandwidth GPU communication inside the rack-scale NVLink domain.

• Integrated accelerator rack: GB200 NVL72 example

  Integrates compute, communication, power, and cooling into one coordinated physical system.

Representation

• Embedding table

  Give discrete tokens a trainable numerical representation.

• Position representation

  Make sequence order available to attention.

• Tokenizer

  Turn text into a sequence of discrete IDs.

Software

• Place the working set, not just the model

  Assign tensor state to tiers using lifetime, mutation, reuse distance, access size and latency requirements.

• MoE experts: resident capacity, routing and misses

  Explain why sparse activation reduces arithmetic without eliminating expert storage or transfer constraints.

• Prefill, decode and the moving KV working set

  Place active and reusable inference state according to the execution phase and service deadline.

• Training state across HBM, host memory and storage

  Distinguish mutable training state, recomputation, offload and durable checkpoints.

System integration

• Compute baseboard and tray

  Integrates processors, memory, adapters, power, cooling, and service interfaces into an operational node.

• Grace Blackwell Superchip

  Couples one Grace CPU with two Blackwell GPUs so host and accelerator work can share a tightly integrated system.

System operations

• Distributed checkpoint and recovery state

  Persists enough execution state to resume a model workflow after interruption.

• Cluster scheduler and resource allocation

  Matches queued workloads to available compute resources under operational policies.

Thermal infrastructure

• Coolant distribution unit and separated loops

  Controls the IT coolant loop and transfers collected heat to another cooling system.

• Facility cooling and final heat rejection

  Carries heat from IT equipment to the outdoor environment or an approved reuse path.

Thermal path

• Liquid-cooling coldplate

  Captures heat at the processor and transfers it into flowing coolant.

• Lid and heat spreader

  Protects the package and spreads concentrated die heat toward a larger cooling contact.

• Thermal interface material

  Fills microscopic gaps so heat can cross from a chip or lid into a cooling surface.

Transformer computation

• Self-attention

  Exchange information between token representations.

• Feed-forward network

  Transform features within each token position.

• Mixture of experts

  Route each token to a subset of learned transformations.

• Residual stream & normalization

  Keep a deep sequence of transformations trainable.

contrasting-ai-methods

• Graph message passing and node identity

  Shows how a model can operate on relationships between entities instead of a simple token sequence.

• Search, value estimates and selected actions

  Explains how a system can spend additional computation selecting an action after receiving an observation.

• A tree ensemble that learns residual corrections

  Provides a contrasting learned prediction mechanism that does not use token attention.

data-evidence-foundations

• Controlled comparisons and confounded improvement

  Makes the design of an experiment part of the evidence for an improvement.

• Human judgments, disagreement and dataset lineage

  Makes the people and decisions behind training labels visible in the resulting dataset.

mathematical-foundations

• Floating-point range, resolution and rounding

  Explains which numerical values a finite bit pattern can represent before those values enter a learned computation.

• Linear maps, rank and retained directions

  Shows how a large array can represent fewer independent transformations than its number of entries suggests.

• Probability, likelihood and stable information

  Connects model scores to probabilities, information and the gradients that guide learning.

• Sampling variability and independent evidence

  Shows why a measured success rate varies even when the underlying teaching probability is held fixed.

• Tensor shapes, indices and contractions

  Specifies what each axis means before the same numbers are scheduled or stored on hardware.

model-lifecycle-base-model

• A differentiable computation graph

  Accumulate every branch contribution to a reused parameter and check it by finite differences.

• Assemble an initialized causal language model

  Join tensor shapes, causal probabilities, gradients and optimizer state into one reproducible trained model.

model-lifecycle-continuation

• Compatible weight averaging and regressions

  Check parameter alignment before averaging and evaluate the resulting function.

• Continue language modeling on a new domain

  Measure new-domain acquisition and old-domain retention from the same starting checkpoint.

• Context extension is a learned-use test

  Separate accepted context length, causal correctness, memory growth and demonstrated long-distance use.

• Describe a stage by its changed contract

  Classify a transition by checkpoint, data, loss, trainable state, optimizer policy and evaluation.

• Retention, replay and transfer

  Reason about a fixed-budget retention tradeoff without treating replay as a guarantee.

model-lifecycle-data

• Coverage across language and source slices

  Compare source slices under explicit byte, token and task denominators.

• Mixtures, epochs and exposure

  Calculate expected exposure per source and distinguish it from one finite seeded draw.

• Filtering, duplication and contamination

  Distinguish exact duplicates, approximate matches and evaluation contamination using explicit document families.

• Packing documents and masking valid targets

  Draw document and target masks, then reconstruct the loss denominator of a packed batch.

• Synthetic supervision and selection loops

  Track a generated example through verification and selection without converting acceptance into ground truth.

model-lifecycle-evaluation

• Human and model judgments under a rubric

  Separate agreement, task correctness and preference under a reproducible judging rubric.

• Version the entire evaluated system

  Attach an evaluation result to the complete system configuration and apply a stated rollback rule.

• Confidence, errors and refusal to guess

  Evaluate probability quality and selective error using distinct calibration and final-test data.

• From lexical retrieval to model answers with evidence

  Track evidence retrieval, answer likelihood and claim support as separate stages.

• Sampling more candidates versus selecting correct ones

  Distinguish finding a correct candidate from selecting one under a finite verification budget.

model-lifecycle-multimodal

• Train a denoiser and sample iteratively

  Compute a noisy training target and distinguish noise regression from a flow vector field.

• Pair images and captions in a shared representation

  Compute a paired image/text similarity loss and test a shortcut with held-out compositions.

• Audio representation, alignment and transcript loss

  Relate audio samples, time frames and transcript errors without mixing their denominators.

• Predict temporal representations without generating pixels

  Compare masked feature prediction with pixel generation and test temporal information use.

• Patches, projection and visual instruction tuning

  Trace visual feature dimensions into a language model and distinguish alignment from visual use.

model-lifecycle-posttraining

• DPO policy/reference likelihood ratios

  Compute DPO from policy and frozen-reference response likelihoods with an explicit pair denominator.

• Frozen base weights and trainable adapters

  Multiply a low-rank update and count trainable factors without confusing them with the training objective.

• Rollouts, advantages and PPO constraints

  Separate sampled advantages, old-policy ratios, clipping and reference KL.

• Learn a reward from pairwise choices

  Compute a pairwise preference probability and distinguish a learned score from verified correctness.

• When reward improves but the task gets worse

  Compare a rewarded proxy with independently scored task success and locate divergence.

• Instruction tuning and assistant-only loss

  Identify exactly which shifted response targets contribute to supervised fine-tuning.

• Transfer outputs without confusing size with quality

  Calculate soft-target loss and gradient, then evaluate the student separately from its teacher.

• Verifiers, grouped advantages and RLVR

  Compute relative group advantages and identify what a verifier does and does not establish.

model-lifecycle-pretraining

• Next-token supervision from raw sequences

  Construct shifted next-token targets and interpret token likelihood separately from answer accuracy.

• Gradient accumulation under a fixed global objective

  Recover a full-batch gradient from unequal target counts before clipping or stepping.

• Initialization, learning rate and stable updates

  Separate gradient clipping, adaptive moments, decay and the learning-rate schedule.

model-lifecycle-world-action

• Predict transitions conditional on actions

  Change an action while holding state fixed and compare predicted transitions with a held-out environment.

• Predict action chunks and replan with feedback

  Separate generated action horizon, executed prefix and replanning interval.

• Learn behavior from a model and test in the environment

  Compare a policy’s imagined return with return in the actual environment and identify model exploitation.

• Translate learned representations into robot action units

  Decode action tokens into physical commands with explicit frame, units and validity checks.

• The 55-year realization challenge and CRG

  Presents Semi AI Foundry’s account of a long-standing computation-to-physical-realization gap.

• Architecture is a contract between computation and silicon

  Chooses execution, memory and communication structures that can satisfy a workload contract.

• From architecture contract to a chip

  Traverses specification, RTL, verification, synthesis, physical design, tapeout and wafer test.

• Communication is the realization medium

  Maps computational state, data movement and control into a graph of physical interfaces.

• Package and system preserve the contract

  Carries chip interfaces through bonding, boards, racks, facilities and runtime services.

• The computation-to-realization gap

  Defines the problem of carrying a computational intent into a physically constrained machine.

• Verification closes the realization loop

  Connects simulation, formal checks, silicon measurement, system qualification and workload evaluation.