The mechanism
The CPU is the general-purpose processor that organizes the system’s work. It runs operating-system services, prepares tasks, launches GPU kernels, handles control-heavy code, and coordinates data and networking. GPUs accelerate many numerical operations, but a full AI service still needs request handling, tokenization, orchestration, memory management, and failure recovery. In a Grace Blackwell system, the CPU and GPUs have a coherent high-bandwidth connection; in other servers, the host path commonly uses PCIe. Neither arrangement makes software coordination free. The CPU can issue a workload, carry on with other work while the GPU executes, and react to the reported completion; coordination and data preparation still cost time. Separate control traffic from the much larger movement of tensor data so the division of labor becomes clear.
- The CPU runs the software that receives requests, prepares data, and organizes the computation.
- It dispatches suitable numerical work to GPUs while continuing other control and system tasks.
- Completion events and results return to the software pipeline for subsequent processing, response delivery, or recovery.
Science and assumptions
The unaccelerated fraction limits total speedup
Overall speedup is the reciprocal of the unaccelerated fraction plus the accelerated fraction divided by its speedup.
Symbols and units
- S
- Overall fixed-workload speedup
dimensionless
- f
- Baseline execution-time fraction accelerated
fraction
- s
- Speedup applied to that fraction
positive dimensionless ratio
f is the fraction of original execution time that a proposed accelerator can speed up, and s is its speedup for that portion. The remaining fraction includes tasks such as preparation, dispatch, serialization, or service logic. Even an arbitrarily fast GPU cannot eliminate time spent elsewhere. Measuring the complete application reveals which part deserves optimization next.
Assumptions Amdahl’s fixed-workload model with unchanged serial work and no added transfer or coordination overhead. Real batching, overlap, and workload scaling can alter the fractions; a peak hardware throughput ratio is not automatically s.
Engineering example
If 10% of a workload is serial, infinite acceleration caps ideal speedup at 10× in Amdahl’s expression.
Distinctions and tradeoffs
A fast GPU does not remove the need for general-purpose system software. CPU preparation, dispatch, and I/O coordination can still limit the overall service.
A faster accelerator can expose CPU scheduling or preprocessing bottlenecks. Moving more work onto the GPU helps some pipelines while making others harder to coordinate.
An AI model becomes a usable service only when its numerical work is integrated with ordinary software tasks. The CPU supplies much of that general-purpose coordination.
Specifications and interfaces
- Grace core count
- 72 Arm Neoverse V2 cores
Per Grace CPU in the official GB200 configuration.
- GB200 pairing
- 1 Grace CPU + 2 Blackwell GPUs
Per Grace Blackwell Superchip.
- CPU–GPU interconnect
- 900 GB/s raw bidirectional NVLink-C2C
Interface rating; useful application bandwidth is workload dependent.
Interfaces
- User/system work Input
- Requests, programs, and operating-system events
- Accelerator dispatch Output
- Kernel launches, descriptors, and control
- Host memory Input
- Program state and prepared data
Check your understanding
Which task naturally belongs to the CPU’s coordination role?
- Serving as the liquid coolant.
- Running the operating system and launching accelerator work.
- Storing all HBM bits in one register.
Answer and reasoning
Running the operating system and launching accelerator work.
The CPU integrates accelerated numerical work into a functioning software system, including control-heavy and operating-system responsibilities.
Historical context
1971 · Intel introduces the 4004 microprocessor
Intel introduced the 4004 in November 1971. Integrating a programmable CPU on a chip made it practical to reuse a hardware design for different instruction sequences, advancing the general-purpose control layer that still coordinates modern accelerator systems.
Evidence and scope
The model excludes communication, memory and changing problem size.
Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.