{"id":"cpu-vector-matrix","level":"compute","name":"Scalar, vector and CPU matrix units","shortName":"Scalar, vector and CPU matrix units","role":"Executes different shapes of arithmetic within a general-purpose processor.","description":"A scalar instruction updates one value; a vector instruction applies an operation across lanes; a matrix instruction consumes a tile containing many values. These are execution resources inside a CPU, not mutually exclusive kinds of computer. Intel AMX exposes tile storage and tile matrix multiplication, while Arm SVE lets software express vector work with predicates and a vector-length-aware model. Loads, address calculation and branches still surround the arithmetic. Efficient code arranges contiguous data, handles tails and chooses supported types so that wider arithmetic produces useful results instead of multiplying padding or waiting for operands.","specs":[{"label":"Vector occupancy","value":"Useful lanes / issued lanes","context":"Dimensionless utilization for a particular instruction."},{"label":"Arithmetic type","value":"Input and accumulator specified separately","context":"A low-precision input does not imply an equally narrow accumulator."}],"interfaces":[{"name":"Cache and load units","direction":"in","signal":"Contiguous or gathered operand vectors and tiles"},{"name":"Register or tile results","direction":"out","signal":"Partial sums consumed by subsequent operations"}],"tradeoff":"Wider units increase arithmetic throughput but raise operand demand and can leave lanes idle on short or irregular arrays.","whyAI":"CPU vector and matrix capabilities can accelerate preprocessing, embedding reductions and smaller inference batches while preserving general-purpose control.","related":["cpu-out-of-order","mac","tensor-core"],"sources":[{"title":"What Is Intel Advanced Matrix Extensions?","url":"https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/what-is-intel-amx.html"},{"title":"Arm Neoverse V2 Core Technical Reference Manual","url":"https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8"}],"scale":"Core to multi-socket server; diagrams are logical","kind":"architecture","branch":"Processing architectures","subbranch":"CPU control and locality","mechanism":["Separate loop control and address calculations from the arithmetic repeated across data elements.","Group independent elements into vector lanes and mask the final partial group rather than reading invalid data.","For a matrix operation, load tiles that permit each operand to contribute to multiple accumulations.","Select instructions whose input format, accumulator format and rounding satisfy the model’s numerical requirements.","Measure complete loop time, including layout conversion and memory access, rather than timing only the arithmetic instruction."],"misconception":"A CPU with matrix instructions remains a CPU; adding a specialized unit does not remove its control pipeline or memory hierarchy.","check":{"question":"A vector width is eight elements and the loop contains ten elements. What limits useful-lane utilization in two issued groups?","options":["Ten useful lanes out of sixteen issued lanes","All sixteen lanes produce valid outputs","Only the first eight elements can be computed"],"answer":0,"explanation":"A masked tail can compute the remaining two elements safely, but six issued lane positions perform no useful element work."},"science":[{"title":"Lane utilization","equation":"u = n / (w⌈n/w⌉)","explanation":"n is useful elements and w is lanes per instruction; u has no units. It measures tail packing, not all causes of CPU utilization loss.","assumptions":"One operation per element with fixed vector width and no additional masking."}],"prerequisites":["mac","cpu-out-of-order"],"learningObjective":"Explain why scalar, vector and matrix execution can coexist and require different data preparation.","workstream":"W07","visualFamily":"architecture-cpu","engineeringExample":"Synthetic loop: ten independent additions on an eight-lane unit need two groups, giving 10/16 = 62.5% useful lane positions. A larger batch reduces this tail fraction without changing vector width.","evidenceNotes":"Intel and Arm examples illustrate programming mechanisms. They do not establish equal precision support or a measured performance comparison.","labId":"library","scope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions.","investigationId":"execution","labScope":"No current calculator directly models this lesson’s main mechanism. Use the library for related bounds while retaining their stated omissions."}