SIMT execution and divergent branches

The mechanism

A GPU kernel describes many threads, but execution resources issue instructions to groups of threads. Threads can have different addresses and control decisions; the machine uses active masks to determine which lanes participate in a particular instruction. When a group takes different branches, the paths may require separate masked execution. Coalescing is a different concern: neighboring addresses can be combined into efficient memory transactions even when the arithmetic is identical. NVIDIA warps and AMD wavefronts instantiate related ideas with generation-specific details. The useful comparison follows the same small operation through these groups, counting both idle lanes and transferred bytes.

Assign logical threads to an execution group under the target architecture’s documented rules.
Issue a common operation for the lanes whose predicates are active at that point.
If control flow splits, execute the required paths with masks and reconverge according to the program.
Translate each participating lane’s address into memory transactions; contiguous addresses usually waste fewer transferred bytes.
Compare useful outputs, active masks and actual transactions before attributing low performance to insufficient arithmetic units.

Science and assumptions

Masked execution efficiency

u=\frac{\sum_i a_i}{\sum_i s_i}

Masked-execution efficiency divides all active lane participations by all issued lane slots over the same instructions.

The introduced $a_{i}$ and $s_{i}$ explicitly name the original $active_{\mathrm{lanes}}$ and $issued_{\mathrm{lane,slots}}$ quantities. The denominator must be nonzero.

Symbols and units

u: Active-lane fraction over issued instructions
fraction
aᵢ: Active lanes in issued instruction i
lane participations
sᵢ: Issued lane slots available in instruction i
lane slots
i: Issued-instruction index
integer index

Count lane participation over issued instructions. The resulting fraction measures mask waste and has no time units.

Assumptions Equal-cost instructions in this toy metric; real latency and throughput differ across instructions.

Engineering example

Synthetic eight-lane group: four lanes require one ten-instruction branch and four require a different ten-instruction branch. If both paths execute under masks, 80 useful lane-instructions occupy 160 issued slots.

Distinctions and tradeoffs

Divergence and uncoalesced memory access are separate effects; either can occur without the other.

Grouping similar requests can reduce divergence and scattered accesses, but rearranging them costs time and can increase request latency.

Irregular token lengths, sparse experts and gathers can leave SIMD-style resources underused even when a model has a large total operation count.

Specifications and interfaces

Active-lane fraction: Useful participating lanes / group width
A dimensionless instruction-level metric, distinct from chip utilization.
Transaction efficiency: Requested bytes / transferred bytes
Depends on address alignment, access size and memory behavior.

Interfaces

Kernel threads Input: Per-thread values, predicates and addresses
Execution group Output: Masked instructions and memory transactions

Check your understanding

All threads run the same instruction but read widely separated addresses. What is the likely issue?

Memory transaction inefficiency
Necessarily branch divergence
Mandatory out-of-order retirement

Answer and reasoning

Memory transaction inefficiency

Uniform control can still generate many memory transactions when the addresses do not fall into shared transaction regions.

Evidence and scope

The eight-lane drawing is pedagogical, not a claimed NVIDIA or AMD execution-group width. Cross-vendor scheduling details are intentionally not equated.

Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.