Cache coherence and NUMA placement

The mechanism

Multiple CPU cores can cache copies of the same memory line. A coherence protocol tracks copies and coordinates ownership when a core writes. Coherence provides an agreement about values; it does not make every access equally close or eliminate software synchronization. In a NUMA system, memory controllers belong to different locality domains, so an address may be directly attached to one CPU and reached over a link from another. Thread placement and allocation policy determine which path dominates. The Linux kernel documents this distinction and exposes memory policy controls for local, bound and interleaved allocation.

Map an address to its physical NUMA node and identify where the requesting thread is running.
Look for a valid local cache copy before issuing a request toward a memory controller or another cache.
On a write, obtain the protocol’s required ownership and invalidate or update conflicting cached copies.
Place frequently used pages near their workers, or deliberately spread pages when aggregate bandwidth matters more.
Measure remote accesses and cache-line transfers; change allocation and worker affinity together when diagnosing a bottleneck.

Science and assumptions

Average access path

L_{\mathrm{avg}}=pL_{\mathrm{local}}+(1-p)L_{\mathrm{remote}}

Mean latency is the local-access fraction times local latency plus the remaining fraction times remote latency.

Symbols and units

L_avg: Mean access latency in the two-path model
ns
p: Local-path access fraction
fraction
L_local: Local-path latency
ns
L_remote: Remote-path latency
ns

p is the fraction of accesses using the local path; L values are latency in ns. The average is a first diagnostic, not a tail-latency estimate.

Assumptions Two paths and independent requests; excludes cache hits, overlap and queueing.

Engineering example

Synthetic two-path example: 75% of requests at 80 ns and 25% at 160 ns gives a 100 ns average. Moving pages changes the fractions; simply adding cores does not shorten either path.

Distinctions and tradeoffs

Cache coherence is neither a guarantee of low latency nor a substitute for locks, atomics or a language memory model.

Interleaving pages can balance memory bandwidth, while placing pages locally can reduce latency; the best choice depends on access sharing and thread migration.

Input pipelines and CPU-managed inference caches can stall accelerators when pages repeatedly cross sockets or writable metadata bounces between cores.

Specifications and interfaces

Coherence granularity: Cache line
Independent variables can still share one coherence unit.
NUMA latency: Local and remote paths
Nanoseconds depend on machine topology and contention; no universal ratio.

Interfaces

Core load/store Input: Address, ownership request and ordering requirements
Inter-socket fabric Output: Remote data or coherence ownership messages

Check your understanding

Two cores update different counters in the same cache line. Why might they interfere?

The line’s ownership may move between cores
Different variables can never interfere
NUMA forces all writes to disk

Answer and reasoning

The line’s ownership may move between cores

False sharing occurs because ownership is managed at cache-line granularity even though the variables are logically independent.

Evidence and scope

Topology illustrations omit protocol state details. Local-versus-remote numbers are invented for calculation, not measurements of a named CPU.

Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.