# Network interface adapter Canonical: https://semiaifoundry.com/learn/atlas/network-adapter/ Revision: 2026-10-07 Rights: https://semiaifoundry.com/ai-atlas/licenses.html Connectivity · Atlas v1.3.0 # Network interface adapter Moves data between the server and the cluster network, often with direct-memory and protocol offloads. [Explore in Atlas](https://semiaifoundry.com/ai-atlas/#network-adapter)Centimeter-scale adapter or module with external copper/optical links Learning objectiveSeparate fixed message startup time from payload transfer time on a network path. ## The mechanism A NIC converts server memory transfers into network packets and receives packets from other machines. High-performance adapters can perform remote direct memory access, allowing registered buffers to be transferred with less involvement from the remote CPU. GPU-aware paths can also reduce unnecessary staging through host memory when the platform and software support them. These capabilities do not eliminate synchronization, congestion, or network topology. On a supported path, a tensor fragment leaves a GPU buffer, passes through the adapter and reaches a remote buffer; control and synchronization govern when the transfer is usable. The NIC belongs to the scale-out path between systems; it should not be confused with NVLink’s local GPU fabric or with the ordinary management network. 1. Software or a supported accelerator path identifies buffers and initiates a network operation. 2. The NIC reads data, applies the appropriate protocol processing, and transmits packets through the network. 3. The remote adapter places received data into supported buffers and reports completion so dependent work can continue. ## Science and assumptions ### Small messages and large transfers have different bottlenecks $$ T_{\mathrm{message}}\approx\alpha+\frac{M}{B_{\mathrm{payload}}} $$ Message time is approximately fixed startup latency plus payload size divided by sustained payload bandwidth. Symbols and units T\_message Message completion time in the path model s α Fixed software/adapter/path startup latency s M Payload size bytes B\_payload Sustained directional payload bandwidth bytes/s α represents fixed software, adapter, and path latency; M is payload bytes; Bpayload is sustained payload bandwidth in bytes per second. For small M, startup dominates. For large M, bandwidth dominates. This explains why buying a faster line-rate adapter may help bulk gradient transfers more than tiny synchronization messages, and why scheduling and aggregation matter. Assumptions A single uncontended path and approximately constant startup cost. Congestion, multiple hops, retransmission, protocol offloads, and overlap can change both terms. Convert bits per second to bytes per second before comparing them. ## Engineering example Synthetic uncontended transfer: α = 5 μs and payload bandwidth = 25 GB/s. A 1 MB message needs 40 μs of serialization, giving about 45 μs total. A 1 kB message needs only 0.04 μs of serialization but still about 5.04 μs total, so startup dominates. ## Distinctions and tradeoffs A 400 Gb/s port is not a 400 GB/s port. Dividing by eight gives 50 GB/s before network and protocol overhead. Faster ports help only if the host path and network can sustain them. Queueing, congestion control, and placement can dominate tail latency even below the nominal line rate. Distributed training exchanges gradients and parameters across servers. Distributed inference and storage access also depend on efficient movement beyond one rack’s local GPU domain. ## Specifications and interfaces ConnectX-7 example Up to 400 Gb/s throughput NVIDIA adapter family rating; selected card and port modes vary. Byte-rate equivalent 50 GB/s at 400 Gb/s Arithmetic conversion before protocol overhead; not 400 GB/s. Acceleration RDMA / RoCE and offloads Availability depends on adapter, firmware, topology, and software. ### Interfaces Local memory transfer Input Registered host or supported GPU buffers Network packets Output Tensor, storage, and application traffic Received packets Input Remote data and protocol responses ## Check your understanding ### What does RDMA principally reduce in a supported transfer? 1. The remote CPU’s work copying and processing the payload. 2. The number of bits in the model vocabulary. 3. The need for any network fabric. Answer and reasoning The remote CPU’s work copying and processing the payload. RDMA places data directly into registered memory with less remote CPU intervention. It still requires a functioning network, protocol, and synchronization. ## Evidence and scope The lesson’s latency-plus-payload model uses decimal bytes and one direction. These are synthetic rates; the linked network references do not make this a promise about RDMA, GPU access, congestion or any deployed adapter. Vary ranks, payload, effective bandwidth and startup in an ideal ring all-reduce; this is not a topology-specific or every-collective traffic model. ## Sources - [NVIDIA ConnectX Ethernet adapters](https://www.nvidia.com/en-us/networking/ethernet-adapters/) - [NVIDIA ConnectX-7 adapter hardware manual](https://networking-docs.nvidia.com/connectx7hw/introduction) [Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/network-adapter-2b661ddb0588a022.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html) ## Continue exploring ### Foundations - [PCI Express interconnect](https://semiaifoundry.com/learn/atlas/pcie-interface/) ### Related lessons - [NVLink GPU interface](https://semiaifoundry.com/learn/atlas/nvlink-port/) - [Host CPU](https://semiaifoundry.com/learn/atlas/host-cpu/) - [Compute baseboard and tray](https://semiaifoundry.com/learn/atlas/compute-baseboard/) - [Leaf switch and endpoint network](https://semiaifoundry.com/learn/atlas/leaf-switch/)