{"id":"dpu-infrastructure-offload","level":"compute","name":"DPU and infrastructure offload","shortName":"DPU and infrastructure offload","role":"Moves selected networking and infrastructure work away from the host CPU.","description":"A data processing unit usually combines a network interface, programmable processor resources and dedicated offload engines. Its purpose is often to handle infrastructure work such as packet steering, virtualization or storage access near the network path. NVIDIA’s BlueField documentation distinguishes DPU mode, where the embedded Arm subsystem owns NIC resources, from NIC mode. The phrase IPU can mean infrastructure processing unit in this context, but other vendors use the same initials differently. Offload improves an AI system only if it removes a measured bottleneck without creating excessive transfer, synchronization or control-plane overhead.","specs":[{"label":"Boundary","value":"Host and infrastructure processor","context":"Ownership and trust differ by operating mode."},{"label":"Service rate","value":"Packets/s or useful bytes/s","context":"Bit rate alone omits per-packet work and protocol overhead."}],"interfaces":[{"name":"Network ingress","direction":"in","signal":"Packets, descriptors and flow metadata"},{"name":"Host or accelerator path","direction":"out","signal":"Steered data and completion notifications"}],"tradeoff":"Offload can free host cores and isolate infrastructure control, while adding software lifecycle, debugging and resource-allocation responsibilities.","whyAI":"AI training needs predictable data and collective delivery; inference services also need networking and storage work beyond matrix multiplication.","related":["network-adapter","pcie-interface","runtime-device-dispatch"],"sources":[{"title":"NVIDIA BlueField DPU Modes of Operation, DOCA 2.5.3 archive","url":"https://docs.nvidia.com/doca/archive/2-5-3/nvidia%2Bbluefield%2Bdpu%2Bmodes%2Bof%2Boperation/index.html"}],"scale":"Network endpoint to wafer; electrical and optical views are conceptual","kind":"architecture","branch":"Processing architectures","subbranch":"Offload and research architectures","mechanism":["Identify which host cycles belong to infrastructure processing and which belong to the model application.","Determine the DPU operating mode and which processor owns network resources and administrative operations.","Steer supported work into dedicated engines or embedded software close to ingress.","Transfer payloads and signal completion with appropriate address translation, isolation and ordering.","Measure complete application improvement and account for fallback paths when a flow cannot be offloaded."],"misconception":"A DPU is not automatically a neural-network tensor engine, and its line rate is not a model-training throughput figure.","check":{"question":"Only 10% of application time is removable networking overhead. What limits ideal overall speedup?","options":["The remaining 90% still takes time","Offload guarantees tenfold speedup","Network bandwidth becomes infinite"],"answer":0,"explanation":"Removing all of one small fraction leaves the unaffected fraction as the runtime floor."},"science":[{"title":"Offload speedup bound","equation":"S = 1 / ((1−p) + p/s)","explanation":"p is the original time fraction accelerated and s its speedup; S is dimensionless overall speedup.","assumptions":"Fixed workload, additive serial times and no new offload overhead or bottleneck migration."}],"prerequisites":["network-adapter","host-cpu"],"learningObjective":"Identify the specific infrastructure work and ownership boundary of a DPU.","workstream":"W07","visualFamily":"architecture-array","engineeringExample":"Synthetic Amdahl example: accelerating 10% of runtime without limit gives at most 1/0.9 ≈ 1.11× overall speedup. This does not account for additional transfers or reduced tail jitter.","evidenceNotes":"BlueField modes are documented in a pinned DOCA archive. IPU is locally defined here; unrelated vendor meanings must not be merged into one architecture.","labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}