{"id":"pcie-interface","level":"server","name":"PCI Express interconnect","shortName":"PCIe","role":"Connects host processors, accelerators, storage, and network adapters through standardized serial links.","description":"PCIe is a general-purpose component interconnect built from serial lanes. A link can use several lanes in parallel, and switches can connect multiple devices into a hierarchy. Transactions carry commands and data; software uses drivers and device capabilities to arrange transfers. The generation and lane count must both be stated when comparing bandwidth. A faster endpoint may operate at a lower rate when the host, switch, or board path supports less. Transmit and receive paths contribute to different directional rates; packet overhead also separates a link rating from one-way application throughput. PCIe is one part of the AI system’s connectivity, alongside specialized coherent links, GPU scale-up fabrics, and the network between servers.","specs":[{"label":"PCIe 5.0 signaling","value":"32 GT/s per lane","context":"Raw transfer rate; payload bandwidth is lower."},{"label":"PCIe 6.0 signaling","value":"64 GT/s per lane","context":"Uses PAM4, FLIT encoding, and forward error correction."},{"label":"PCIe 6.0 ×16","value":"256 GB/s raw bidirectional","context":"128 GB/s each direction before protocol overhead; not a GB200 board-wide guarantee."}],"interfaces":[{"name":"Host transactions","direction":"in","signal":"Memory reads/writes, commands, and completions"},{"name":"Device transfers","direction":"out","signal":"DMA data and completion traffic"},{"name":"Link training","direction":"in","signal":"Negotiated width, speed, and signal equalization"}],"tradeoff":"Higher speeds increase transfer capacity but tighten electrical and board-design requirements. Real throughput also depends on packet overhead, topology, transfer size, and device behavior.","whyAI":"NICs and NVMe drives use PCIe to deliver data into the compute system. Congestion on these paths can starve accelerators even when the GPUs themselves are fast.","related":["host-cpu","network-adapter","nvme-storage","compute-baseboard"],"sources":[{"title":"PCI-SIG: PCIe 5.0 signaling rates","url":"https://pcisig.com/what-bit-rates-does-pcie-50-specification-support-and-how-does-it-compare-prior-pcie-generations"},{"title":"PCI-SIG: PCIe 6.0 FAQ","url":"https://pcisig.com/faq?field_category_value%5B%5D=pci_express_6.0&keys=PAM4"},{"title":"Intel: PCI Express Gen4/Gen5 architecture and encoding","url":"https://edc.intel.com/content/www/us/en/design/products/platforms/details/raptor-lake-s/13th-generation-core-processors-datasheet-volume-1-of-2/004/pci-express-architecture/"}],"scale":"Centimeter board traces and connectors; logical lane pairs enlarged","kind":"serial-lanes-and-packets","mechanism":["The endpoints train a link and agree on a supported speed and lane width.","Transactions are packetized and serialized across separate transmit and receive paths.","The receiving device handles requests or data and sends completions as required by the protocol."],"misconception":"GT/s, GB/s, and bidirectional aggregate bandwidth are different measures. A raw line rate must account for lane count, direction, and protocol overhead before comparison with application throughput.","check":{"question":"What happens if a fast PCIe device is connected through a slower supported host path?","options":["The link must always use the device’s maximum advertised generation.","It operates at a mutually supported speed and width.","It automatically becomes an NVLink interface."],"answer":1,"explanation":"PCIe capability is negotiated across the actual link. The endpoint, host, switches, and board support determine what is usable."},"branch":"Connectivity","subbranch":"Board component links","science":[{"title":"Encoding and protocol reduce raw lane bandwidth","equation":"BGen5,one-way = lanes × 32×10⁹ × (128/130) / 8","explanation":"For a Gen5 link, each lane runs at 32 GT/s with 128b/130b encoding. Sixteen lanes yield about 63.0 GB/s per direction after this encoding overhead, before transaction and link-protocol costs. Full-duplex operation can carry traffic both ways, but summing the directions does not make a single transfer twice as fast.","assumptions":"PCIe Gen5 at negotiated full lane width, using NRZ and 128b/130b encoding. Excludes packet headers, flow control, acknowledgments, and implementation limits. PCIe Gen6 uses a different signaling and FLIT model and must be calculated separately."}],"history":[],"learningObjective":"Keep lane rate, encoding and byte rate in one boundary.","engineeringExample":"A Gen5 x16 one-way payload estimate multiplies lanes, raw rate and 128/130 encoding efficiency, then divides bits by eight.","evidenceNotes":"Protocol packets, flow control and platform limits reduce delivered throughput.","visualFamily":"signal-power","prerequisites":["interconnect","host-cpu"],"labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}