NVLink vs PCIe: match each interconnect to its bandwidth tier in a modern multi-GPU server.

Inside a typical AI server you have two completely different kinds of bandwidth, and senior engineers care about both. **Memory bandwidth** is how fast bytes move between a GPU and its own HBM, measured in TB/s. **Interconnect bandwidth** is how fast bytes move between chips: CPU to GPU, or GPU to GPU. That second category is where NVLink and PCIe show up. **PCIe** is the general-purpose CPU-to-GPU bus that every server has. A modern PCIe Gen5 x16 link delivers about 64 GB/s per direction (so 128 GB/s bidirectional). This is plenty for streaming weights into a GPU once at startup, pulling input batches in, and sending output tokens back to the CPU. It is also how the GPU talks to the rest of the system: storage, network cards, the host operating system. **NVLink** is NVIDIA's purpose-built GPU-to-GPU fabric. On an H100 SXM card, NVLink 4 delivers about 900 GB/s aggregate per card (split across multiple links to neighboring GPUs). On the newer B200, NVLink 5 reaches roughly 1.8 TB/s. That is more than 10x what PCIe Gen5 offers, and it exists for one reason: when you split a large model across multiple GPUs, those GPUs need to exchange activation tensors at the speed of compute, not at the speed of the storage bus.

**PCIe** is the standardized CPU-to-GPU and peripheral bus. Generation 5 at x16 width gives roughly 64 GB/s per direction (~128 GB/s bidirectional). It connects every GPU to the host CPU, system memory, network cards, and storage. Single-GPU inference workloads mostly only need PCIe: weights load once over PCIe at startup, inputs and outputs flow across it per request, and that is the end of the story. **NVLink** is NVIDIA's proprietary GPU-to-GPU interconnect. The generations roughly track GPU generations: NVLink 3 on A100 SXM at ~600 GB/s aggregate, NVLink 4 on H100 SXM at ~900 GB/s, NVLink 5 on B200 at ~1.8 TB/s. NVLink only exists on SXM-form-factor cards; PCIe-form-factor variants of the same GPU silicon either omit NVLink entirely or expose only a limited subset. NVSwitch chips inside DGX and HGX boxes wire all NVLinks into a fully-connected fabric so any GPU can talk to any other at full NVLink speed. The workload that cares most about this distinction is **tensor parallelism**. When a model is too big to fit on one GPU, you split each weight matrix across multiple GPUs. Every layer's forward pass requires an all-reduce of partial results, which is a collective communication pattern with bandwidth proportional to (model dim) x (batch x sequence). On NVLink this all-reduce takes microseconds. On PCIe it takes milliseconds, dominating decode latency and erasing the throughput gain from parallelism. This is why production large-model serving uses NVLink-connected GPUs and not PCIe-bridged ones.