Nvidia no longer supports NVLink on consumer GPUs, with the RTX 3090 being the last model to feature it. NVLink bridges are currently expensive, costing between $200-$400. This article investigates the benefits of NVLink for AI workloads, specifically model inference and training, using two RTX 3090 GPUs.

Tests revealed that NVLink significantly boosted prompt processing speed by about 30% during tensor parallel inference and offered a nearly 3x improvement for FSDP training. However, other tested workloads showed no performance difference. A key finding is that Nvidia artificially restricts Peer-to-Peer (P2P) communication on consumer GPUs via drivers, forcing a fallback to slower Shared Host Memory (SHM). Enabling P2P through a third-party driver patch significantly closes the performance gap between NVLink and PCIe.

NVLink proves most beneficial on consumer platforms where multiple GPUs are forced to share PCIe bandwidth (e.g., x8/x8 or lower). For users with server-grade hardware offering full PCIe x16 bandwidth per GPU, or those willing to use the P2P driver patch, NVLink may not offer substantial advantages. It is most impactful for FSDP training and, to a lesser extent, tensor-parallel prompt processing.