Every major inference framework shipped prefill-decode disaggregation this year. NVIDIA built it into Dynamo. SGLang made it the default for large-scale deployments. v LLM added a KV connector API to support it natively.

The consensus is forming fast: split your prefill and decode onto separate GPU pools, and throughput improves. The consensus is wrong for most teams. Doubleword's analysis shows that a balanced disaggregated deployment matches colocated throughput, but at small GPU counts the rounding losses dominate: you can't allocate fractional GPUs, so the specialization gains get eaten by incomplete worker utilization.

The practical benefit at that scale is independent SLO tuning, not throughput. A June 2025 study evaluating hundreds of thousands of design points found that disaggregation is most effective for prefill-heavy traffic patterns and larger models. For the mixed-traffic workloads most teams actually run, queueing and inter-node KV cache transfer dominated end-to-end latency.

The teams that disaggregated moved the bottleneck. They didn't remove it. I ran into this on an inference workload serving a mid-size classification model.

TPOT spiked under bursty traffic, and the first instinct was to separate prefill from decode. Instead, I enabled chunked prefill on the same GPU pool. TPOT stabilized.

The problem was scheduling interference, and chunked prefill handled it without adding a network hop. Chunked prefill breaks long prefill requests into smaller chunks and interleaves them with decode batches on the same GPU. No separate node pools.

No KV cache transfer over the network. No P: D ratio to tune. TNG Technology Consulting measured a 50 percent increase in total token throughput using standard v LLM with chunked prefill enabled.

The decode batches still ran between prefill chunks on the same hardware, but the scheduling interference dropped to a level most production workloads can tolerate. For workloads below roughly 50 requests per second with moderate prompt lengths, that bound is tight enough.