Speculative decoding converts underused CPU compute into faster token generation without changing model output. In vLLM tests, DFlash delivered 3.92x autoregressive throughput with Qwen3.5-9B on Intel Xeon 6 at concurrency 1. This represents a 74% cost reduction per generated token.

DFlash speculative decoding support for CPUs was enabled in vLLM v0.25.0. Testing used an r8i AWS instance with Intel Xeon 6 processors featuring Performance-cores. The technique works by having a lightweight draft model propose several future tokens, which the larger target model verifies in a single pass.

Speculative decoding is lossless acceleration since rejection sampling recovers the target distribution. It's especially relevant for Xeon inference at small batch sizes, where decode spends time moving model weights for minimal work per weight. Verification transforms per-token matrix-vector operations leveraging Intel AVX-512 into matrix-matrix operations accelerated by Intel AMX.

DFlash, developed by Z Lab, predicts blocks in one pass using a small block-diffusion drafter instead of generating draft tokens serially. It injects hidden features from the target model into the draft model's KV cache, improving draft quality.