Our Qwen3-TTS 1.7B Custom Voice implementation achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) on a single NVIDIA H100 SXM, maintaining real-time playback. Under Poisson open-loop traffic, we surpass v LLM-Omni, SGLang-Omni, Vox Serve, and M*.

We define real-time TTS as low audible TTFA, zero underruns, scalable capacity, and intelligible output. Benchmarks run five minutes using Fireworks AI's LLM methodology. Tuning each engine involved removing leading silence via dynamic RMS trimming (~80ms improvement) and optimizing frame accumulation.

Performance after tuning shows Vox Serve reaches sub-50 ms p95 TTFA at 1 RPS, while others hit ~100 ms by 6 RPS. Our approach coordinates Qwen3-TTS's three modules—Talker, Code Predictor, and Codec—under one scheduler, enabling urgent request prioritization and efficient batching.

Key optimizations include capturing the Code Predictor's fixed 15-step loop as a CUDA graph with Triton attention kernels, and implementing a state-cache-based Codec that avoids reprocessing full history. At $4.29/hour, we deliver ~$2 per 1M characters—significantly cheaper than Eleven Labs V3 ($100) and Cartesia Sonic 3.5 ($49). Implementation and benchmarks are open-sourced.