The Diffusion Gemma Team introduces Diffusion Gemma, an experimental open-weight language model leveraging discrete diffusion for rapid text generation. Unlike traditional autoregressive models, it refines blocks of 256 tokens in parallel, bypassing sequential decoding bottlenecks.

Fine-tuned from the Gemma 4 mixture-of-experts model (3.8B activated, 25.2B total parameters), Diffusion Gemma uses a two-stage training pipeline requiring fewer than 10% of the original model's token budget. Stage one employs supervised fine-tuning for bidirectional denoising, while stage two combines reinforcement learning with sampler distillation.

Evaluations show Diffusion Gemma generates ~20 tokens per forward pass and reaches 1,500 output tokens per second on a single NVIDIA H100 GPU, outperforming autoregressive models even with speculative decoding. It preserves Gemma's capabilities including thinking mode, multimodal inputs, and long contexts, while maintaining AR generation compatibility. This establishes a new Pareto frontier for speed versus capability tradeoffs.