Reinforcement learning with verifiable rewards, known as zero RL, is effective for chain-of-thought reasoning but has been limited to small models due to computational constraints.

Researchers present Ring-Zero, a stable and efficient training pipeline that addresses issues like poor readability and redundant tokens often seen in naive scaling. This pipeline incorporates optimizations such as clipped importance sampling and mixed-precision control.

Experiments with Ring-2.5-1T-Zero reveal that scaling to 1 trillion parameters significantly boosts sample efficiency and performance. The training process involves a discovery phase followed by a sharpening phase. The model spontaneously develops advanced cognitive behaviors, including anthropomorphism, self-verification, and parallel reasoning, making hand-crafted heuristics unnecessary. Evaluated on seven mathematical benchmarks, the model shows competitive performance and excels in producing structured and concise reasoning traces, assessed through a new framework evaluating comprehensibility, reproducibility, and efficiency.