Qwen3-30B-Instruct achieves a 55.3% pass@1 score on LiveCodeBench v6 using simple self-distillation (SSD), up from 42.4%. This method samples model outputs with specific temperature settings, then fine-tunes without external verifiers or reinforcement learning. The technique works across Llama 4B, 8B, and 30B models, including both instruct and thinking variants, suggesting broad applicability.

SSD addresses a precision-exploration conflict in LLM decoding. By reshaping token distributions contextually, it suppresses low-precision distractor tails while maintaining diversity in useful solutions. This explains why SSD succeeds on harder problems and generalizes across model sizes.

The approach offers a complementary post-training strategy for code generation improvements. Unlike traditional methods requiring human feedback or teacher models, SSD uses raw model outputs, reducing dependency on external systems. This could lower costs and complexity for developers.

Why does this matter? As code generation becomes critical for software development, methods like SSD enable models to self-improve efficiently. With minimal engineering overhead, SSD could become a standard step in LLM training pipelines for coding tasks.