Foundation models struggle with continual learning—the ability to acquire new skills without forgetting old ones. Traditional approaches either require explicit reward functions for reinforcement learning or rely on supervised fine-tuning that causes catastrophic forgetting. Researchers have been seeking methods that enable models to learn sequentially from demonstrations while preserving prior knowledge.

The new Self-Distillation Fine-Tuning (SDFT) method addresses this by using the model itself as a teacher. Instead of relying on external rewards, SDFT leverages in-context learning where a demonstration-conditioned model generates its own training signals. This creates an on-policy learning framework directly from demonstrations, avoiding the off-policy limitations of standard supervised fine-tuning.

Experimental results show SDFT consistently outperforms traditional supervised fine-tuning across skill learning and knowledge acquisition tasks. The approach achieves higher accuracy on new tasks while substantially reducing catastrophic forgetting. In sequential learning experiments, a single model successfully accumulated multiple skills over time without performance regression.

These findings establish on-policy distillation as a practical pathway toward continual learning from demonstrations, potentially enabling more efficient and stable AI systems that can adapt continuously without retraining from scratch.