图像和视频模型在过去几年有了显著提升，尽管这些模型的内部结构自Stable Diffusion 3以来几乎未变。当然，GPT-Image流行的自回归扩散等小型变体除外。但从高层来看，基本上全是带有v-prediction目标的Transformer骨干网络的流匹配。

在我们的经验中，大部分收益直接归因于3种数据改进：数据过滤与重平衡、数据标注和合成数据生成。强化学习已带来一些改进，但仅在过去几个月才真正开始在图像和视频领域起作用。

couple of years ago, the prevailing wisdom across all generative models was to aggregate as much data as humanly possible for pre-training. Luckily, the field has gotten a lot smarter about this. If you throw a bunch of low-quality data into pre-training, your model will waste capacity learning to mimic that slice of data. If you filter your dataset well, your model will have an easier time learning what you want it to learn.

We walk you through how our approach to data filtering has evolved since 2024, and hopefully save you from a couple of headaches if you end up training your own generative models down the line.