HeadlinesBriefing favicon HeadlinesBriefing.com

Diffusion Bench launches holistic diffusion transformer benchmark

Hacker News •
×

GitHub hosts the new Diffusion Bench repository, a unified codebase aimed at holistic evaluation of generative diffusion transformers. The project bundles training and testing pipelines for image‑net and text‑to‑image tasks behind a single interface, letting researchers swap configs with one path change. Samples shown are 256×256 images generated after 200K training iterations. The initiative answers community calls for broader metrics beyond ImageNet.

The repo supplies scripts for data preparation, stage‑1 tokenizer training, and stage‑2 diffusion model training across VAE, RAE, REG and pixel families. Config directories mirror each other, so evaluation configs automatically load the corresponding checkpoint and guidance. Built‑in metrics cover FID, IS, PSNR, SSIM, LPIPS and newer benchmarks like DPGBench and Gen AIBench, enabling consistent comparison across methods.

Contributors can launch experiments with the uv project manager, link runs to Weights & Biases, and reproduce published results by following the provided stage‑1 and stage‑2 commands. The code marks several external projects—RAE, RAEv2, REPA—as dependencies, and the maintainers invite pull requests to expand axes, metrics, and model families. The benchmark now offers a reproducible baseline for diffusion research and standardized reporting for future studies.