arXiv 2610.09221
Consistent Distribution Matching for Data-Free Diffusion Distillation
AI12 upvotes
Authors: Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
Overview
The authors propose Consistent Distribution Matching, a simulation-free and data-free method that distills diffusion and flow models with a single student network that handles both sample generation and score estimation. The framework needs only a frozen teacher and a trainable student, optimized with one objective. They prove that minimizing this objective yields Wasserstein convergence of the student's flow-map pushforwards to the teacher's marginals.
Introduction
Flow and diffusion models produce strong samples but are slow at inference because they rely on computationally expensive numerical integration. Existing distillation methods often require curated datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate training and scaling.
Results
On ImageNet 256×256, the method reaches an FID of 2.04 with one function evaluation (1-NFE), surpassing state-of-the-art distillation baselines that do not use data. With 4-NFE, it reaches an FID of 1.37 within 40 epochs of training.
Conclusion
This shows that diffusion and flow models can be accelerated to one or few steps without any training data or auxiliary networks while keeping strong generative quality, which simplifies scaling.