Diffusion model
last edited
Diffusion models create images by iteratively denoising random noise. During training the model watches images get progressively corrupted with noise and learns to reverse each step. At generation time it starts from pure noise and, guided by a prompt, removes noise over a few dozen steps until an image emerges.
Architecture
This denoise-loop architecture underpins most diffusion-based image generators — Stable Diffusion, FLUX, Midjourney — and its diffusion-based video successors. Most production models are latent diffusion models. They denoise in a compressed latent space rather than pixel space, with a VAE translating between the two, which is what makes them fast enough to run on consumer GPUs.
Components
The text encoder turns the prompt into embeddings the denoiser can condition on, the sampler schedules the steps, and CFG sets how literally the prompt is followed.
Nearly every technique in the artist's toolkit — img2img, inpainting, ControlNet, LoRA fine-tuning — is a way of steering or modifying this same denoising process. See core techniques for how they combine in practice.
Sources
Foundational research on diffusion models is documented in the denoising diffusion probabilistic models paper. Latent diffusion research describes moving the denoising process into a compressed representation for image synthesis.