Autoencoder

last edited

An autoencoder is a neural network trained to copy its input through a narrow bottleneck. The encoder compresses data into a short code, and the decoder rebuilds the input from that code. Many diffusion models use an autoencoder as their VAE, though recent work shows the best design is still unsettled.

Origins and variants

A 2006 paper by Hinton and Salakhutdinov trained deep autoencoders as a nonlinear alternative to principal component analysis. Denoising autoencoders followed in 2008 and learned to restore corrupted inputs.

The variational autoencoder of 2013 turned the code into a probability distribution, so new data can be sampled. The VQ-VAE of 2017 swapped continuous codes for entries from a learned codebook. Masked autoencoders hide 75 percent of image patches and train vision models to fill the gaps.

Role in image generation

Latent diffusion runs the denoising loop inside an autoencoder's compressed latent space instead of on pixels. Its autoencoder compresses images by 8x spatially.

Training mixes a pixel reconstruction loss with perceptual and adversarial terms. The compression level sets the cost of every denoising step. Small text, distant faces and fine texture lose detail on the round trip, so upscaling passes often follow.

Recent directions

DC-AE raised the spatial compression ratio up to 128 while keeping reconstruction quality. Representation autoencoders pair a frozen pretrained vision encoder with a trained decoder for diffusion transformers.

Qwen-Image-VAE-2.0 was submitted in May 2026. It offers f16 and f32 models. Channels go up to 192. The paper adds a benchmark for text-rich images.

Prior-Aligned Autoencoders appeared in May 2026. This tokenizer shapes the latent manifold for latent diffusion. VFM-VAE uses a frozen vision foundation model as the tokenizer.

Sources