Text encoder
last edited
A text encoder is the language model that turns a prompt into embeddings the image model can condition on. It sits between the text and the diffusion model. Its training data and architecture set the limits of what the conditioning signal can carry.
Text encoders
Stable Diffusion 1.5 and 2.0 relied on a single CLIP-L encoder. CLIP was trained on image-caption pairs, as described in the original CLIP paper from 2021. That pretraining ties the encoder to short captions. Stable Diffusion 3 moved to three encoders, FLUX to two, and a newer model to four.
T5-XXL comes from the T5 text-to-text architecture. It serves as a language-model option in those designs.
Newer systems use decoder language or vision-language models. A compact FLUX 3 image model reads its text encoder at eight of its 36 layers, not only the last layer. The FLUX 3 Action Base model card lists an unmodified instruct vision-language model as its shared text encoder. The FLUX 3 family was announced in July 2026.
Customization
An embedding (textual inversion) is a learned pseudo-word in the encoder's input space. It extends the vocabulary with a concept the encoder lacked.
Related methods that bypass text appear in core techniques. Stable Diffusion's history is covered in Stable Diffusion.
Sources
- CLIP paper: Learning Transferable Visual Models From Natural Language Supervision covers contrastive image-caption pretraining.
- T5 paper: Exploring the Limits of Transfer Learning presents the T5 text-to-text architecture.
- FLUX 3 Hugging Face blog post on Diffusers support describes the encoder layers.
- FLUX 3 Action Base model card names the shared text encoder.