Inpainting

last edited

Inpainting regenerates only a masked region of an image while the rest stays fixed. It is the editing counterpart of img2img, and the diffusers library documents it on a page separate from image-to-image. The artist supplies a mask and a prompt describing what should appear inside it.

Mechanism

In the diffusers convention, white pixels mark the area to inpaint and black pixels mark the area to keep. The blur_factor option softens the mask edge so the patch blends with its surroundings.

The RePaint paper describes a diffusion approach that alters only the reverse diffusion iterations. It samples the unmasked regions using the given image information. Known content is re-injected at each step, so lighting, perspective, and style stay coherent across the seam.

Models and tooling

Dedicated inpainting checkpoints are finetuned to fill holes given context. The diffusers guide describes its base example as a latent diffusion model finetuned on 512x512 images. The same guide also lists SDXL pipelines and a second-generation pipeline from another model family, linked below in the diffusers documentation.

Moebius is a 0.2B inpainting model accepted to ECCV 2026. It released code and weights on 18 June 2026.

In late July, the beta main_2026 branch of Inpaint Anything moved to SAM 3 for segmentation. It also added a video inpainting model for clips. Text-guided fill uses SDXL or optional FLUX.1-Fill. SD 2 is no longer downloadable there.

Practice

Inpainting is where finished work gets made: fixing broken hands, swapping an object, changing an expression, or cleaning up artifacts. Chained passes, such as face, then hands, then background, each with its own prompt, turn a raw generation into a polished piece. Outpainting applies the same operation to blank space beyond the canvas.

ControlNet can pin structure inside the mask, and a fixed seed makes each fix reproducible. See core techniques.

Sources