Overfitting and underfitting

last edited

Overfitting and underfitting are opposing failures in model training. Diagnosis compares training loss, held-out loss, dataset quality, captions, learning rate, and duration. Each pattern requires a different correction, and recurring failure signatures feed the pattern recognition agenda in training research.

Overfitting and underfitting

Overfitting occurs when a model learns training data too closely, including noise. Training performance improves while results on unseen data weaken. Held-out loss may reach a minimum and then rise as training loss keeps falling.

Other signals include a widening loss gap and images that reproduce dataset details instead of the intended subject. Underfitting is the reverse case. Training and held-out losses stay high, and target features remain weak across several checkpoints. Captions should contain the trigger word and the intended concept.

Training balance

The bias-variance tradeoff distinguishes 2 error patterns. Bias reflects an overly simple representation and can produce underfitting. Variance reflects sensitivity to training-data changes and commonly appears as overfitting.

Unstable convergence can resemble either condition. Review captions, learning rate, step count, and dataset quality together. Loss alone is not a sufficient signal.

Recent tooling adds diagnostics. Release v0.11.1 of sd-scripts (June 2026) logs timestep sampling settings and adds a timestep-distribution visualization. Release v0.12.1 (October 2026) passes the real step count and warmup steps to custom schedulers.

Flux adapter case

Issue 1492 is a 2024 report about a Flux adapter. A 140-image run showed a loss near 0.38. A 12-image run reached about 0.08. Dataset size alone does not ensure convergence.

Release v0.12.3 adds `--timestep_sampling truncated_logit_normal` for FLUX.1. It shipped on October 26. Release v0.11.0 (June 2026) was a major internal refactor, so older tutorial paths may not match current scripts.

Related pages: Training Tools, Training Tool Comparisons, and the Training Pipeline. Further diagnostics appear in Advanced Training Techniques 2026 — Organized Reference.

Sources