Model Collapse
ELI5 — The Vibe Check
When AI trains on AI-generated data and slowly goes insane. Each generation gets slightly worse, like a photocopy of a photocopy of a photocopy. Eventually the model forgets rare but important things and produces bland, repetitive garbage. It's the AI equivalent of inbreeding.
Real Talk
Model collapse is a degenerative phenomenon where models trained on AI-generated (synthetic) data progressively lose the tails of the original data distribution over successive generations. This results in reduced diversity, amplified biases, and degraded output quality. It's a growing concern as AI-generated content proliferates online.
When You'll Hear This
"If we keep training on AI slop, model collapse is inevitable." / "Model collapse is why you need real human data in the mix."
Related Terms
Fine-tuning
Fine-tuning is like taking a smart graduate student who knows everything and then sending them to a specialist bootcamp.
Hallucination
When an AI confidently makes something up — like citing a library that doesn't exist or generating code that calls a function that was never written.
Synthetic Data
Synthetic data is fake data that's good enough to train real models.
Training
Training is the long, expensive process where an AI learns from data.