Data Leakage

ELI5 - The Vibe Check

Data leakage is when your model accidentally gets to peek at the answer key before the test. Maybe a feature you included is basically a disguised version of the thing you're predicting, or some future information snuck into your training data. The model looks brilliant during testing, then falls apart in production because the real world doesn't hand out answer keys.

Real Talk

Data leakage occurs when information from outside the training dataset, often data that wouldn't be available at prediction time, inadvertently influences the model, causing inflated performance metrics that don't hold up in production. Common causes include target leakage, where a feature is derived from the label, temporal leakage, where future data leaks into past training examples, and preprocessing before splitting data into train and test sets.

When You'll Hear This

"The model's 99% accuracy was too good to be true, turned out there was data leakage from a feature that included the outcome." / "Always split before you preprocess, or you'll leak test set statistics into training."