Train-Test Split

ELI5 - The Vibe Check

A train-test split is holding back a chunk of your data that the model never gets to see while learning, specifically so you have something honest to grade it on afterward. Training a model and then testing it on the same data it already memorized is like grading a student on the exact quiz they used to study, of course they'll ace it.

Real Talk

A train-test split divides a dataset into a portion used to fit the model, the training set, and a portion held out entirely until evaluation, the test set, so performance metrics reflect how the model generalizes to unseen data rather than how well it memorized the training data. A three-way split adds a validation set, used for tuning hyperparameters, so the test set stays untouched until the very end.

Show Me The Code

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

When You'll Hear This

"You evaluated on the training data? That number means nothing, do a proper train-test split." / "We use an 80/20 train-test split, plus a separate validation set for tuning."