Experiment Tracking
ELI5 - The Vibe Check
Experiment tracking is refusing to let wait, which version of the model was this, and what settings did I use ever be a real question again. Every training run automatically logs its code version, its settings, its metrics, and its resulting model, so six weeks later you can actually answer which of your forty attempts was the good one, and reproduce it.
Real Talk
Experiment tracking is the practice of systematically logging the parameters, code version, dataset version, metrics, and artifacts of every model training run, typically using tools like MLflow or Weights and Biases, so runs can be compared, reproduced, and audited later. Without it, teams rapidly lose track of which combination of hyperparameters and data actually produced their best-performing model.
When You'll Hear This
"Log this run to the experiment tracker before you change the learning rate, or we'll lose track of what actually worked." / "We compared forty experiment tracking runs and the winning model just used a lower learning rate and more trees."
Related Terms
Hyperparameter
Hyperparameters are the settings you configure BEFORE training starts — as opposed to parameters (weights) which the model learns ON ITS OWN.
Machine Learning (ML)
Machine Learning is teaching a computer by showing it thousands of examples instead of writing out every rule.
Model Registry
A model registry is the version control system for trained models instead of code.
Training
Training is the long, expensive process where an AI learns from data.