Feature Engineering
ELI5 - The Vibe Check
Feature engineering is the unglamorous, hugely important work of turning raw, messy real-world data into the specific columns a model can actually learn from. A raw timestamp isn't very useful, but is this a weekend or hours since last purchase might be exactly the signal your model needed. Most of the actual skill in classical ML lives here, not in picking the fancier algorithm.
Real Talk
Feature engineering is the process of transforming raw data into input features that better represent the underlying patterns a model needs to learn, including creating new features from existing ones, encoding categorical variables, scaling numeric ranges, and extracting signal from dates, text, or other unstructured fields. It's often the single highest-leverage step in a classical ML pipeline, frequently mattering more to final performance than the choice of algorithm.
When You'll Hear This
"We spent two weeks on feature engineering and it moved the needle way more than switching algorithms did." / "Adding days-since-last-login as a feature was the single biggest jump in accuracy we got."
Related Terms
Data Leakage
Data leakage is when your model accidentally gets to peek at the answer key before the test.
Machine Learning (ML)
Machine Learning is teaching a computer by showing it thousands of examples instead of writing out every rule.
One-Hot Encoding
Models are bad at math with words, so if a column says red, blue, or green, you can't just feed that in directly.
Random Forest
A random forest is what you get when you refuse to trust a single decision tree's opinion and instead grow hundreds of slightly different trees, each train...