Decision Tree
ELI5 - The Vibe Check
A decision tree is a model that learns to play twenty questions with your data. Is the income over $50k? Yes. Is the age over 30? No. Then predict: approved. It keeps splitting the data on the question that separates the outcomes best, until it's confident enough to make a call, and you can literally draw the whole thing as a flowchart.
Real Talk
A decision tree recursively splits a dataset on the feature and threshold that best separates the target variable at each node, forming a tree of if-then rules that can be used for both classification and regression. Its main appeal is interpretability, you can trace exactly why it made a prediction, but individual trees overfit easily, which is why they're usually combined into ensembles like random forests.
When You'll Hear This
"Show the decision tree to the compliance team, they need to see exactly why the model rejected the application." / "A single decision tree overfits like crazy, that's why we moved to a random forest."
Related Terms
Classification
Classification is teaching an AI to sort things into categories. Is this email spam or not? Is this image a cat, dog, or bird?
Gradient Boosting
Gradient boosting builds a model the way you'd improve a rough first draft: you write it, look specifically at what it got wrong, then write a second pass...
Overfitting
Overfitting is when your model gets TOO good at the training data and becomes useless on new data.
Random Forest
A random forest is what you get when you refuse to trust a single decision tree's opinion and instead grow hundreds of slightly different trees, each train...