[{"data":1,"prerenderedAt":77},["ShallowReactive",2],{"term-g\u002Fgradient-boosting":3,"related-g\u002Fgradient-boosting":60},{"id":4,"title":5,"acronym":6,"body":7,"category":40,"description":41,"difficulty":42,"extension":43,"letter":44,"meta":45,"navigation":46,"path":47,"related":48,"seo":54,"sitemap":55,"stem":58,"subcategory":6,"__hash__":59},"terms\u002Fterms\u002Fg\u002Fgradient-boosting.md","Gradient Boosting",null,{"type":8,"value":9,"toc":33},"minimark",[10,15,19,23,26,30],[11,12,14],"h2",{"id":13},"eli5-the-vibe-check","ELI5 - The Vibe Check",[16,17,18],"p",{},"Gradient boosting builds a model the way you'd improve a rough first draft: you write it, look specifically at what it got wrong, then write a second pass that focuses on fixing exactly those mistakes, then a third pass fixing what's still wrong, and so on. Each new tiny model is trained specifically to correct the errors of everything that came before it, and the whole chain added together becomes surprisingly accurate.",[11,20,22],{"id":21},"real-talk","Real Talk",[16,24,25],{},"Gradient boosting builds an ensemble of weak learners, typically shallow decision trees, sequentially, where each new tree is trained to predict the residual errors of the combined ensemble so far, using gradient descent to minimize a loss function. Unlike random forests, which build trees independently and average them, boosting builds trees dependently, each one correcting the last, which tends to produce higher accuracy but is more prone to overfitting without careful tuning.",[11,27,29],{"id":28},"when-youll-hear-this","When You'll Hear This",[16,31,32],{},"\"Gradient boosting squeezed out another 3 points of accuracy over the random forest, but it took way more tuning to get there.\" \u002F \"Watch the validation loss closely with gradient boosting, it'll happily overfit if you let it run too many rounds.\"",{"title":34,"searchDepth":35,"depth":35,"links":36},"",2,[37,38,39],{"id":13,"depth":35,"text":14},{"id":21,"depth":35,"text":22},{"id":28,"depth":35,"text":29},"ai","Gradient boosting builds a model the way you'd improve a rough first draft: you write it, look specifically at what it got wrong, then write a second pass...","advanced","md","g",{},true,"\u002Fterms\u002Fg\u002Fgradient-boosting",[49,50,51,52,53],"Random Forest","XGBoost","Gradient Descent","Overfitting","Loss Function",{"title":5,"description":41},{"changefreq":56,"priority":57},"weekly",0.7,"terms\u002Fg\u002Fgradient-boosting","8foLMt68hSfiuHNxF96CRWq--JzImtujIgNDSElLSAw",[61,64,68,71,74],{"title":51,"path":62,"acronym":6,"category":40,"difficulty":42,"description":63},"\u002Fterms\u002Fg\u002Fgradient-descent","Gradient Descent is how an AI learns — it's the algorithm that nudges the model's weights in the right direction after each mistake.",{"title":53,"path":65,"acronym":6,"category":40,"difficulty":66,"description":67},"\u002Fterms\u002Fl\u002Floss-function","intermediate","The loss function is the AI's score of how badly it's doing.",{"title":52,"path":69,"acronym":6,"category":40,"difficulty":66,"description":70},"\u002Fterms\u002Fo\u002Foverfitting","Overfitting is when your model gets TOO good at the training data and becomes useless on new data.",{"title":49,"path":72,"acronym":6,"category":40,"difficulty":66,"description":73},"\u002Fterms\u002Fr\u002Frandom-forest","A random forest is what you get when you refuse to trust a single decision tree's opinion and instead grow hundreds of slightly different trees, each train...",{"title":50,"path":75,"acronym":6,"category":40,"difficulty":42,"description":76},"\u002Fterms\u002Fx\u002Fxgboost","XGBoost is the specific, ruthlessly optimized software library that turned gradient boosting from a solid idea into the thing that wins Kaggle competitions...",1792528088957]