Machine Learning Section 006

Ensembles and Gradient Boosting

Many weak models beat one strong model on tabular data. This section is why, and how to tune the ones that win competitions.

10 of 10 lessons published Three reading levels on every lesson

Start with “Why ensembles work”

Lessons in order

Work top to bottom. Each lesson assumes the one above it.

  1. Why ensembles work
  2. Bagging
  3. Out-of-bag evaluation
  4. AdaBoost
  5. LightGBM
  6. CatBoost
  7. Tuning gradient-boosted trees
  8. Monotonic constraints
  9. Feature importance done right
  10. Stacking