jzhao.xyz

Search

Search IconIcon to open search

Ensemble method

Last updated Sep 23, 2022 Edit Source

Ensemble methods are classifiers that have classifiers as input (and often have higher accuracy than regular input classifiers). This is also called “meta-learning” and it only works if the individual classifiers make independent errors

# Boosting

Improves training error of classifiers with high $E_{train}$

Models that use the boosting ensemble method:

  1. XGBoost (regularized regression trees)

# Averaging

Improves approximation error of classifiers with high $E_{approx}$

Models that uses the averaging ensemble method:

  1. Random Forest

# Methods

  1. Voting: take the mode of the predictions across the classifiers
  2. Stacking: fit another classifier that uses the predictions as features