Ensemble methods are classifiers that have classifiers as input (and often have higher accuracy than regular input classifiers). This is also called “meta-learning” and it only works if the individual classifiers make independent errors

See also: emergent behaviour


Improves training error of classifiers with high

Models that use the boosting ensemble method:

  1. XGBoost (regularized regression trees)


Improves approximation error of classifiers with high

Models that uses the averaging ensemble method:

  1. Random Forest


  1. Voting: take the mode of the predictions across the classifiers
  2. Stacking: fit another classifier that uses the predictions as features