jzhao.xyz

Search

Search IconIcon to open search

Supervised learning

Last updated Sep 9, 2022 Edit Source

Generally, the most successful machine learning technique (with the exception of games)

Examples:

Tradeoffs:

Decision trees Naive Bayes
# Features used Sequences of rules based on 1 feature All features
Training $O(dn)$, one pass per depth $O(n)$, just counting
New data May need to recompute tree Just update counts
Accuracy Good if simple rules based on individual features work Good if features almost independent given label
Interpretability easy to see how decisions are made easy to see how each feature influences decision

# Notation

# Parametric vs Non-parametric

# General Rules

# Decision Theory

Are we equally concerned about each potential outcome? Usually not! Sometimes, false negatives or false positives have outsized impact – the cost of mistakes might be different

We can look to decision theory to help us here. Denote $cost(\hat y_i, \tilde y_i)$ as the cost of predicting $\hat y_i$ instead of the actual label $\tilde y_i$.

Then, instead of predicting the most probable label, compute all possible actions and take the action with the lowest expected cost: $\mathbb E [cost(\hat y_i, \tilde y_i)]$