Non-parametric basis. Can think about it as a sum of gaussian ‘bumps’.

Replace with

where

Gaussian RBFs are universal approximators

  • Enough bumps can approximate any continuous function to arbitrary precision.
  • Achieve optimal test error as ‘n’ goes to infinity.