Chapter 12. From the Perceptron to Artificial Neural Networks

In Chapter 10, The Perceptron we introduced the perceptron, a linear model for binary classification. We learned that the perceptron is not a universal function approximator; its decision boundary must be a hyperplane. In Chapter 11, From the Perceptron to Support Vector Machines we introduced the SVM, which addresses some of the perceptron's limitations by using kernels to efficiently map the feature representations to a higher dimensional space in which the classes may be linearly separable. In this chapter, we will discuss ANNs, powerful nonlinear models for supervised and unsupervised tasks that use a different strategy to overcome the perceptron's limitations. If the perceptron is analogous to a neuron, an ANN, or neural net, is analogous to a brain. As billions of neurons with trillions of synapses comprise a human brain, an ANN is a directed graph of artificial neurons. The graph's edges are weighted; these weights are...