Adversarial machine learning algorithms

Before studying adversarial machine learning, let's explore two important terminologies: overfitting and underfitting.

Overfitting and underfitting

Overfitting is one of the biggest obstacles that machine learning practitioners face. Knowing how to spot overfitting is a required skill for building robust machine learning models, because achieving 99% accuracy is not the end of the story. In machine learning, we make predictions. By definition, the fit is how well we approximate the target function. As we saw in the first chapter, the aim of supervised learning is to map the function between the input data and the targets. Thus, a good fit is a good approximation of that function.

Overfitting happens when a model learns the details and noise in the training data, to the extent that it negatively impacts the performance of the model. In other words, noise is picked up and learned by the model, so it can no longer generalize well when it is fed new data...