Chapter 9. Training at Scale

So far in this book, the datasets we have used or looked at have ranged in size from the tens of thousands (MNIST) of samples to just over a million (ImageNet). Although all these datasets were considered huge when they first came out, and required state-of-the-art machines to use, the great speed at which technologies such as GPUs and cloud computing have advanced has now made them both easy and quick to train by people with relatively low-power machines.

However, some of the amazing power of deep neural networks comes from their ability to scale with the amount of data fed to them. In simple terms, this means that the more good, clean data you can use to train your model, the better the result is going to be. Researchers are aware of this, and we can see that the number of training samples in new public datasets has continued to increase.

As a result of this, it is highly likely that, if you start working on problems in the industry or maybe even just the latest...