The intuition of hyperparameter tuning

In order to gain a practical intuition of the need for hyperparameter tuning, let's go through the following scenario in predicting the accuracy of a given neural network architecture on the MNIST dataset:

• Scenario 1: High number of epochs and low learning rate
• Scenario 2: Low number of epochs and high learning rate

Let us create the train and test datasets in a Google Cloud environment, as follows:

1. Download the dataset:

mkdir data
curl -O https://s3.amazonaws.com/img-datasets/mnist.pkl.gz
gzip -d mnist.pkl.gz
mv mnist.pkl data/ 

The preceding code creates a new folder named data, downloads the MNIST dataset, and moves it into the data folder.

2. Open Python in Terminal and import the required packages:

from __future__ import print_function 
import tensorflow as tf
import pickle # for handling the new data source
import numpy as np
from datetime import datetime # for filename conventions
from tensorflow.python.lib.io import file_io # for better file I/O
import...