The machine learning briefcase

We just created nice, clean, pickle files with preprocessed images to train and test our classifier. However, we've ended up with 20 pickle files. There are two problems with this. First, we have too many files to keep track of easily. Secondly, we've only completed part of our pipeline, where we've processed our image sets but have not prepared a TensorFlow consumable file.

Now we will need to create our three major sets—the training set, the validation set, and the test set. The training set will be used to nudge our classifier, while the validation set will be used to gauge progress on each iteration. The test set will be kept secret until the end of the training, at which point, it will be used to test how well we've trained the model.

The code to do all this is long, so we'll leave you to review the Git repository. Pay close attention to the following three functions:

 def randomize(dataset, labels): 
    permutation = np.random.permutation(labels.shape[0...