Recognizing the user's sketch
In this section, we will first review the dataset and model we will use to guess what the user is drawing. We will then proceed to integrate it into the workflow of the user who is sketching, and implement the functionality to support swapping out the user's sketch with a selected image.
Reviewing the training data and model
For this chapter, a CNN was trained on the dataset that was used and made available from the research paper How Do Humans Sketch Objects? by Mathias Eitz, James Hays, and Marc Alexa. The paper, presented at SIGGRAPH in 2012, compares the performance of humans classifying sketches to that of a machine. The dataset consists of 20,000 sketches evenly distributed across 250 object categories, ranging from airplanes to zebras; a few examples are shown here:
From a perceptual study, they found that humans correctly identified the object category (such as snowman, grapes, and many more) of a sketch 73% of the time. The competitor, their ML model,...
