Extracting embeddings from artificial neural networks

Neural networks are the new gold standard of models for machine learning. Thanks to this structure, impressive progress has been made, from image analysis to speech recognition, and computers are now able to perform increasingly complex tasks. One surprising application of neural networks is their ability to model complex objects, such as images, text, or audio records, with fewer dimensions, while still preserving some aspects of the original dataset (shapes in the image, frequencies in the audio, and so on). In this section, following a quick general review of neural networks, we will focus on one architecture called skip-gram, which was first used in the context of word embedding but can be extended to graphs as well.