Generative adversarial networks

Generative adversarial networks (GAN) are another very recent type of generative model that got attention due to their impressive results. A GAN is composed of two networks together: a generator network and a discriminator network. During training, they both play a zero-sum game, where the discriminator network tries to discover whether the images input to it are real or fake. At the same time, the generator network tries to create fake images that are good enough to fool the discriminator.

The idea is that after some time of training, both the discriminator and the generator become very good at their tasks. As a result, the generator is forced to try and create images that look closer and closer to the original dataset. To be able to do this, it must capture the probability distribution of the dataset.

The following diagram gives an overview of how this GAN model looks:

Both discriminator and generator will have their own loss function, but both of their losses...