Generating text using RNNs

We used Long Short-Term Memory (LSTMs) and Gated Recurrent Units (GRUs) in previous chapters for text classification. In addition to being used for predictive tasks, RNNs can be used to create generative models, as well. RNNs can learn long-term dependencies from an input text, and can therefore generate completely new sequences. This generative model can be either character or word-based. In the next section, we will look at a simple word-based text generation model.

Generating Linux kernel code with a GRU

We will now look at a simple, fun example, to generate Linux kernel code using an RNN. The complete Jupyter Notebook for this example is available in the book's code repository, under Chapter08. For the training data, we will first extract the kernel code from the Linux source. You can download the latest (or an earlier) version of the Linux kernel from the kernel archives at https://www.kernel.org/.

We will extract the tar file and use only the core kernel under...