Python Machine Learning By Example: The easiest way to get into machine learning

The 20 newsgroup dataset is composed of text, taken from news articles as its name implies. It was originally collected by Ken Lang, and is now widely used for experiments in text applications of machine learning techniques, specifically natural language processing techniques.

Natural language processing (NLP) is a significant subfield of machine learning, which deals with the interactions between machine (computer) and human (natural) languages. Natural languages are not limited to speech and conversation. They can be in writing and sign languages as well. The data for NLP tasks can be in different forms, for example, text from social media posts, web pages, even medical prescription, audio from voice mail, commands to control systems, even a favorite music or movie. Nowadays, NLP has been broadly involved in our daily lives: we can not live without machine translation; weather forecast scripts are automatically generated; we find voice search convenient; we get the answer to...

After a short list of real-world applications of NLP, we will be touring the essential stack of Python NLP libraries in this chapter. These packages handle a wide range of NLP tasks as mentioned above as well as others such as sentiment analysis, text classification, named entity recognition, and many more.

The most famous NLP libraries in Python include Natural Language Toolkit (NLTK), Gensim and TextBlob. The scikit-learn library also has NLP related features. NLTK (http://www.nltk.org/) was originally developed for education purposes and is now being widely used in industries as well. There is a saying that you can't talk about NLP without mentioning NLTK. It is the most famous and leading platform for building Python-based NLP applications. We can install it simply by running the sudo pip install -U nltk command in Terminal.

NLTK comes with over 50 collections of large and well-structured text datasets, which are called corpora in NLP. Corpora can...

The first project in this book is about the 20 newsgroups dataset found in scikit-learn. The data contains approximately 20,000 across 20 online newsgroups. A newsgroup is a place on the Internet where you can ask and answer questions about a certain topic. The data is already split into training and test sets. The cutoff point is at a certain date. The original data comes from http://qwone.com/~jason/20Newsgroups/. 20 different newsgroups are listed as follows:

All the documents in the dataset are in English. And from the newsgroup names, you can deduce the topics.

Some of the newsgroups are closely related or even overlapping, for instance, those five computer...

It is possible to download the data manually from the original website or many online repositories. However, there are also many versions of the dataset--some are cleaned in a certain way and some in the raw form. To avoid confusion, it is best to use a consistent acquisition method. The scikit-learn library provides a utility function of loading the dataset.Once the dataset is downloaded, it is automatically cached. We won’t need to download the same dataset twice. In most cases, caching the dataset, especially for a relatively small one, is considered a good practice. Other Python libraries also support download utilities, but not all of them implement automatic caching. This is another reason why we love scikit-learn.

To load the data, we can import the loader function for the 20 newsgroups data as follows:

>>> from sklearn.datasets import fetch_20newsgroups

Then we can download the dataset with the default parameters:

>>> groups = fetch_20newsgroups()
Downloading...

After we download the 20 newsgroups by whatever means we prefer, the data object called groups is now available in the program. The data object is in the form of key-value dictionary. Its keys are as follows:

>>> groups.keys()
dict_keys(['description', 'target_names', 'target', 'filenames',
  'DESCR', 'data'])

The target_names key gives the newsgroups names:

>>> groups['target_names']
['alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware', 'comp.sys.mac.hardware', 'comp.windows.x', 'misc.forsale', 'rec.autos', 'rec.motorcycles', 'rec.sport.baseball', 'rec.sport.hockey', 'sci.crypt', 'sci.electronics', 'sci.med', 'sci.space', 'soc.religion.christian', 'talk.politics.guns', 'talk.politics.mideast', 'talk.politics.misc', 'talk.religion.misc']

The target key corresponds to a newsgroup but is encoded as an integer:

>>> groups.target
array([7, 4, 4, ..., 3, 1, 8])

Then what are the distinct values for these integers...

It's good to visualize to get a general idea of how the data is structured, what possible issues may arise, and if there are any irregularities that we have to take care of.

In the context of multiple topics or categories, it is important to know what the distribution of topics is. A uniform class distribution is the easiest to deal with because there are no under-represented or over-represented categories. However, we frequently have a skewed distribution with one or more categories dominating. We herein use the seaborn package (https://seaborn.pydata.org/) to compute the histogram of categories and plot it utilizing the matplotlib package (https://matplotlib.org/). We can install both packages via pip. Now let’s display the distribution of the classes as follows:

>>> import seaborn as sns
>>> sns.distplot(groups.target)
<matplotlib.axes._subplots.AxesSubplot object at 0x108ada6a0>
>>> import matplotlib.pyplot as plt
>>> plt.show()

Refer...

We see items, which are obviously not words, such as 00 and 000. Maybe we should ignore items that contain only digits. However, 0d and 0t are also not words. We also see items as __, so maybe we should only allow items that consist only of letters. The posts contain names such as andrew as well. We can filter names with the Names corpus from NLTK we just worked with. Of course, with every filtering we apply, we have to make sure that we don't lose information. Finally, we see words that are very similar, such as try and trying, and word and words.

We have two basic strategies to deal words from the same root--stemming and lemmatization. Stemming is the more quick and dirty type approach. It involves chopping, if necessary, off letters, for example, 'words' becomes 'word' after stemming. The result of stemming doesn't have to be a valid word. Lemmatizing, on the other hand, is slower but more accurate. Lemmatizing performs a dictionary lookup and guarantees to return a...

Clustering divides a dataset into clusters. This is an unsupervised learning task since we typically don't have any labels. In the most realistic cases, the complexity is so high that we are not able to find the best division in clusters; however, we can usually find a decent approximation. The clustering analysis task requires a distance function, which indicates how close items are to each other. A common distance is Euclidean distance, which is the distance as a bird flies. Another common distance is taxicab distance, which measures distance in city blocks. Clustering was first used in the 1930s by social science researchers without modern computers.

Clustering can be hard or soft. In hard clustering, an item belongs to only to a cluster, while in soft clustering, an item can belong to multiple clusters with varying probabilities. In this book, I have used only the hard clustering method.

We can also have items that do not belong to any cluster. These items are considered outliers...

Topics in natural language processing don't exactly match the dictionary definition and correspond to more of a nebulous statistical concept. We speak of topic models and probability distributions of words linked to topics, as we know them. When we read a text, we expect certain words appearing in the title or the body of the text to capture the semantic context of the document. An article about Python programming will have words such as class and function, while a story about snakes will have words such as eggs and afraid. Documents usually have multiple topics, for instance, this recipe is about topic models and non-negative matrix factorization, which we will discuss shortly. We can, therefore, define an additive model for topics by assigning different weights to topics.

One of the topic modeling algorithms is non-negative matrix factorization (NMF). This algorithm factorizes a matrix into a product of two smaller matrices in such a way that the three matrices have no negative...

In this chapter, we acquired the fundamental concepts of NLP as an important subfield in machine learning, including tokenization, stemming and lemmatization, POS tagging. We also explored three powerful NLP packages and realized some common tasks using NLTK. Then we continued with the main project newsgroups topic modeling. We started with extracting features with tokenization techniques as well as stemming and lemmatization. We then went through clustering and implementations of k-means clustering and non-negative matrix factorization for topic modeling. We gained hands-on experience in working with text data and tackling topic modeling problems in an unsupervised learning manner. We briefly mentioned the corpora resources available in NLTK. It would be a great idea to apply what we've learned on some of the corpora. What topics can you extract from the Shakespeare corpus?