An overview of topic modeling
In Chapter 10, Probabilistic Graphical Models, we saw how we can use a bag of words as a feature of a Naïve Bayes model in order to perform sentiment analysis. There, the specific predictive task involved determining whether a particular movie review was expressing a positive sentiment or a negative sentiment. We explicitly assumed that the movie review was exclusively expressing only one possible sentiment. Each of the words used as features (such as bad, good, fun, and so on) had a different likelihood of appearing in a review under each sentiment.
To compute the model's decision, we basically computed the likelihood of all the words in a particular review under one class, and compared this to the likelihood of all the words having been generated by the other class. We adjusted these likelihoods using the prior probability of each class, so that, when we know that one class is more popular in the training data, we expect to find it more frequently represented...
