Chapter 8.  Analyzing Unstructured Data

In this Big Data era, the proliferation of unstructured data is overwhelming. Numerous methods such as data mining, Natural Language Processing (NLP), information retrieval, and so on, exist for analyzing unstructured data. Due to the rapid growth of unstructured data in all kinds of businesses, scalable solutions have become the need of the hour. Apache Spark is equipped with out of the box algorithms for text analytics, and it also supports custom development of algorithms that are not available by default.

In the previous chapter we have shown how SparkR, an R API to Spark for R programmers can harness the power of Spark, without learning a new language .  In this chapter, we are going to step into a whole new dimension and explore algorithms and techniques to extract information out of unstructured data by leveraging Spark.

As a prerequisite for this chapter, a basic understanding of programming in Python or Scala and an overall understanding...