GraphX

As shown in the preceding section, we can model many real-life use cases as Graphs with a set of vertices and a set of edges linking the vertices. We also wrote simple code trying to implement some basic graph operations and queries such as, Is X a friend of Y ? However, as we explored further, the only get more complicated along with use cases and also the size of graphs is much much larger than can be handled on one machine.

Note

It is not possible to fit one billion Facebook users along with all their friendship relations into one machine or even a few machines.

What we need to do is to look beyond the one machine and few machines thrown together and rather start considering highly scalable architectures to implement the complex graph algorithms, which can handle the volume of data and complex interconnections of the data elements. We have already seen an introduction to Spark, how Spark solves some challenges of distributed computing and big data analytics. We also saw real-time...