Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Arrow up icon
GO TO TOP
Fast Data Processing with Spark 2

You're reading from   Fast Data Processing with Spark 2 Accelerate your data for rapid insight

Arrow left icon
Product type Paperback
Published in Oct 2016
Publisher Packt
ISBN-13 9781785889271
Length 274 pages
Edition 3rd Edition
Languages
Arrow right icon
Authors (2):
Arrow left icon
Krishna Sankar Krishna Sankar
Author Profile Icon Krishna Sankar
Krishna Sankar
 Karau Karau
Author Profile Icon Karau
Karau
Arrow right icon
View More author details
Toc

Table of Contents (18) Chapters Close

Fast Data Processing with Spark 2 Third Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
1. Installing Spark and Setting Up Your Cluster FREE CHAPTER 2. Using the Spark Shell 3. Building and Running a Spark Application 4. Creating a SparkSession Object 5. Loading and Saving Data in Spark 6. Manipulating Your RDD 7. Spark 2.0 Concepts 8. Spark SQL 9. Foundations of Datasets/DataFrames – The Proverbial Workhorse for DataScientists 10. Spark with Big Data 11. Machine Learning with Spark ML Pipelines 12. GraphX

GraphX - computational model


Before we dive into creating a graph and applying algorithms, we need to understand the computational model. Needless to say, GraphX has a rich yet simple computational model:

  1. It consists of vertices connected by edges.

  2. It is a property graph, which means the vertices and edges can have arbitrary objects as properties, and most importantly, these properties are visible to the APIs.

It is also a directed multigraph, meaning the edges have a direction and there can be any number of edges between the vertices. This is important to note down, because some of the algorithms can be tricky when faced with loops and cyclic graphs. GraphX has APIs that will come in handy, for example, the removeSelfEdges method will be helpful when you want to remove loops.

With this model, many kinds of graphs can be created, including bipartite and tripartite graphs (for example, the Users-Tags-Web pages). The following diagram illustrates the computational model of GraphX:

A vertex consists...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at £13.99/month. Cancel anytime
Visually different images