Architect and implement an end-to-end data streaming pipeline

$9.99 ~~$137.99~~

By Anghel Leonard

Time 7hrs 51mins

Published in Nov 2018

Product Type Video

Edition 1st Edition

What do you get with a video?

Download this video in MP4 format

DRM FREE - Read whenever, wherever and however you want

Key benefits

Comprehensive coverage of Apache Spark, Kafka, and Spring Boot integration

Hands-on demonstrations and practical examples

Expert insights and best practices

Real-world applications and use cases

Description

Today, organizations have a difficult time working with huge numbers of datasets. In addition, data processing and analyzing need to be done in real time to gain insights. This is where data streaming comes in. As big data is no longer a niche topic, having the skillset to architect and develop robust data streaming pipelines is a must for all developers. In addition, they also need to think of the entire pipeline, including the trade-offs for every tier. This course starts by explaining the blueprint architecture for developing a completely functional data streaming pipeline and installing the technologies used. With the help of live coding sessions, you will get hands-on with architecting every tier of the pipeline. You will also handle specific issues encountered working with streaming data. You will input a live data stream of Meetup RSVPs that will be analyzed and displayed via Google Maps. By the end of the course, you will have built an efficient data streaming pipeline and will be able to analyze its various tiers, ensuring a continuous flow of data. Please note: The link meetup.com is not functional anymore. Here are a few alternatives for free WebSocket streaming endpoints : Finnhub offers a free WebSocket endpoint for streaming real-time stock market data. https://finnhub.io/docs/api/websocket-trades EOD Historical Data also offers a free WebSocket endpoint for streaming real-time stock market data. https://github.com/EodHistoricalData PieSocket offers a free tier of WebSockets that allows you to connect up to 100 clients. https://www.piesocket.com/blog/tag/realtime Socket.IO also offers a free tier of WebSockets that allows you to connect up to 100 clients. https://socket.io/ They can be used by adapting the course code. The principles are the same. All the code and supporting files for this course are available at https://github.com/PacktPublishing/-Data-Stream-Development-with-Apache-Spark-Kafka-and-Spring-Boot

What you will learn

Analyze real-time data streams

Deploy collection and message queuing tiers

Establish a robust data access tier with MongoDB

Implement powerful analysis tiers using Apache Spark

Ensure fault tolerance and mitigate data loss

Frequently bought together