Learn the right cutting-edge skills and knowledge to leverage Spark Streaming to implement a wide array of real-time, streaming applications. Pro Spark Streaming walks you through end-to-end real-time application development using real-world applications, data, and code. Taking an application-first approach, each chapter introduces use cases from a specific industry and uses publicly available datasets from that domain to unravel the intricacies of production-grade design and implementation. The domains covered in the book include social media, the sharing economy, finance, online advertising, telecommunication, and IoT.
In the last few years, Spark has become synonymous with big data processing. DStreams enhance the underlying Spark processing engine to support streaming analysis with a novel micro-batch processing model. Pro Spark Streaming by Zubair Nabi will enable you to become a specialist of latency sensitive applications by leveraging the key features of DStreams, micro-batch processing, and functional programming. To this end, the book includes ready-to-deploy examples and actual code. Pro Spark Streaming will act as the bible of Spark Streaming.
What You'll Learn
Spark Streaming application development and best practices
Low-level details of discretized streams
The application and vitality of streaming analytics to a number of industries and domains
Optimization of production-grade deployments of Spark Streaming via configuration recipes and instrumentation using Graphite, collectd, and Nagios
Ingestion of data from disparate sources including MQTT, Flume, Kafka, Twitter, and a custom HTTP receiver
Integration and coupling with HBase, Cassandra, and Redis
Design patterns for side-effects and maintaining state across the Spark Streaming micro-batch model
Real-time and scalable ETL using data frames, SparkSQL, Hive, and SparkR
Streaming machine learning, predictive analytics, and recommendations
Meshing batch processing with stream processing via the Lambda architecture
Who This Book Is For
The audience includes data scientists, big data experts, BI analysts, and data architects.
Table of Contents
Chapter 1: The Hitchhiker’s Guide to Big Data
Chapter 2: Introduction to Spark
Chapter 3: DStreams: Real-Time RDDs
Chapter 4: High-Velocity Streams: Parallelism and Other Stories
Chapter 5: Real-Time Route 66: Linking External Data Sources
Chapter 6: The Art of Side Effects
Chapter 7: Getting Ready for Prime Time
Chapter 8: Real-Time ETL and Analytics Magic
Chapter 9: Machine Learning at Scale
Chapter 10: Of Clouds, Lambdas, and Pythons
