Pro Spark Streaming: The Zen of Real-Time Analytics Using Apache Spark

Learn the right cutting-edge skills and knowledge to leverage Spark Streaming to implement a wide array of real-time, streaming applications. This book walks you through end-to-end real-time application development using real-world applications, data, and code. Taking an application-first approach, each chapter introduces use cases from a specific industry and uses publicly available datasets from that domain to unravel the intricacies of production-grade design and implementation. The domains covered in Pro Spark Streaming include social media, the sharing economy, finance, online advertising, telecommunication, and IoT.

What You'll Learn

• Discover Spark Streaming application development and best practices
• Work with the low-level details of discretized streams
• Optimize production-grade deployments of Spark Streaming via configuration recipes and instrumentation using Graphite, collectd, and Nagios
• Ingest data from disparate sources including MQTT, Flume, Kafka, Twitter, and a custom HTTP receiver
• Integrate and couple with HBase, Cassandra, and Redis
• Take advantage of design patterns for side-effects and maintaining state across the Spark Streaming micro-batch model
• Implement real-time and scalable ETL using data frames, SparkSQL, Hive, and SparkR
• Use streaming machine learning, predictive analytics, and recommendations
• Mesh batch processing with stream processing via the Lambda architecture