Apache Spark 2.x Machine Learning Cookbook

Simplify machine learning model implementations with Spark

About This Book

• Solve the day-to-day problems of data science with Spark
• This unique cookbook consists of exciting and intuitive numerical recipes
• Optimize your work by acquiring, cleaning, analyzing, predicting, and visualizing your data

Who This Book Is For

This book is for Scala developers with a fairly good exposure to and understanding of machine learning techniques, but lack practical implementations with Spark. A solid knowledge of machine learning algorithms is assumed, as well as hands-on experience of implementing ML algorithms with Scala. However, you do not need to be acquainted with the Spark ML libraries and ecosystem.

What You Will Learn

• Get to know how Scala and Spark go hand-in-hand for developers when developing ML systems with Spark
• Build a recommendation engine that scales with Spark
• Find out how to build unsupervised clustering systems to classify data in Spark
• Build machine learning systems with the Decision Tree and Ensemble models in Spark
• Deal with the curse of high-dimensionality in big data using Spark
• Implement Text analytics for Search Engines in Spark
• Streaming Machine Learning System implementation using Spark

In Detail

Machine learning aims to extract knowledge from data, relying on fundamental concepts in computer science, statistics, probability, and optimization. Learning about algorithms enables a wide range of applications, from everyday tasks such as product recommendations and spam filtering to cutting edge applications such as self-driving cars and personalized medicine. You will gain hands-on experience of applying these principles using Apache Spark, a resilient cluster computing system well suited for large-scale machine learning tasks.

This book begins with a quick overview of setting up the necessary IDEs to facilitate the execution of code examples that will be covered in various chapters. It also highlights some key issues developers face while working with machine learning algorithms on the Spark platform. We progress by uncovering the various Spark APIs and the implementation of ML algorithms with developing classification systems, recommendation engines, text analytics, clustering, and learning systems. Toward the final chapters, we ll focus on building high-end applications and explain various unsupervised methodologies and challenges to tackle when implementing with big data ML systems.

Style and approach

This book is packed with intuitive recipes supported with line-by-line explanations to help you understand how to optimize your work flow and resolve problems when working with complex data modeling tasks and predictive algorithms. This is a valuable resource for data scientists and those working on large scale data projects.

Siamak Amirghodsi is a world-class senior technology executive leader with an entrepreneurial track record of overseeing big data strategies, cloud transformation, quantitative risk management, advanced analytics, large-scale regulatory data platforming, enterprise architecture, technology road mapping, multi-project execution, and organizational streamlining in Fortune 20 environments in a global setting.

Siamak is a hands-on big data, cloud, machine learning, and AI expert, and is currently overseeing the large-scale cloud data platforming and advanced risk analytics build out for a tier-1 financial institution in the United States. Siamak's interests include building advanced technical teams, executive management, Spark, Hadoop, big data analytics, AI, deep learning nets, TensorFlow, cognitive models, swarm algorithms, real-time streaming systems, quantum computing, financial risk management, trading signal discovery, econometrics, long-term financial cycles, IoT, blockchain, probabilistic graphical models, cryptography, and NLP.

Meenakshi Rajendran is a hands-on big data analytics and data governance manager with expertise in large-scale data platforming and machine learning program execution on a global scale. She is experienced in the end-to-end delivery of data analytics and data science products for leading financial institutions. Meenakshi holds a master's degree in business administration and is a certified PMP with over 13 years of experience in global software delivery environments. She not only understands the underpinnings of big data and data science technology but also has a solid understanding of the human side of the equation as well.

Meenakshi's favorite languages are Python, R, Julia, and Scala. Her areas of research and interest are Apache Spark, cloud, regulatory data governance, machine learning, Cassandra, and managing global data teams at scale. In her free time, she dabbles in software engineering management literature, cognitive psychology, and chess for relaxation.

Broderick Hall is a hands-on big data analytics expert and holds a master's degree in computer science with 20 years of experience in designing and developing complex enterprise-wide software applications with real-time and regulatory requirements at a global scale. He has an extensive experience in designing and building real-time financial applications for some of the largest financial institutions and exchanges in USA. He is a deep learning early adopter and is currently working on a large-scale cloud-based data platform with deep learning net augmentation.

Shuen Mei is a big data analytic platforms expert with 15+ years of experience in the financial services industry. He is experienced in designing, building, and executing large-scale, enterprise-distributed financial systems with mission-critical low-latency requirements. He is certified in the Apache Spark, Cloudera Big Data platform, including Developer, Admin, and HBase.

Shuen is also a certified AWS solutions architect with emphasis on peta-byte range real-time data platform systems. Shuen is a skilled software engineer with extensive experience in delivering infrastructure, code, data architecture, and performance tuning solutions in trading and finance for Fortune 100 companies.