Mastering Large Datasets with Python : Parallelize and Distribute Your Python Code

John T Wolohan

ISBN 10: 1617296236 / ISBN 13: 9781617296239

Published by Manning Publications Jan 2020, 2020

About this Item

Neuware - With an emphasis on clarity, style, and performance, author J.T. Wolohan expertly guides you through implementing a functionally-influenced approach to Python coding. You'll get familiar with Python's functional built-ins like the functools operator and itertools modules, as well as the tools library.Mastering Large Datasets teaches you to write easily readable, easily scalable Python code that can efficiently process large volumes of structured and unstructured data. By the end of this comprehensive guide, you'll have a solid grasp on the tools and methods that will take your code beyond the laptop and your data science career to the next level!Key features- An introduction to functional and parallel programming- Data science workflow- Profiling code for better performance- Fulfilling different quality objectives for a single unifying task- Python multiprocessing- Practical exercises including full-scale distributed applicationsAudienceReaders should have intermediate Python programming skills.About the technologyPython is a data scientist's dream-come-true, thanks to readily available libraries that support tasks like data analysis, machine learning, visualization, and numerical computing.

Seller Inventory # 9781617296239

Bibliographic Details

• Title: Mastering Large Datasets with Python : Parallelize and Distribute Your Python Code
• Publisher: Manning Publications Jan 2020
• Publication Date: 2020
• Language: English
• ISBN 10: 1617296236
• ISBN 13: 9781617296239
• Binding: Taschenbuch
• Condition: Neu

About this title

Synopsis

Summary

Modern data science solutions need to be clean, easy to read, and scalable. In Mastering Large Datasets with Python, author J.T. Wolohan teaches you how to take a small project and scale it up using a functionally influenced approach to Python coding. You'll explore methods and built-in Python tools that lend themselves to clarity and scalability, like the high-performing parallelism method, as well as distributed technologies that allow for high data throughput. The abundant hands-on exercises in this practical tutorial will lock in these essential skills for any large-scale data science project.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology

Programming techniques that work well on laptop-sized data can slow to a crawl--or fail altogether--when applied to massive files or distributed datasets. By mastering the powerful map and reduce paradigm, along with the Python-based tools that support it, you can write data-centric applications that scale efficiently without requiring codebase rewrites as your requirements change.

About the book

Mastering Large Datasets with Python teaches you to write code that can handle datasets of any size. You'll start with laptop-sized datasets that teach you to parallelize data analysis by breaking large tasks into smaller ones that can run simultaneously. You'll then scale those same programs to industrial-sized datasets on a cluster of cloud servers. With the map and reduce paradigm firmly in place, you'll explore tools like Hadoop and PySpark to efficiently process massive distributed datasets, speed up decision-making with machine learning, and simplify your data storage with AWS S3. What's inside

• An introduction to the map and reduce paradigm
• Parallelization with the multiprocessing module and pathos framework
• Hadoop and Spark for distributed computing
• Running AWS jobs to process large datasets

About the Author

J.T. Wolohan is a lead data scientist at Booz Allen Hamilton and a PhD researcher at Indiana University, Bloomington, affiliated with the Department of Information and Library Science and the School of Informatics and Computing. His professional work focuses on rapid prototyping and scalable AI. His research focuses on computational analysis of social uses of language online.