Fast Python: High performance techniques for large datasets

Master Python techniques and libraries to reduce run times, efficiently handle huge datasets, and optimize execution for complex machine learning applications.

Fast Python is a toolbox of techniques for high performance Python including:

• Writing efficient pure-Python code
• Optimizing the NumPy and pandas libraries
• Rewriting critical code in Cython
• Designing persistent data structures
• Tailoring code for different architectures
• Implementing Python GPU computing

Fast Python is your guide to optimizing every part of your Python-based data analysis process, from the pure Python code you write to managing the resources of modern hardware and GPUs. You'll learn to rewrite inefficient data structures, improve underperforming code with multithreading, and simplify your datasets without sacrificing accuracy.

Written for experienced practitioners, this book dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working.

About the Technology

Face it. Slow code will kill a big data project. Fast pure-Python code, optimized libraries, and fully utilized multiprocessor hardware are the price of entry for machine learning and large-scale data analysis. What you need are reliable solutions that respond faster to computing requirements while using less resources, and saving money.

About the Book

Fast Python is a toolbox of techniques for speeding up Python, with an emphasis on big data applications. Following the clear examples and precisely articulated details, you’ll learn how to use common libraries like NumPy and pandas in more performant ways and transform data for efficient storage and I/O. More importantly, Fast Python takes a holistic approach to performance, so you’ll see how to optimize the whole system, from code to architecture.

What’s Inside

• Rewriting critical code in Cython
• Designing persistent data structures
• Tailoring code for different architectures
• Implementing Python GPU computing

About the Reader

For intermediate Python programmers familiar with the basics of concurrency.

About the Author

Tiago Antão is one of the co-authors of Biopython, a major bioinformatics package written in Python.

Table of Contents:

PART 1 - FOUNDATIONAL APPROACHES
1 An urgent need for efficiency in data processing
2 Extracting maximum performance from built-in features
3 Concurrency, parallelism, and asynchronous processing
4 High-performance NumPy
PART 2 - HARDWARE
5 Re-implementing critical code with Cython
6 Memory hierarchy, storage, and networking
PART 3 - APPLICATIONS AND LIBRARIES FOR MODERN DATA PROCESSING
7 High-performance pandas and Apache Arrow
8 Storing big data
PART 4 - ADVANCED TOPICS
9 Data analysis using GPU computing
10 Analyzing big data with Dask

Tiago Antao works in the field of genetics, analyzing very large datasets and implementing complex algorithms to process the data. He leverages Python with all its libraries to do scientific computing and data engineering tasks. He is one of the co-authors of Biopython, a major bioinformatics package written in Python. He holds a BE in informatics and a PhD in bioinformatics.