Scaling Up with R and Apache Arrow

Analyze large datasets directly from R. Scaling Up With R and Arrow provides a guide to working efficiently with larger-than-memory datasets using the arrow R package. As data grows in size and complexity, traditional data analysis methods in R often hit technical limitations. In this book, you'll learn how to overcome these hurdles without needing to set up complex infrastructure.

You'll learn about the Apache Arrow project's origins, goals, and its significance in bridging the gap between data science and big data ecosystems. You'll also learn how to leverage the arrow R package to work directly with files in various formats, such as CSV and Parquet, using familiar dplyr syntax. This book explores practical topics like data manipulation, file formats, working with larger datasets, and optimizing workflows for data in cloud storage. Advanced chapters examine user-defined functions, integration with other tools like DuckDB, and extending Arrow's capabilities to work with geospatial data.

Written by developers of the Arrow R package, this guide is essential for anyone looking to scale their data processing capabilities in R.

Nic Crane is an R developer, educator, and general enthusiast, with a background in data science and software engineering. Nic is a member of the Apache Arrow Project Management Committee (PMC) and part of the team who maintains the arrow R package.

Jonathan Keane is an engineering manager with a background in software engineering and data science. Jonathan is a part of the team who maintains the Arrow project including the Arrow R package.

Neal Richardson is an engineering leader focused on building software that helps people work with data. He is a member of the Arrow PMC and one of the top contributors to the project.