Big data processing and analytics at speed and scale using command line tools.
Key Features
- Perform string processing, numerical computations, and more using CLI tools
- Understand the essential components of data science development workflow
- Automate data pipeline scripts and visualization with the command line
Book Description
The Command Line has been in existence on UNIX-based OSes in the form of Bash shell for over 3 decades. However, very little is known to developers as to how command-line tools can be OSEMN (pronounced as awesome and standing for Obtaining, Scrubbing, Exploring, Modeling, and iNterpreting data) for carrying out simple-to-advanced data science tasks at speed.
This book will start with the requisite concepts and installation steps for carrying out data science tasks using the command line. You will learn to create a data pipeline to solve the problem of working with small-to medium-sized files on a single machine. You will understand the power of the command line, learn how to edit files using a text-based and an. You will not only learn how to automate jobs and scripts, but also learn how to visualize data using the command line.
By the end of this book, you will learn how to speed up the process and perform automated tasks using command-line tools.
What you will learn
- Understand how to set up the command line for data science
- Use AWK programming language commands to search quickly in large datasets.
- Work with files and APIs using the command line
- Share and collect data with CLI tools
- Perform visualization with commands and functions
- Uncover machine-level programming practices with a modern approach to data science
Who this book is for
This book is for data scientists and data analysts with little to no knowledge of the command line but has an understanding of data science. Perform everyday data science tasks using the power of command line tools.
Table of Contents
- Data Science at the Command line and Setting it up
- Essential Commands
- Obtaining and Working with Data,Detached Processing and Terminal Multiplexers
- Bash Functions and Data Visualization
- Loops, Functions and String Processing
- The Command Line as a Database, Math in Bash, and Bringing It All Together
Jason Morris is a systems and research engineer with over 18 years of experience in system architecture, research engineering, and large data analysis. His primary focus is machine learning with TensorFlow, CUDA, and Apache Spark.
Jason is also a speaker and a consultant for designing large-scale architectures, implementing best security practices on the cloud, creating near real-time image detection analytics with deep learning, and developing serverless architectures to aid in ETL. His most recent roles include solution architect, big data engineer, big data specialist, and instructor at Amazon Web Services. He is currently the Chief Technology Officer of Next Rev Technologies and his favorite command line program is netcat.
Chris McCubbin is a data scientist and software developer with 20 years experience in developing complex systems and analytics. He co-founded the successful big data security startup Sqrrl, since acquired by Amazon. He has also developed smart swarming systems for drones, social network analysis systems in MapReduce and big data security analytic platforms using the Apache projects Accumulo and Spark. He has been using the Unix command line starting on IRIX platforms in college and his favorite command line program is find.
