Hyperparameter Tuning for Machine and Deep Learning with R: A Practical Guide

This open access book provides a wealth of hands-on examples that illustrate how hyperparameter tuning can be applied in practice and gives deep insights into the working mechanisms of machine learning (ML) and deep learning (DL) methods. The aim of the book is to equip readers with the ability to achieve better results with significantly less time, costs, effort and resources using the methods described here. The case studies presented in this book can be run on a regular desktop or notebook computer. No high-performance computing facilities are required.

The idea for the book originated in a study conducted by Bartz & Bartz GmbH for the Federal Statistical Office of Germany (Destatis). Building on that study, the book is addressed to practitioners in industry as well as researchers, teachers and students in academia. The content focuses on the hyperparameter tuning of ML and DL algorithms, and is divided into two main parts: theory (Part I) and application (Part II).Essential topics covered include: a survey of important model parameters; four parameter tuning studies and one extensive global parameter tuning study; statistical analysis of the performance of ML and DL methods based on severity; and a new, consensus-ranking-based way to aggregate and analyze results from multiple algorithms. The book presents analyses of more than 30 hyperparameters from six relevant ML and DL methods, and provides source code so that users can reproduce the results. Accordingly, it serves as a handbook and textbook alike.

Prof. Dr. Thomas Bartz-Beielstein is an artificial intelligence expert with 30+ years of experience. He is a professor of applied mathematics at TH Köln in Germany and the director of the Institute for Data Science, Engineering, and Analytics (IDE+A). His research lies in artificial intelligence, machine learning, simulation, and optimization. Hedeveloped the Sequential Parameter Optimization (SPO). SPO integrates approaches from surrogate model-based optimization and evolutionary computing. He has worked on diverse topics from applied mathematics and statistics, design of experiments, simulation-based optimization and applications in domains as water industry, elevator control, or mechanical engineering.

Prof. Dr. Martin Zaefferer is a professor at Duale Hochschule Baden-Württemberg Ravensburg, teaching subjects related to data science in business informatics. Previously, he worked as a consultant at Bartz & Bartz GmbH and as a researcher at TH Köln, where he also studied electrical engineering and automation. He received a PhD from the Department of Computer Science at TU Dortmund University. Subsequently, he developed a keen interest in researching methods from the intersection of optimization and machine learning algorithms. He is passionate about the analysis of complex processes and finding novel solutions to challenging real-world problems.

Prof. Dr. Olaf Mersmann is a professor of data science at TH Köln-University of Applied Sciences in Germany and a member of the Institute for Data Science, Engineering, and Analytics (IDE+A). Having studied physics, statistics and data science, his research interests include landscape analysis for black box optimization problems and industrial machine learning applications. He is one of the developers of the exploratory landscape analysis approach to characterize continuous function landscapes.