Multi-modal Robotic Intelligence: An Active Perception Approach

Recently, substantial progress has been made in the machine perception, particularly computer vision, largely due to the advancements in deep learning techniques. However, robots often operate in unstructured environments, which differ greatly from the well-defined problems typically addressed in computer vision. Consequently, many existing computer vision solutions are not directly applicable to robotics. Additionally, modern intelligent robots have access to multi-modal sensory information, rather than relying on a single modality. Therefore, it is essential to explore the specific challenges of multi-modal robotic intelligence.

A key requirement for all intelligent robots is the capability of active perception. Active perception is an effective approach to bridge the gap between robotics and machine perception. Intelligence emerges when the robot actively interacts with the environment, embodying a close coupling of perception and action within a continuous feedback loop. Perception guided action, and each movement generate information that informs subsequent movements. With a variety of available multi-modal sensory information, it is crucial for robots to leverage active perception techniques to achieve multi-modal robotic intelligence.

This book introduces multi-modal robotic intelligence from the perspective of active perception. Extensive robotic multi-modal active perception problems are formulated and corresponding case studies are described. Specifically, this book is organized in three parts. Part I covers core concepts and approaches for multi-modal robotic intelligence. In Part II, active perception for robotic intelligence is described, which presents several typical active perception tasks. Part III further describes the multi-modal active perception for robotic intelligence. The book is primarily intended for researchers and graduates with a foundational knowledge of machine learning, spanning in a wide range of disciplines, particularly those involved in robotic intelligence and sensor fusion.

Di Guo is a professor of Beijing University of Posts and Telecommunications. She received her PhD degree in the Department of Computer Science and Technology, Tsinghua University, Beijing, China in 2017. Her main research interests include robotic manipulation and embodied perception. She is the associate editor of International Conference on Robotics and Automation and IEEE/RSJ International Conference on Intelligent Robots and Systems. She serves as the area chair of Robotics: Science and Systems. 

Huaping Liu received his PhD degree from Tsinghua University, Beijing, China, in 2004. He is currently a professor in the Department of Computer Science and Technology at Tsinghua University. His research interests include robot perception and learning. Dr. Liu received the National Science Fund for Distinguished Young Scholars and served as the area chair for Robotics Science and Systems multiple times. He is a senior editor of the International Journal of Robotics Research. Dr. Liu published three books with Springer, including the “Robotic Tactile Perception and Understanding” (published in 2018).