Audio Visual Person Tracking: A Practical Approach

This book deals with the creation of the algorithmic backbone that enables a computer to perceive humans in a monitored space. This is performed using the same signals that humans process, i.e., audio and video. Computers reproduce the same type of perception using sensors and algorithms in order to detect and track multiple interacting humans, by way of multiple cues, like bodies, faces or speech. This application domain is challenging, because audio and visual signals are cluttered by both background and foreground objects. First, particle filtering is established as the framework for tracking. Then, audio, visual and also audio-visual tracking systems are separately explained. Each modality is analyzed, starting with sensor configuration, detection for tracker initialization and the trackers themselves. Techniques to fuse the modalities are then considered. Instead of offering a monolithic approach to the tracking problem, this book also focuses on implementation by providing MATLAB code for every presented component. This way, the reader can connect every concept with corresponding code. Finally, the applications of the various tracking systems in different domains are studied.

This book deals with the creation of the algorithm backbone that can enable a computer to perceive humans in some monitored space by processing the same signals that humans use in order to perform the same task, i.e., audio and video. To do so, computers use sensors and algorithms to detect and track multiple interacting humans, their faces and hands or their voices. This application domain is challenging, because audio and visual signals are cluttered both by background and foreground audiovisual objects. After establishing particle filtering as the framework for tracking, audio, visual and audiovisual tracking is then detailed. Each modality is analyzed starting with sensor configurations, detection for tracker initialization and the trackers themselves. Techniques to fuse the modalities are then considered. Instead of offering a monolithic approach to the tracking problem, the book also focuses on implementation issues by providing MATLAB code for every presented component. This way, the reader connects every presented concept with a corresponding piece of code that follows immediately after the theory. Finally, the applications of the tracking systems in different application domains are studied and considered.