Handbook of Multimodal-multisensor Interfaces : Language Processing, Software, Commercialization, and Emerging Directions

The Handbook of Multimodal-Multisensor Interfaces provides the first authoritative resource on what has become the dominant paradigm for new computer interfaces-user input involving new media (speech, multi-touch, hand and body gestures, facial expressions, writing) embedded in multimodal-multisensor interfaces.

This three-volume handbook is written by international experts and pioneers in the field. It provides a textbook, reference, and technology roadmap for professionals working in this and related areas.

This third volume focuses on state-of-the-art multimodal language and dialogue processing, including semantic integration of modalities. The development of increasingly expressive embodied agents and robots has become an active test bed for coordinating multimodal dialogue input and output, including processing of language and nonverbal communication. In addition, major application areas are featured for commercializing multimodal-multisensor systems, including automotive, robotic, manufacturing, machine translation, banking, communications, and others. These systems rely heavily on software tools, data resources, and international standards to facilitate their development. For insights into the future, emerging multimodal-multisensor technology trends are highlighted in medicine, robotics, interaction with smart spaces, and similar areas. Finally, this volume discusses the societal impact of more widespread adoption of these systems, such as privacy risks and how to mitigate them. The handbook chapters provide a number of walk-through examples of system design and processing, information on practical resources for developing and evaluating new systems, and terminology and tutorial support for mastering this emerging field. In the final section of this volume, experts exchange views on a timely and controversial challenge topic, and how they believe multimodal-multisensor interfaces need to be equipped to most effectively advance human performance during the next decade.

Sharon Oviatt (Incaa Designs) is internationally known for her multidisciplinary work on multimodal and mobile interfaces, human-centered interfaces, educational interfaces and learning analytics. She has been recipient of the inaugural ACM-ICMI Sustained Accomplishment Award, National Science Foundation Special Creativity Award, and ACM-SIGCHI CHI Academy award. She has published over 160 scientific articles in a wide range of venues, and is an Associate Editor of the main journals and edited book collections in the field of human-centered interfaces. Her other books include The Design of Future Educational Interfaces (2013, Routledge) and The Paradigm Shift to Multimodality in Contemporary Computer Interfaces (2015, Morgan Claypool).

Björn Schuller (University of Passau and Imperial College London) is currently Chair of Complex and Intelligent Systems at University of Passau and Reader in Machine Learning at Imperial College. He is best known for his work on multisensorial/multimodal intelligent signal processing for affective, behavioral, and human-centered computing. In 2015 and 2016, he was honored by the World Economic Forum as one of 40/50 extraordinary scientists under age 40. His further awards include the CHiME, MediaEval, and MIREX competitions. He has published over 500 peer-reviewed scientific contributions across a range of disciplines and venues, and is Editor-in-Chief of IEEE Transactions on Affective Computing. His books include Intelligent Audio Analysis (2013, Springer) and Computational Paralinguistics (2013, Wiley).

Philip Cohen (VoiceBox Technologies) is Senior Vice President and Chief Scientist for Artificial Intelligence, whose research interests include multimodal interaction, human-computer dialogue, and multi-agent systems. He is a Fellow of the American Association for Artificial Intelligence, past President of the Association for Computational Linguistics, and recipient (with Hector Levesque) of an Inaugural Influential Paper Award by the International Foundation for Autonomous Agents and Multi-Agent Systems. He was the Founder of Adapx, Professor and Co-Director of the Center for Human-Computer Communication in Computer Science at Oregon Health and Science University, and Director of Natural Language in the Artificial Intelligence Center at SRI International.

Daniel Sonntag (German Research Center for Artificial Intelligence, DFKI) is a Principal Researcher and Research Fellow. His research interests include multimodal and mobile AI-based interfaces, common-sense modeling, and semantic machine learning methods for cognitive computing and improved usability. He has published over 120 scientific articles, and has been recipient of the German High Tech Champion Award in 2011 and the AAAI Recognition and IAAI Deployed Application Award in 2013. He is an editorial board member of the German Journal on Artificial Intelligence (KI). Currently, he leads both national and European projects from the Federal Ministry of Education and Research, the Federal Ministry for Economic Affairs and Energy, and Horizon 2020.

Gerasimos Potamianos (University of Thessaly) is Associate Professor and Director of Graduate Studies in Electrical and Computer Engineering. His research spans multisensory and multimodal speech processing and scene analysis, with applications to human-computer interaction and ambient intelligence. He has authored over 120 articles, and has seven patents. He holds a Diploma Degree from the National Technical University of Athens, and a M.Sc. and Ph.D. from the Johns Hopkins University, all in Electrical and Computer Engineering. In addition to his academic experience, he has worked at AT&T Research Labs, IBM T.J. Watson Research Center (US), and at the FORTH and NCSR 'Demokritos' Research Centers in Greece.