Recent years have witnessed important advancements in our understanding of the psychological underpinnings of subjective properties of visual information, such as aesthetics, memorability, or induced emotions. Concurrently, computational models of objective visual properties such as semantic labelling and geometric relationships have made significant breakthroughs using the latest achievements in machine learning and large-scale data collection. There has also been limited but important work exploiting these breakthroughs to improve computational modelling of subjective visual properties. The time is ripe to explore how advances in both of these fields of study can be mutually enriching and lead to further progress.
This book combines perspectives from psychology and machine learning to showcase a new, unified understanding of how images and videos influence high-level visual perception - particularly interestingness, affective values and emotions, aesthetic values, memorability, novelty, complexity, visual composition and stylistic attributes, and creativity. These human-based metrics are interesting for a very broad range of current applications, ranging from content retrieval and search, storytelling, to targeted advertising, education and learning, and content filtering.
Work already exists in the literature that studies the psychological aspects of these notions or investigates potential correlations between two or more of these human concepts. Attempts at building computational models capable of predicting such notions can also be found, using state-of-the-art machine learning techniques. Nevertheless their performance proves that there is still room for improvement, as the tasks are by nature highly challenging and multifaceted, requiring thought on both the psychological implications of the human concepts, as well as their translation to machines.
Bogdan Ionescu is General Manager of the CAMPUS Research Center and Leader of the AI Multimedia Lab at Politehnica University of Bucharest (UPB). He holds a double Ph.D. degree in image/video processing from UPB and University of Savoie, France. He is currently a tenured Professor with ETTI-UPB. His main research interests cover multimedia/video/image processing and analysis, multimedia content-based retrieval and machine learning for multimedia. He has authored over 170 scientific publications. He serves/served as guest co-editor for Image and Vision Computing, and Multimedia Tools and Applications; conference committee chair for various conferences, e.g., General Chair for ACM ICMR 2017, General Chair for CLEF 2021, area chair for ACM Multimedia 2018-2021; lead organizer/co-organizer for several benchmark campaigns, e.g., MediaEval Retrieving Diverse Social Images 2013-2017, Violent Scenes Detection 2013-2014, Affective Impact of Movies 2015, Predicting Media Interestingness 2016-2017, Predicting Media Memorability 2018-2021, ImageCLEF 2017-2021, ChaLearn ICPR Multimedia Information Processing for Personality & Social Networks Analysis Challenge 2018. He contributed to over 22 Romanian/EU funded research and strategic programmes, as principal investigator or as part of the research team.
Wilma A. Bainbridge is an Assistant Professor in the Department of Psychology at the University of Chicago, with appointments in the Neuroscience Institute and the Program in Computational Social Science. She received her Ph.D in Brain and Cognitive Sciences at the Massachusetts Institute of Technology in 2016, after getting her B.A. in Cognitive Science from Yale University in 2010. She completed postdoctoral training at the Laboratory of Brain and Cognition at the National Institute of Mental Health (USA). Her research focuses on the cognitive neuroscience of perception and memory, looking at how certain items are intrinsically more memorable than others, and how the brain is sensitive to this information. In her work, she uses behavioral experiments, computer vision, machine learning, online crowd-sourcing, and functional MRI to understand what makes an item intrinsically memorable, and how the brain processes these items differently.
Naila Murray obtained a BSE in electrical engineering from Princeton University in 2007. In 2012, she received her Ph.D. from the Universitat Autonoma de Barcelona, in affiliation with the Computer Vision Center. She joined Xerox Research Centre Europe in 2013 as a research scientist in the computer vision team, working on topics including fine-grained visual categorization, image retrieval and visual attention. From 2015 to 2019 she led the computer vision team at Xerox Research Centre Europe, and continued to serve in this role after its acquisition and transition to becoming NAVER LABS Europe. In 2019, she became the director of science at NAVER LABS Europe. In 2020, she joined Facebook AI Research where she is a senior research engineering manager. She has served as area chair for multiple computer vision and machine learning conferences, and as program chair for ICLR 2021. Her current research interests include video understanding and multi-modal search.
