Plenoptic Imaging and Processing (Advances in Computer Vision and Pattern Recognition)

This open access book delves into the fundamental principles and cutting-edge techniques of plenoptic imaging and processing. Derived from the Latin words "plenus" (meaning "full") and "optic," plenoptic imaging offers a transformative approach to optical imaging. Unlike conventional systems that rely solely on the pinhole camera model to capture spatial information, plenoptic imaging aims to detect and reconstruct multidimensional and multiscale information from light rays in space.

Chapter 1 begins with the introduction of the basic principle of the plenoptic function and the historical development of plenoptic imaging. Next, Chapter 2 describes representative plenoptic sensing systems, including single-sensor devices with lenslet arrays, coded-aperture masks, structured camera arrays, and unstructured camera arrays. Then, Chapter 3 introduces gigapixel plenoptic sensing techniques capable of capturing large-scale dynamic scenes with extremely high resolution. Further, chapter 4 examines typical plenoptic reconstruction methods, including light-field image reconstruction, image-based, and RGBD-based geometry reconstruction. After that, chapter 5 tackles the challenges of large-scale plenoptic reconstruction by introducing sparse-view priors, high-resolution observations, and semantic information. Finally, chapter 6 discusses the frontier issues of plenoptic processing, including the gigapixel-level video dataset PANDA and corresponding visual intelligent algorithms.

Lu FANG is currently a full professor at the Dept. of Electronic Engineering, Tsinghua University. She received Ph.D from Hong Kong Univ. of Science and Technology in 2011, and B.E. from Univ. of Science and Technology of China in 2007. Dr. Fang performs multidisciplinary researches in the fields of Computational Imaging and Neuromorphic Computing. To transcend the boundaries of visual computing technology, she focused on hardware-software co-designed imaging and opto-electronic co-optimized computing. She published more than 100 peer-reviewed journal and conference papers, including Science, Nature, Nature Photonics, Nature Machine Intelligence, Nature Methods, IEEE TPMAI etc., and holds 76 international/national patents. Dr. Fang supervised 16 PhD and 21 MPhil students in total. She received NSFC Distinguished Young Scholar Award, DAMO Academy Young Fellow Award, MIT TR35 innovators (China) Award, Multimedia Rising Star Award in ICME 2019, Best Student Paper Award in ICME 2017 etc. She serves as the General Chair of IEEE MMSP 2025, Student Activities Co-Chair of IEEE ICIP 2026, Demo Co-chair of IEEE ICME 2023, Grand Challenge Co-chair of ACM MM 2021 and the leading organizer of GigaVision Challenges/Workshops in CVPR, ECCV, ICCV etc. Dr. Fang is IEEE Senior Member, TC member of MMSP-TC, and serves as Associate Editor for IEEE TIP and OSA Optica.