Linear Algebra for Pattern Processing: Projection, Singular Value Decomposition, and Pseudoinverse (Synthesis Lectures on Signal Processing)

Linear algebra is one of the most basic foundations of a wide range of scientific domains, and most textbooks of linear algebra are written by mathematicians. However, this book is specifically intended to students and researchers of pattern information processing, analyzing signals such as images and exploring computer vision and computer graphics applications. The author himself is a researcher of this domain. Such pattern information processing deals with a large amount of data, which are represented by high-dimensional vectors and matrices. There, the role of linear algebra is not merely numerical computation of large-scale vectors and matrices. In fact, data processing is usually accompanied with "geometric interpretation." For example, we can think of one data set being "orthogonal" to another and define a "distance" between them or invoke geometric relationships such as "projecting" some data onto some space. Such geometric concepts not only help us mentally visualize abstracthigh-dimensional spaces in intuitive terms but also lead us to find what kind of processing is appropriate for what kind of goals. First, we take up the concept of "projection" of linear spaces and describe "spectral decomposition," "singular value decomposition," and "pseudoinverse" in terms of projection. As their applications, we discuss least-squares solutions of simultaneous linear equations and covariance matrices of probability distributions of vector random variables that are not necessarily positive definite. We also discuss fitting subspaces to point data and factorizing matrices in high dimensions in relation to motion image analysis. Finally, we introduce a computer vision application of reconstructing the 3D location of a point from three camera views to illustrate the role of linear algebra in dealing with data with noise. This book is expected to help students and researchers of pattern information processing deepen the geometric understanding of linear algebra.

Kenichi Kanatani received his B.E., M.S., and Ph.D. in applied mathematics from the University of Tokyo in 1972, 1974, and 1979, respectively. After serving as Professor of computer science at Gunma University, Gunma, Japan, and Okayama University, Okayama, Japan, he retired in 2013 and is now Professor Emeritus of Okayama University. He was a visiting researcher at the University of Maryland, U.S. (1985–1986, 1988–1989, 1992), the University of Copenhagen, Denmark (1988), the University of Oxford, U.K. (1991), INRIA at Rhone Alpes, France (1988), ETH, Switzerland (2013), University of Paris-Est, France (2014), Linkoping University, Sweden (2015), and National Taiwan Normal University, Taiwan (2019). He is the author of K. Kanatani, Group-Theoretical Methods in Image Understanding (Springer, 1990), K. Kanatani, Geometric Computation for Machine Vision (Oxford University Press, 1993), K. Kanatani, Statistical Optimization for Geometric Computation: Theory and Practice (Elsevier, 1996; reprinted Dover, 2005), K. Kanatani, Understanding Geometric Algebra: Hamilton, Grassmann, and Clifford for Computer Vision and Graphics (AK Peters/CRC Press 2015), K. Kanatani, Y. Sugaya, and Y. Kanazawa, Ellipse Fitting for Computer Vision: Implementation and Applications (Morgan & Claypool, 2016). K. Kanatani, Y. Sugaya, and Y. Kanazawa, Guide to 3D Vision Computation: Geometric Analysis and Implementation (Springer, 2016), and K. Kanatani, 3D Rotations: Parameter Computation and Lie Algebra based Optimization (AK Peters/CRC Press 2020). He received many awards including the best paper awards from IPSJ (1987), IEICE (2005), and PSIVT (2009). He is a Fellow of IEICE, IEEE, and IAPR.