CONTINUOUS OPTIMIZATION FOR DATA SCIENCE

The text is divided into three main parts: unconstrained optimization, constrained optimization, and linear programming. The first part addresses unconstrained optimization in single-variable and multivariable functions, introducing key algorithms such as steepest descent, Newton, and quasi-Newton methods. The second part focuses on constrained optimization, starting with linear equality constraints and extending to more general cases, including inequality constraints. It details optimality conditions, sensitivity analysis, and relevant algorithms for solving these problems. The third part covers linear programming, presenting the formulation of LP problems, the simplex algorithm, and sensitivity analysis. Throughout, the text provides numerous applications to data science, such as linear regression, maximum likelihood estimation, expectation-maximization algorithms, support vector machines, and linear neural networks.

Moshe Haviv received his BSc in Mathematics from Tel Aviv University in 1979 and his MA and PhD in Operations Research from Yale University in 1982 and 1983, respectively. He joined the Department of Statistics at the Hebrew University in 1984 and has been a Professor since 2002 until his retirement in 2020. He served as Head of Department from 2008 to 2012. In 2010, he joined the Center for the Study of Rationality. During leaves, he taught at the University of British Columbia and the University of Sydney. Moshe was the president of the Operations Research Society of Israel from 2011 to 2014. Currently, he is a Professor at the School of Data Science, Chinese University of Hong Kong, Shenzhen. His research areas include queueing systems in general and strategic decision making in queues in particular. Other areas of interest are numerical issues in Markov chains and Markov decision processes. Among his publications is the book titled To queue or not to queue: Equilibrium behavior in queueing systems, co-authored with Refael Hassin (Kluwer, 2003). He also published the textbook Queues: A Course in Queueing Theory (Springer, 2013). His latest textbook is titled Linear Algebra for Data Science (World Scientific, 2023). He has also published more than 80 research papers.