Dynamic Programming and Optimal Control - Hardcover

Bertsekas, Dimitri P.

Hardcover

ISBN 10: 1886529434 ISBN 13: 9781886529434

Publisher: Athena Scientific, 2017

Synopsis

This 4th edition is a major revision of Vol. I of the leading two-volume dynamic programming textbook by Bertsekas, and contains a substantial amount of new material, particularly on approximate DP in Chapter 6. This chapter was thoroughly reorganized and rewritten, to bring it in line, both with the contents of Vol. II, whose latest edition appeared in 2012, and with recent developments, which have propelled approximate DP to the forefront of attention.

Some of the highlights of the revision of Chapter 6 are an increased emphasis on one-step and multistep lookahead methods, parametric approximation architectures, neural networks, rollout, and Monte Carlo tree search. Among other applications, these methods have been instrumental in the recent spectacular success of computer Go programs. The material on approximate DP also provides an introduction and some perspective for the more analytically oriented treatment of Vol. II.

The book includes a substantial number of examples, and exercises, detailed solutions of many of which are posted on the internet. It was developed through teaching graduate courses at M.I.T., and is supported by a large amount of educational material, such as slides and videos, posted at the MIT Open Courseware, the author's, and the publisher's web sites.

Contents: 1. The Dynamic Programming Algorithm. 2. Deterministic Systems and the Shortest Path
Problem. 3. Problems with Perfect State Information. 4. Problems with Imperfect State Information. 5. Introduction to Infinite Horizon Problems. 6. Approximate Dynamic Programming. 7. Deterministic Continuous-Time Optimal Control.

About this title

About the Author

Dimitri Bertsekas is McAffee Professor of Electrical Engineering and Computer Science at the Massachusetts Institute of Technology, and a member of the National Academy of Engineering. He has researched a broad variety of subjects from optimization theory, control theory, parallel and distributed computation, systems analysis, and data communication networks. He has written numerous papers in each of these areas, and he has authored or coauthored sixteen textbooks. Professor Bertsekas was awarded the INFORMS 1997 Prize for Research Excellence in the Interface Between Operations Research and Computer Science for his book "Neuro-Dynamic Programming" (co-authored with John Tsitsiklis), the 2001 ACC John R. Ragazzini Education Award, the 2009 INFORMS Expository Writing Award, the 2014 ACC Richard E. Bellman Control Heritage Award for "contributions to the foundations of deterministic and stochastic optimization-based methods in systems and control," the 2014 Khachiyan Prize for Life-Time Accomplishments in Optimization, and the 2015 George B. Dantzig Prize. In 2018, he was awarded jointly with John Tsitsiklis, the INFORMS John von Neumann Theory Prize, for the contributions of the research monographs "Parallel and Distributed Computation" and "Neuro-Dynamic Programming". In 2001, he was elected to the United States National Academy of Engineering for "pioneering contributions to fundamental research, practice and education of optimization/control theory"

Review

In addition to being very well written and organized, the material has several special features that make the book unique in the class of introductory textbooks on dynamic programming. For instance, it presents both deterministic and stochastic control problems, in both discrete- and continuous-time, and it also presents the Pontryagin minimum principle for deterministic systems together with several extensions. It contains problems with perfect and imperfect information, as well as minimax control methods (also known as worst-case control problems or games against nature). I also has a full chapter on suboptimal control and many related techniques, such as open-loop feedback controls, limited lookahead policies, rollout algorithms, and model predictive control, to name a few. ... In conclusion the book is highly recommendable for an introductory course on dynamic programming and its applications. --Onesimo Hernandez Lerma, in Mathematic Reviews

By its comprehensive coverage, very good material organization, readability of the exposition, included theoretical results, and its challenging examples and exercises, the reviewed book is highly recommended for a graduate course in dynamic programming or for self-study. It is a valuable reference for control theorists, mathematicians, and all those who use systems and control theory in their work. Students will for sure find the approach very readable, clear, and concise. Misprints are extremely few. --Vasile Sima, in SIAM Review

Edition details

• Publisher: Athena Scientific
• Publication date: 2017
• ISBN 10: 1886529434
• ISBN 13: 9781886529434
• Binding: Hardcover
• Language: English
• Edition number: 4
• Number of pages: 555