Stochastic Control in Discrete and Continuous Time

This book contains an introduction to three topics in stochastic control: discrete time stochastic control, i. e. , stochastic dynamic programming (Chapter 1), piecewise - terministic control problems (Chapter 3), and control of Ito diffusions (Chapter 4). The chapters include treatments of optimal stopping problems. An Appendix - calls material from elementary probability theory and gives heuristic explanations of certain more advanced tools in probability theory. The book will hopefully be of interest to students in several ?elds: economics, engineering, operations research, ?nance, business, mathematics. In economics and business administration, graduate students should readily be able to read it, and the mathematical level can be suitable for advanced undergraduates in mathem- ics and science. The prerequisites for reading the book are only a calculus course and a course in elementary probability. (Certain technical comments may demand a slightly better background. ) As this book perhaps (and hopefully) will be read by readers with widely diff- ing backgrounds, some general advice may be useful: Don’t be put off if paragraphs, comments, or remarks contain material of a seemingly more technical nature that you don’t understand. Just skip such material and continue reading, it will surely not be needed in order to understand the main ideas and results. The presentation avoids the use of measure theory.

This book provides a comprehensive introduction to stochastic control problems in discrete and continuous time. The material is presented logically, beginning with the discrete-time case before proceeding to the stochastic continuous-time models. Central themes are dynamic programming in discrete time and HJB-equations in continuous time. Topics covered include stochastic maximum principles for discrete time and continuous time, even for problems with terminal conditions. Numerous illustrative examples and exercises, with solutions at the end of the book, are included to enhance the understanding of the reader. By interlinking many fields in stochastic control, the material gives the student the opportunity to see the connections between different fields and the underlying ideas that unify them.

This text will benefit students in applied mathematics, economics, engineering, and related fields. Prerequisites include a course in calculus and elementary probability theory. No knowledge of measure theory is assumed.