Stochastic Control of Hereditary Systems and Applications

ThisresearchmonographdevelopstheHamilton-Jacobi-Bellman(HJB)theory viathedynamicprogrammingprincipleforaclassofoptimalcontrolproblems for stochastic hereditary di?erential equations (SHDEs) driven by a standard Brownian motion and with a bounded or an unbounded but fading m- ory. These equations represent a class of in?nite-dimensional stochastic s- tems that become increasingly important and have wide range of applications in physics, chemistry, biology, engineering, and economics/?nance. The wide applicability of these systems is due to the fact that the reaction of re- world systems to exogenous e?ects/signals is never “instantaneous” and it needs some time, time that can be translated into a mathematical language by some delay terms. Therefore, to describe these delayed e?ects, the drift and di?usion coe?cients of these stochastic equations depend not only on the current state but also explicitly on the past history of the state variable. The theory developed herein extends the ?nite-dimensional HJB theory of controlled di?usion processes to its in?nite-dimensional counterpart for c- trolledSHDEsinwhichacertainin?nite-dimensionalBanachspaceorHilbert space is critically involved in order to account for the bounded or unbounded memory. Another type of in?nite-dimensional HJB theory that is not treated in this monograph but arises from real-world application problems can often be modeled by controlled stochastic partial di?erential equations. Although they are both in?nite dimensional in nature and are both in the infancy of their developments, the SHDE exhibits many characteristics that are not in common with stochastic partial di?erential equations. Consequently, the HJB theory for controlled SHDEs is parallel to and cannot betreated as a subset of the theory developed for controlled stochastic partial di?erential equations.

This research monograph develops the Hamilton-Jacobi-Bellman (HJB) theory through dynamic programming principle for a class of optimal control problems for stochastic hereditary differential systems. It is driven by a standard Brownian motion and with a bounded memory or an infinite but fading memory.

The optimal control problems treated in this book include optimal classical control and optimal stopping with a bounded memory and over finite time horizon.

This book can be used as an introduction for researchers and graduate students who have a special interest in learning and entering the research areas in stochastic control theory with memories. Each chapter contains a summary.

Mou-Hsiung Chang is a program manager at the Division of Mathematical Sciences for the U.S. Army Research Office.