Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning (Operations Research/Computer Science Interfaces Series, 55) - Hardcover

Gosavi, Abhijit

Hardcover

ISBN 10: 1489974903 ISBN 13: 9781489974907

Publisher: Springer, 2014

Synopsis

This book introduces the reader to the evolving area of simulation-based optimization, also known as simulation optimization. The book should serve as an accessible introduction to this topic and requires a background only in elementary mathematics. It brings the reader up to date on cutting-edge advances in simulation-optimization methodologies, including dynamic controls, also called Reinforcement Learning (RL) or Approximate Dynamic Programming (ADP), and static optimization techniques, e.g.,Simultaneous Perturbation, Nested Partitions, Backtracking Adaptive Search, Response Surfaces, and Meta-Heuristics. Special features of this book include:

Stochastic Control Optimization:

• An Accessible Introduction to Reinforcement Learning Techniques for Solving Markov Decision Processes (MDPs), with Step-by-Step Descriptions of Numerous Algorithms, e.g., Q-Learning, SARSA, R-SMART, Actor-Critics, Q-P-Learning, and Classical Approximate Policy Iteration
• A Detailed Discussion on Dynamic Programing for Solving MDPs and Semi-MDPs (SMDPs), Including Steps for Value Iteration and Policy Iteration
• An Introduction to Function Approximation with Reinforcement Learning
• An In-Depth Treatment of Reinforcement Learning Methods for SMDPs, Average Reward Problems, Finite Horizon Problems, and Two Time Scales
• Computer Programs (available online)
• A Gentle Introduction to Convergence Analysis via Banach Fixed Point Theory and Ordinary Differential Equations (ODEs)

Stochastic Static Optimization:

• A Step-by-Step Description of Stochastic Adaptive Search Algorithms, e.g., Simultaneous Perturbation, Nested Partitions, Backtracking Adaptive Search, Stochastic Ruler, and Meta-Heuristics, e.g., Simulated Annealing, Tabu Search, and Genetic Algorithms 
• A Clear and Simple Introduction to the Methodology of Neural Networks 

The book ends with a chapter on case studies that explain how these methods can be applied in real-world settings; an online repository of computer programs that can be downloaded from a website is also available.

The book was written for students and researchers in the fields ofengineering (industrial, electrical, and computer), computer science,operations research, management science, and applied mathematics.  Anattractive feature of this book is its accessibility to readers new to this topic.

About this title

About the Author

Abhijit Gosavi is a researcher who works in the area of reinforcement learning, stochastic dynamic programming, and simulation-based optimization. The first edition of his Springer book "Simulation-Based Optimization" that appeared in 2003 was the first text to have appeared on that topic. He is regularly an invited speaker at major national and international academic conferences.

He has published more than fifty journal and conference articles - many of which have appeared in leading scholarly journals such as Management Science, Automatica, INFORMS Journal on Computing, Machine Learning, Journal of Retailing, Systems and Control Letters and the European Journal of Operational Research.


"The statements of science are not of what is true and what is not true, but statements of what is known with different degrees of certainty." --- Richard Feynman

From the Back Cover

Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning introduces the evolving area of static and dynamic simulation-based optimization. Covered in detail are model-free optimization techniques – especially designed for those discrete-event, stochastic systems which can be simulated but whose analytical models are difficult to find in closed mathematical forms.

Key features of this revised and improved Second Edition include:

· Extensive coverage, via step-by-step recipes, of powerful new algorithms for static simulation optimization, including simultaneous perturbation, backtracking adaptive search, and nested partitions, in addition to traditional methods, such as response surfaces, Nelder-Mead search, and meta-heuristics (simulated annealing, tabu search, and genetic algorithms)

· Detailed coverage of the Bellman equation framework for Markov Decision Processes (MDPs), along with dynamic programming (value and policy iteration) for  discounted, average, and total reward performance metrics

· An in-depth consideration of dynamic simulation optimization via temporal differences and Reinforcement Learning: Q-Learning, SARSA, and R-SMART algorithms, and policy search, via API, Q-P-Learning, actor-critics, and learning automata

· A special examination of neural-network-based function approximation for Reinforcement Learning, semi-Markov decision processes (SMDPs), finite-horizon problems, two time scales, case studies for industrial tasks, computer codes (placed online), and convergence proofs, via Banach fixed point theory and Ordinary Differential Equations

Themed around three areas in separate sets of chapters – Static Simulation Optimization, Reinforcement Learning, and Convergence Analysis – this book is written for researchers and students in the fields of engineering (industrial, systems, electrical, and computer), operations research, computer science, and applied mathematics.

Edition details

• Publisher: Springer
• Publication date: 2014
• ISBN 10: 1489974903
• ISBN 13: 9781489974907
• Binding: Hardcover
• Language: English
• Edition number: 2
• Number of pages: 534