Hierarchical Decomposition in Reinforcement Learning

Revision with unchanged content. Reinforcement learning is an area of artificial intelligence that studies the ability of autonomous agents to improve their behavior in the absence of an informed instructor. Although reinforcement learning has achieved success in a wide range of applications, it becomes less consistent as the size of a task grows. This book attempts to improve the efficiency of reinforcement learning in realistic tasks by identifying a certain type of task structure. A task that displays this type of structure can be decomposed into a hierarchy of subtasks. Each subtask can be simplified using state abstraction so that it is much easier to solve than the original task. Reinforcement learning can be applied to produce solutions to the subtasks, and the solutions can be combined to achieve a solution to the original task. Experimental results indicate that hierarchical decomposition combined with state abstraction can significantly simplify the solution of realistic tasks. The book thus contributes to increasing the potential of reinforcement learning in realistic tasks. The book is directed towards researchers in Artificial Intelligence, but can also be used as a reference by professionals in Robotics and Autonomous Control Engineering.

Ph.D., completed his doctoral studies in computer science at the University of Massachusetts Amherst, USA, in 2006. Currently he is a visiting lecturer in computer science at Universitat Pompeu Fabra, Barcelona, Spain. His research focuses on exploiting structure in sequential decision problems to make them tractable.