Sequential Decision Making for AI: Interaction, Learning, Memory, and Control

What changes when a learning system does more than predict—when it acts, changes what it will observe next, remembers a history, and must control a process over time?

Sequential Decision Making for AI develops the mathematical language for that setting. Beginning with decision theory, Markov decision processes, bandits, and dynamic programming, it builds toward reinforcement learning under exploration, function approximation, offline data, partial observability, planning, constraints, and multi-agent interaction. The unifying theme is interaction: once actions influence future data, the objects that govern learning change.

The book separates familiar ideas from the assumptions that make them valid. Representability is distinguished from learnability; Bellman closure from mere function-class membership; offline sample size from policy coverage; belief-state sufficiency from minimal memory; and statistical information from computational access. Upper bounds, lower bounds, and algorithmic guarantees are tied to the observation model, interaction protocol, horizon, and resource being counted.

The same framework is connected to modern foundation-model post-training and agent design: behavioural cloning, preference and reward modelling, KL-regularised optimisation, offline evaluation, context and recurrent memory, test-time planning, tool use, constrained control, and multi-agent interaction. These are interpreted through theorem-native quantities such as interaction rounds, independent samples, effective horizon, coverage, planning depth, memory bits, and computation.

Written for graduate students and researchers in machine learning, statistics, applied mathematics, control, and AI, this volume is not an algorithm catalogue. It is a resource-aware guide to what sequential-learning theorems establish, what they leave open, and what additional assumptions are required before they become claims about real AI agents.