Generative AI and Stochastic Thermodynamics: A Tale of Free Energies

Originating from lectures delivered at the African Institute of Mathematical Sciences, this book presents a unifying perspective on traditional and modern methods in generative AI and stochastic thermodynamics. By relating the core topics in machine learning to the notion of (variational) free-energy, a bridge is built between methods such as latent variable models, variational auto-encoders, optimal control, optimal transport, normalizing flows and diffusion models and concepts such as entropy production and fluctuation theorems in stochastic thermodynamics. Structured into three main parts, the book commences by setting up the required mathematical and statistical physics preliminaries needed to make it broadly accessible. The largest part of the book then focuses on building intuition of major advances in generative AI by considering discrete time processes and their relationship to topics in stochastic thermodynamics. Finally, the authors take a short excursion to the continuous time domain for the more advanced learner.

Max Welling is the co-founder and Chief Technology Officer of the startup CuspAI, and a Full Professor of Machine Learning at the University of Amsterdam. He is a member of the Dutch Royal Academy of Sciences and the Canadian Institute for Advanced Research, and a fellow of the European Lab for Learning and Intelligent Systems. Professor Welling received the ECCV Koenderink Prize in 2010, the 2021 ICML Test of Time Award, and the 2024 ICLR Test of Time Award.

Sirui Lu is a doctoral researcher at the Max Planck Institute of Quantum Optics, Germany. His research focuses on the intersection of (quantum) physics and artificial intelligence. He previously earned his master's and bachelor's degrees in physics from the Technical University of Munich and Tsinghua University, Beijing, respectively.

Lars Holdijk is a PhD student at the University of Oxford. His research focuses on the intersection of generative artificial intelligence, computational biochemistry, and statistical physics.