An Introduction to Bayesian Scientific Computing: Ten Lectures on Subjective Computing

The book of nature, according to Galilei, is written in the language of mat- matics. The nature of mathematics is being exact, and its exactness is und- lined by the formalism used by mathematicians to write it. This formalism, characterized by theorems and proofs, and syncopated with occasional l- mas, remarks and corollaries, is so deeply ingrained that mathematicians feel uncomfortable when the pattern is broken, to the point of giving the - pression that the attitude of mathematicians towards the way mathematics should be written is almost moralistic. There is a de?nition often quoted, “A mathematician is a person who proves theorems”, and a similar, more alchemistic one, credited to Paul Erd? os, but more likely going back to Alfr´ ed R´ enyi,statingthat“Amathematicianisamachinethattransformsco?eeinto 1 theorems ”. Therefore it seems to be the form, not the content, that char- terizes mathematics, similarly to what happens in any formal moralistic code wherein form takes precedence over content. This book is deliberately written in a very di?erent manner, without a single theorem or proof. Since morality has its subjective component, to pa- phrase Manuel Vasquez Montalban, we could call it Ten Immoral Mathemat- 2 ical Recipes . Does the lack of theorems and proofs mean that the book is more inaccurate than traditional books of mathematics? Or is it possibly just a sign of lack of co?ee? This is our ?rst open question. Exactness is an interesting concept.

A combination of the concepts subjective – or Bayesian – statistics and scientific computing, the book provides an integrated view across numerical linear algebra and computational statistics. Inverse problems act as the bridge between these two fields where the goal is to estimate an unknown parameter that is not directly observable by using measured data and a mathematical model linking the observed and the unknown.

Inverse problems are closely related to statistical inference problems, where the observations are used to infer on an underlying probability distribution. This connection between statistical inference and inverse problems is a central topic of the book. Inverse problems are typically ill-posed: small uncertainties in data may propagate in huge uncertainties in the estimates of the unknowns. To cope with such problems, efficient regularization techniques are developed in the framework of numerical analysis. The counterpart of regularization in the framework of statistical inference is the use prior information. This observation opens the door to a fruitful interplay between statistics and numerical analysis: the statistical framework provides a rich source of methods that can be used to improve the quality of solutions in numerical analysis, and vice versa, the efficient numerical methods bring computational efficiency to the statistical inference problems.

This book is intended as an easily accessible reader for those who need numerical and statistical methods in applied sciences.