Advanced Markov Chain Monte Carlo Methods: Learning from Past Samples - Hardcover

Liang, Faming; Liu, Chuanhai; Carroll, Raymond

Hardcover

ISBN 10: 0470748265 ISBN 13: 9780470748268

Publisher: Wiley, 2010

Synopsis

Markov Chain Monte Carlo (MCMC) methods are now an indispensable tool in scientific computing. This book discusses recent developments of MCMC methods with an emphasis on those making use of past sample information during simulations. The application examples are drawn from diverse fields such as bioinformatics, machine learning, social science, combinatorial optimization, and computational physics.

Key Features:

• Expanded coverage of the stochastic approximation Monte Carlo and dynamic weighting algorithms that are essentially immune to local trap problems.
• A detailed discussion of the Monte Carlo Metropolis-Hastings algorithm that can be used for sampling from distributions with intractable normalizing constants.
• Up-to-date accounts of recent developments of the Gibbs sampler.
• Comprehensive overviews of the population-based MCMC algorithms and the MCMC algorithms with adaptive proposals.

This book can be used as a textbook or a reference book for a one-semester graduate course in statistics, computational biology, engineering, and computer sciences. Applied or theoretical researchers will also find this book beneficial.

About this title

About the Author

Faming Liang, Associate Professor, Department of Statistics, Texas A&M University.

Chuanhai Liu, Professor, Department of Statistics, Purdue University.

Raymond J. Carroll, Distinguished Professor, Department of Statistics, Texas A&M University.

Key Features:

• Expanded coverage of the stochastic approximation Monte Carlo and dynamic weighting algorithms that are essentially immune to local trap problems.
• A detailed discussion of the Monte Carlo Metropolis-Hastings algorithm that can be used for sampling from distributions with intractable normalizing constants.
• Up-to-date accounts of recent developments of the Gibbs sampler.
• Comprehensive overviews of the population-based MCMC algorithms and the MCMC algorithms with adaptive proposals.

• Accompanied by a supporting website featuring datasets used in the book, along with codes used for some simulation examples.

This book can be used as a textbook or a reference book for a one-semester graduate course in statistics, computational biology, engineering, and computer sciences. Applied or theoretical researchers will also find this book beneficial.

Key Features:

• Expanded coverage of the stochastic approximation Monte Carlo and dynamic weighting algorithms that are essentially immune to local trap problems.
• A detailed discussion of the Monte Carlo Metropolis-Hastings algorithm that can be used for sampling from distributions with intractable normalizing constants.
• Up-to-date accounts of recent developments of the Gibbs sampler.
• Comprehensive overviews of the population-based MCMC algorithms and the MCMC algorithms with adaptive proposals.

• Accompanied by a supporting website featuring datasets used in the book, along with codes used for some simulation examples.

This book can be used as a textbook or a reference book for a one-semester graduate course in statistics, computational biology, engineering, and computer sciences. Applied or theoretical researchers will also find this book beneficial.

Excerpt. © Reprinted by permission. All rights reserved.

Advanced Markov Chain Monte Carlo Methods

Learning from Past SamplesBy Faming Liang Chuanhai Liu Raymond Carroll

John Wiley & Sons

Copyright © 2010 John Wiley & Sons, Ltd
All right reserved.
ISBN: 978-0-470-74826-8

Chapter One

Bayesian Inference and Markov Chain Monte Carlo

1.1 Bayes

Bayesian inference is a probabilistic inferential method. In the last two decades, it has become more popular than ever due to affordable computing power and recent advances in Markov chain Monte Carlo (MCMC) methods for approximating high dimensional integrals.

Bayesian inference can be traced back to Thomas Bayes (1764), who derived the inverse probability of the success probability θ in a sequence of independent Bernoulli trials, where θ was taken from the uniform distribution on the unit interval (0, 1) but treated as unobserved. For later reference, we describe his experiment using familiar modern terminology as follows.

* Example 1.1 The Bernoulli (or Binomial) Model With Known Prior

Suppose that θ ~ Unif(0, 1), the uniform distribution over the unit interval (0, 1), and that x₁, ..., x_n is a sample from Bernoulli(θ), which has the sample space X = {0, 1} and probability mass function (pmf)

Pr (X = 1|θ) = θ and Pr (X = 0|θ) = 1 - θ, (1.1)

where X denotes the Bernoulli random variable (r.v.) with X = 1 for success and X = 0 for failure. Write [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], the observed number of successes in the n Bernoulli trials. Then N|θ~ Binomial(n, θ), the Binomial distribution with parameters size n and probability of success θ.

The inverse probability of θ given x₁, ..., x_n, known as the posterior distribution, is obtained from Bayes' theorem, or more rigorously in modern probability theory, the definition of conditional distribution, as the Beta distribution Beta(1 + N, 1 + n-N) with probability density function (pdf)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.2)

where B(·, ·) stands for the Beta function.

1.1.1 Specification of Bayesian Models

Real world problems in statistical inference involve the unknown quantity θ and observed data X. For different views on the philosophical foundations of Bayesian approach, see Savage (1967a, b), Berger (1985), Rubin (1984), and Bernardo and Smith (1994). As far as the mathematical description of a Bayesian model is concerned, Bayesian data analysis amounts to

(i) specifying a sampling model for the observed data X, conditioned on an unknown quantity θ,

X ~ f(X|θ) (X [member of] X, θ [member of] Θ), (1.3)

where f(X|θ) stands for either pdf or pmf as appropriate, and

(ii) specifying a marginal distribution π(θ) for θ, called the prior distribution or simply the prior for short,

(θ) ~ π(θ) (θ [member of] Θ0. (1.4)

Technically, data analysis for producing inferential results on assertions of interest is reduced to computing integrals with respect to the posterior distribution, or posterior for short,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], (1.5)

where L(θ|X) [varies] f(X|θ) in θ, called the likelihood of θ given X. Our focus in this book is on efficient and accurate approximations to these integrals for scientific inference. Thus, limited discussion of Bayesian inference is necessary.

1.1.2 The Jeffreys Priors and Beyond

By its nature, Bayesian inference is necessarily subjective because specification of the full Bayesian model amounts to practically summarizing available information in terms of precise probabilities. Specification of probability models is unavoidable even for frequentist methods, which requires specification of the sampling model, either parametric or non-parametric, for the observed data X. In addition to the sampling model of the observed data X for developing frequentist procedures concerning the unknown quantity θ, Bayesian inference demands a fully specified prior for θ. This is natural when prior information on θ is available and can be summarized precisely by a probability distribution. For situations where such information is neither available nor easily quantified with a precise probability distribution, especially for high dimensional problems, a commonly used method in practice is the Jeffreys method, which suggests the prior of the form

π_J (θ) [varies] |I(θ)|^1/2 (θ [member of] Θ), (1.6)

where I(θ) denotes the Fisher information

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

The Jeffreys priors have the appealing property that they are invariant under reparameterization. A theoretical adjustment in terms of frequency properties in the context of large samples can be found in Welch and Peers (1963). Note that prior distributions do not need to be proper as long as the posteriors are proper and produce sensible inferential results. The following Gaussian example shows that the Jeffreys prior is sensible for single parameters.

* Example 1.2 The Gaussian N(μ, 1) Model

Suppose that a sample is considered to have taken from the Gaussian population N(μ, 1) with unit variance and unknown mean μ to be inferred. The Fisher information is obtained as

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where φ (x - μ) = (2π)^-1/2 exp{-1/2 (x - μ)²} is the pdf of N(μ, 1). It follows that the Jeffreys prior for θ is the flat prior

π_J (μ) [varies] 1 (-∞ < μ < ∞), (1.7)

resulting in the corresponding posterior distribution of θ given X

π_J (μ|X) = N(X, 1). (1.8)

Care must be taken when using the Jeffreys rule. For example, it is easy to show that applying the Jeffreys rule to the Gaussian model N(μ, σ²) with both mean μ and variance σ² unknown leads to the prior

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

However, this is not the commonly used prior that has better frequency properties (for inference about μ or σ) and is given by

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

that is, μ and σ² are independent and the distributions for both μ and ln σ² are flat. For high dimensional problems with small samples, the Jeffreys rule often becomes even less appealing. There are also different perspectives, provided by the extensive work on reference priors by José Bernardo and James Berger (see, e.g., Bernardo, 1979; Berger, 1985). For more discussion of prior specifications, see Kass and Wasserman (1996).

For practical purposes, we refer to Box and Tiao (1973) and Gelman et al. (2004) for discussion on specification of prior distributions. The general guidance for specification of priors when no prior information is available, as is typical in Bayesian analysis, is to find priors that lead to posteriors having good frequency properties (see, e.g., Rubin, 1984; Dawid, 1985). Materials on probabilistic inference without using difficult-to-specify priors are available but beyond the scope of Bayesian inference and therefore will not be discussed in this book. Readers interested in this fascinating area are referred to Fisher (1973), Dempster (2008), and Martin et al. (2009). We note that MCMC methods can be applied there as well.

1.2 Bayes Output

Bayesian analysis for scientific inference does not end with posterior derivation and computation. It is thus critical for posterior distributions to have clear interpretation. For the sake of clarity, probability used in this book has a long-run frequency interpretation in repeated experiments. Thus, standard probability theory, such as conditioning and marginalization, can be applied. Interpretation also suggests how to report Bayesian output as our assessment of assertions of interest on quantities in the specified model. In the following two subsections, we discuss two types of commonly used Bayes output, credible intervals for estimation and Bayes factors for hypothesis testing.

1.2.1 Credible Intervals and Regions

Credible intervals are simply posterior probability intervals. They are used for purposes similar to those of confidence intervals in frequentist statistics and thereby are also known as Bayesian confidence intervals. For example, the 95% left-sided Bayesian credible interval for the parameter μ in the Gaussian Example 1.2 is [-∞, X + 1.64], meaning that the posterior probability that μ lies in the interval from -∞ to X + 1.64 is 0.95. Similar to frequentist construction of two-sided intervals, for given α [member of] (0, 1), a 100(1 - α)% two-sided Bayesian credible interval for a single parameter θ with equal posterior tail probabilities is defined as

[θ_α/2, θ_1-α/2] (1.9)

where the two end points are the α/2 and 1-α/2 quantiles of the (marginal) posterior distribution of θ. For the the Gaussian Example 1.2, the two-sided 95% Bayesian credible interval is [X - 1.96, X + 1.96].

In dealing simultaneously with more than one unknown quantity, the term credible region is used in place of credible interval. For a more general term, we refer to credible intervals and regions as credible sets. Constructing credible sets is somewhat subjective and usually depends on the problems of interest. A common way is to choose the region with highest posterior density (h.p.d.). The 100(1 - α)% h.p.d. region is given by

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.10)

for some θ_1-α satisfying

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

For the the Gaussian Example 1.2, the 95% h.p.d. interval is [X - 1.96, X + 1.96], the same as the two-sided 95% Bayesian credible interval because the posterior of μ is unimodal and symmetric. We note that the concept of h.p.d. can also be used for functions of θ such as components of θ in high dimensional situations.

For a given probability content (1 - α), the h.p.d. region has the smallest volume in the space of θ. This is attractive but depends on the functional form of unknown quantities, such as θ and θ². An alternative credible set is obtained by replacing the posterior density π(θ|X) in (1.10) with the likelihood L(θ|X):

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.11)

for some θ_1-α satisfying

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

The likelihood based credible region does not depend on transformation of θ. This is appealing, in particular when no prior information is available on θ, that is, when the specified prior works merely as a working prior leading to inference having good frequency properties.

1.2.2 Hypothesis Testing: Bayes Factors

While the use of credible intervals is a Bayesian alternative to frequentist confidence intervals, the use of Bayes factors has been a Bayesian alternative to classical hypothesis testing. Bayes factors have also been used to develop Bayesian methods for model comparison and selection. Here we review the basics of Bayes factors. For more discussion on Bayes factors, including its history, applications, and difficulties, see Kass and Raftery (1995), Gelman et al. (2004), and references therein.

The concept of Bayes factors is introduced in the situation with a common observed data X and two competing hypotheses denoted by H₁ and H₂. A full Bayesian analysis requires

(i) specifying a prior distribution on H₁ and H₂, denoted by, Pr (H₁) and Pr (H₂), and

(ii) for each k = 1 and 2, specifying the likelihood L_k(θ_k|X) = f_k(X|θ_k) and prior π(θ_k|H_k) for θ_k, conditioned on the truth of H_k, where θ_k is the parameter under H_k.

Integrating out θ_k yields

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.12)

for k = 1 and 2. The Bayes factor is the posterior odds of one hypothesis when the prior probabilities of the two hypotheses are equal. More precisely, the Bayes factor in favor of H₁ over H₂ is defined as

B₁₂ = Pr (X|H₁)/Pr (X|H₂). (1.13)

The use of Bayes factors for hypothesis testing is similar to the likelihood ratio test, but instead of maximizing the likelihood, Bayesians in favor of Bayes factors average it over the parameters. According to the definition of Bayes factors, proper priors are often required. Thus, care must be taken in specification of priors so that inferential results are meaningful. In addition, the use of Bayes factors renders lack of probabilistic feature of Bayesian inference. In other words, it is consistent with the likelihood principle, but lacks of a metric or a probability scale to measure the strength of evidence. For a summary of evidence provided by data in favor of H₁ over H₂, Jeffreys (1961) (see also Kass and Raftery (1995)) proposed to interpret the Bayes factor as shown in Table 1.1.

The use of Bayes factor is illustrated by the following binomial example.

* Example 1.3 The Binomial Model (continued with a numerical example)

Suppose we take a sample of n = 100 from Bernoulli(θ) with unknown θ, and observe N = 63 successes and n - N = 37 failures. Suppose that two competing hypotheses are

H₁ : θ = 1/2 and H₂ : θ ≠ 1/2. (1.14)

Under H₁, the likelihood is calculated according to the binomial distribution:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Under H₂, instead of the uniform over the unit interval we consider the Jeffreys prior

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

the proper Beta distribution with shape parameters 1/2 and 1/2. Hence, we have

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

The Bayes factor log₁₀(B₁₂) is then -0.4, which is 'barely worth mentioning' even if it points very slightly towards H₂.

It has been recognized that Bayes factor can be sensitive to the prior, which is related to what is known as Lindley's paradox (see Shafer (1982)).

This is shown in Figure 1.1 for a class of Beta priors Beta(α, 1 - α) for 0 ≤ α ≤ 1. The Bayes factor is infinity at the two extreme priors corresponding to α = 0 and α = 1. It can be shown that this class of priors is necessary in the context of imprecise Bayes for producing inferential results that have desired frequency properties. This supports the idea that care must be taken in interpreting Bayesian factors in scientific inference.

Bayesian factors are not the same as a classical likelihood ratio test. A frequentist hypothesis test of H₁ considered as a null hypothesis would have produced a more dramatic result, saying that H₁ could be rejected at the 1% significance level, since the probability of getting 63 or more successes from a sample of 100 if θ = 1/2 is 0.0060, and as a normal approximation based two-tailed test of getting a figure as extreme as or more extreme than 63 is 0.0093. Note that 63 is more than two standard deviations away from 50, the expected count under H₁.

1.3 Monte Carlo Integration

1.3.1 The Problem

Let ν be a probability measure over the Borel σ-field X on the sample space X [suvset or equal to] R^d, where R^d denotes the d-dimensional Euclidian space. A commonly encountered challenging problem is to evaluate integrals of the form

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.15)

where h(x) is a measurable function. Suppose that ν has a pdf f(x). Then (1.15) can be written as

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1.16)

(Continues...)

Excerpted from Advanced Markov Chain Monte Carlo Methodsby Faming Liang Chuanhai Liu Raymond Carroll Copyright © 2010 by John Wiley & Sons, Ltd. Excerpted by permission of John Wiley & Sons. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Edition details

• Publisher: Wiley
• Publication date: 2010
• Language: English
• ISBN 10: 0470748265
• ISBN 13: 9780470748268
• Binding: Hardcover
• Edition number: 1
• Number of pages: 384