# Sampling from multimodal distributions using tempered transitions

@article{Neal1996SamplingFM, title={Sampling from multimodal distributions using tempered transitions}, author={Radford M. Neal}, journal={Statistics and Computing}, year={1996}, volume={6}, pages={353-366} }

I present a new Markov chain sampling method appropriate for distributions with isolated modes. Like the recently developed method of ‘simulated tempering’, the ‘tempered transition’ method uses a series of distributions that interpolate between the distribution of interest and a distribution for which sampling is easier. The new method has the advantage that it does not require approximate values for the normalizing constants of these distributions, which are needed for simulated tempering… Expand

#### Topics from this paper

#### Paper Mentions

#### 336 Citations

Annealed importance sampling

- Mathematics, Physics
- Stat. Comput.
- 2001

It is shown how one can use the Markov chain transitions for such an annealing sequence to define an importance sampler, which can be seen as a generalization of a recently-proposed variant of sequential importance sampling. Expand

Tuning tempered transitions

- Mathematics, Computer Science
- Stat. Comput.
- 2012

This work considers how the tempered transitions algorithm may be tuned to increase the acceptance rates for a given number of temperatures and finds that the commonly assumed geometric spacing of temperatures is reasonable in many but not all applications. Expand

On a Selection of Advanced Markov Chain Monte Carlo Algorithms for Everyday Use: Weighted Particle Tempering, Practical Reversible Jump, and Extensions

- Computer Science
- 2016

It is shown that weighted particle tempering outperforms two similar methods: parallel tempering and parallel hierarchical sampling, and it is proved that the parallelized form of weighted particles tempering preserves detailed balance in an asymptotic sense. Expand

MCMC algorithms for sampling from multimodal and changing distributions

- Computer Science
- 2019

This thesis develops a general method to prove mixing time using “soft decompositions” of Markov processes, and uses it to prove rapid mixing for (polynomial) mixtures of log-concave distributions, and addresses the problem of sampling from multimodal distributions. Expand

Umbrella Sampling and Simulated Tempering

- Mathematics
- 1998

In principle, the Metropolis algorithm allows one to construct a Markov chain that simulates the equilibrium distribution of a wide variety of polymeric and other physical systems. Unfortunately,… Expand

Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition

- Computer Science, Mathematics
- ArXiv
- 2018

This work combines Langevin diffusion with simulated tempering to create a Markov chain that mixes more rapidly by transitioning between different temperatures of the distribution, and introduces novel techniques for proving spectral gaps based on decomposing the action of the generator of the diffusion. Expand

Dynamic Tempered Transitions for Exploring Multimodal Posterior Distributions

- Mathematics
- Political Analysis
- 2004

Multimodal, high-dimension posterior distributions are well known to cause mixing problems for standard Markov chain Monte Carlo (MCMC) procedures; unfortunately such functional forms readily occur… Expand

Weighted particle tempering

- Mathematics, Computer Science
- Comput. Stat. Data Anal.
- 2017

Weighted particle tempering is shown to outperform two similar methods: parallel tempering and parallel hierarchical sampling and two case studies are explored: breast cancer classification and graphical models for financial data. Expand

Importance tempering

- Mathematics, Computer Science
- Stat. Comput.
- 2010

A new optimal method for combining multiple IS estimators is derived and it is proved that the resulting estimator has a highly desirable property related to the notion of effective sample size. Expand

Advances in Markov chain Monte Carlo methods

- Mathematics
- 2007

Probability distributions over many variables occur frequently in Bayesian inference, statistical physics and simulation studies. Samples from distributions give insight into their typical behavior… Expand

#### References

SHOWING 1-10 OF 14 REFERENCES

Annealing Markov chain Monte Carlo with applications to ancestral inference

- Mathematics
- 1995

Abstract Markov chain Monte Carlo (MCMC; the Metropolis-Hastings algorithm) has been used for many statistical problems, including Bayesian inference, likelihood inference, and tests of significance.… Expand

Simulated Tempering: A New Monte Carlo Scheme

- Physics
- 1992

We propose a new global optimization method (Simulated Tempering) for simulating effectively a system with a rough free-energy landscape (i.e., many coexisting states) at finite nonzero temperature.… Expand

Bayesian Computation and Stochastic Systems

- Mathematics
- 1995

Markov chain Monte Carlo (MCMC) methods have been used extensively in statistical physics over the last 40 years, in spatial statistics for the past 20 and in Bayesian image analysis over the last… Expand

The theory of hybrid stochastic algorithms

- Mathematics
- 1990

These lectures introduce the family of Hybrid Stochastic Algorithms for performing Monte Carlo calculations in Quantum Field Theory. After explaining the basic concepts of Monte Carlo integration we… Expand

Markov Chain Monte Carlo Maximum Likelihood

- Mathematics
- 1991

Markov chain Monte Carlo (e. g., the Metropolis algorithm and Gibbs sampler) is a general tool for simulation of complex stochastic processes useful in many types of statistical inference. The basics… Expand

Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus

- Mathematics
- 1987

The use of the Gibbs sampler for Bayesian computation is reviewed and illustrated in the context of some canonical examples. Other Markov chain Monte Carlo simulation methods are also briefly… Expand

Optimization by Simulated Annealing

- Medicine, Computer Science
- Science
- 1983

A detailed analogy with annealing in solids provides a framework for optimization of the properties of very large and complex systems. Expand

New approach to spin-glass simulations.

- Physics, Medicine
- Physical review letters
- 1992

A recursive procedure is presented to calculate the parameters of the recently introduced multicanonical ensemble and the appraoch for spin glasses and evidence that the large L increase of the ergodicity time is greatly improved is provided. Expand

Hybrid Monte Carlo

- Physics
- 1987

We present a new method for the numerical simulation of lattice field theory. A hybrid (molecular dynamics/Langevin) algorithm is used to guide a Monte Carlo simulation. There are no discretization… Expand

Annealing Markov chain Monte Carlo

- 1994