Header logo is

Evaluation of marginal likelihoods via the density of states

2012

Conference Paper

ei


Bayesian model comparison involves the evaluation of the marginal likelihood, the expectation of the likelihood under the prior distribution. Typically, this high-dimensional integral over all model parameters is approximated using Markov chain Monte Carlo methods. Thermodynamic integration is a popular method to estimate the marginal likelihood by using samples from annealed posteriors. Here we show that there exists a robust and flexible alternative. The new method estimates the density of states, which counts the number of states associated with a particular value of the likelihood. If the density of states is known, computation of the marginal likelihood reduces to a one- dimensional integral. We outline a maximum likelihood procedure to estimate the density of states from annealed posterior samples. We apply our method to various likelihoods and show that it is superior to thermodynamic integration in that it is more flexible with regard to the annealing schedule and the family of bridging distributions. Finally, we discuss the relation of our method with Skilling's nested sampling.

Author(s): Habeck, M.
Book Title: Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2012)
Journal: Journal of Machine Learning Research
Volume: 22
Pages: 486--494
Year: 2012
Day: 0
Editors: N Lawrence and M Girolami
Publisher: JMLR: W\&CP 22

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: AISTATS 2012
Event Place: La Palma, Canary Islands

Digital: 0

Links: PDF

BibTex

@inproceedings{Habeck2012_3,
  title = {Evaluation of marginal likelihoods via the density of states},
  author = {Habeck, M.},
  journal = {Journal of Machine Learning Research},
  booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2012)
  },
  volume = {22},
  pages = {486--494},
  editors = {N Lawrence and M Girolami},
  publisher = {JMLR: W\&CP 22},
  year = {2012},
  doi = {}
}