Header logo is

Bayesian Inference and Optimal Design for the Sparse Linear Model

2008

Article

ei


The linear model with sparsity-favouring prior on the coefficients has important applications in many different domains. In machine learning, most methods to date search for maximum a posteriori sparse solutions and neglect to represent posterior uncertainties. In this paper, we address problems of Bayesian optimal design (or experiment planning), for which accurate estimates of uncertainty are essential. To this end, we employ expectation propagation approximate inference for the linear model with Laplace prior, giving new insight into numerical stability properties and proposing a robust algorithm. We also show how to estimate model hyperparameters by empirical Bayesian maximisation of the marginal likelihood, and propose ideas in order to scale up the method to very large underdetermined problems. We demonstrate the versatility of our framework on the application of gene regulatory network identification from micro-array expression data, where both the Laplace prior and the active experimental design approach are shown to result in significant improvements. We also address the problem of sparse coding of natural images, and show how our framework can be used for compressive sensing tasks.

Author(s): Seeger, MW.
Journal: Journal of Machine Learning Research
Volume: 9
Pages: 759-813
Year: 2008
Month: April
Day: 0

Department(s): Empirische Inferenz
Bibtex Type: Article (article)

Digital: 0
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF
PDF

BibTex

@article{5160,
  title = {Bayesian Inference and Optimal Design for the Sparse Linear Model},
  author = {Seeger, MW.},
  journal = {Journal of Machine Learning Research},
  volume = {9},
  pages = {759-813},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  month = apr,
  year = {2008},
  doi = {},
  month_numeric = {4}
}