Inference algorithms and learning theory for Bayesian sparse factor analysis
2009
Article
ei
Bayesian sparse factor analysis has many applications; for example, it has been applied to the problem of inferring a sparse regulatory network from gene expression data. We describe a number of inference algorithms for Bayesian sparse factor analysis using a slab and spike mixture prior. These include well-established Markov chain Monte Carlo (MCMC) and variational Bayes (VB) algorithms as well as a novel hybrid of VB and Expectation Propagation (EP). For the case of a single latent factor we derive a theory for learning performance using the replica method. We compare the MCMC and VB/EP algorithm results with simulated data to the theoretical prediction. The results for MCMC agree closely with the theory as expected. Results for VB/EP are slightly sub-optimal but show that the new algorithm is effective for sparse inference. In large-scale problems MCMC is infeasible due to computational limitations and the VB/EP algorithm then provides a very useful computationally efficient alternative.
Author(s): | Rattray, M. and Stegle, O. and Sharp, K. and Winn, J. |
Book Title: | IW-SMI 2009 |
Journal: | Journal of Physics: Conference Series |
Volume: | 197 |
Number (issue): | 1: International Workshop on Statistical-Mechanical Informatics 2009 |
Pages: | 1-10 |
Year: | 2009 |
Month: | September |
Day: | 0 |
Editors: | Inoue, M. , S. Ishii, Y. Kabashima, M. Okada |
Publisher: | Institute of Physics |
Department(s): | Empirical Inference |
Bibtex Type: | Article (article) |
Address: | Bristol, UK |
Digital: | 0 |
DOI: | 10.1088/1742-6596/197/1/012002 |
Event Name: | International Workshop on Statistical-Mechanical Informatics 2009 (IW-SMI 2009) |
Event Place: | Kyoto, Japan |
Language: | en |
Organization: | Max-Planck-Gesellschaft |
School: | Biologische Kybernetik |
Links: |
PDF
Web |
BibTex @article{6296, title = {Inference algorithms and learning theory for Bayesian sparse factor analysis}, author = {Rattray, M. and Stegle, O. and Sharp, K. and Winn, J.}, journal = {Journal of Physics: Conference Series }, booktitle = {IW-SMI 2009}, volume = {197}, number = {1: International Workshop on Statistical-Mechanical Informatics 2009}, pages = {1-10}, editors = {Inoue, M. , S. Ishii, Y. Kabashima, M. Okada}, publisher = {Institute of Physics}, organization = {Max-Planck-Gesellschaft}, school = {Biologische Kybernetik}, address = {Bristol, UK}, month = sep, year = {2009}, doi = {10.1088/1742-6596/197/1/012002}, month_numeric = {9} } |