Incremental Gaussian Processes
2003
Conference Paper
ei
In this paper, we consider Tipping‘s relevance vector machine (RVM) and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call subspace EM. Working with a subset of active basis functions, the sparsity of the RVM solution will ensure that the number of basis functions and thereby the computational complexity is kept low. We also introduce a mean field approach to the intractable classification model that is expected to give a very good approximation to exact Bayesian inference and contains the Laplace approximation as a special case. We test the algorithms on two large data sets with O(10^3-10^4) examples. The results indicate that Bayesian learning of large data sets, e.g. the MNIST database is realistic.
Author(s): | Quinonero Candela, J. and Winther, O. |
Book Title: | Advances in Neural Information Processing Systems 15 |
Journal: | Advances in Neural Information Processing Systems 15 |
Pages: | 1001-1008 |
Year: | 2003 |
Month: | October |
Day: | 0 |
Editors: | Becker, S. , S. Thrun, K. Obermayer |
Publisher: | MIT Press |
Department(s): | Empirical Inference |
Bibtex Type: | Conference Paper (inproceedings) |
Event Name: | Sixteenth Annual Conference on Neural Information Processing Systems (NIPS 2002) |
Event Place: | Vancouver, BC, Canada |
Address: | Cambridge, MA, USA |
Digital: | 0 |
Institution: | Informatics and Mathematical Modelling, Technical University of Denmark |
ISBN: | 0-262-02550-7 |
Organization: | Max-Planck-Gesellschaft |
School: | Biologische Kybernetik |
Links: |
PDF
Web |
BibTex @inproceedings{2800, title = {Incremental Gaussian Processes}, author = {Quinonero Candela, J. and Winther, O.}, journal = {Advances in Neural Information Processing Systems 15}, booktitle = {Advances in Neural Information Processing Systems 15}, pages = {1001-1008}, editors = {Becker, S. , S. Thrun, K. Obermayer}, publisher = {MIT Press}, organization = {Max-Planck-Gesellschaft}, institution = {Informatics and Mathematical Modelling, Technical University of Denmark}, school = {Biologische Kybernetik}, address = {Cambridge, MA, USA}, month = oct, year = {2003}, doi = {}, month_numeric = {10} } |