Header logo is de

Efficient learning and feature detection in high dimensional regression




We present a novel algorithm for efficient learning and feature selection in high- dimensional regression problems. We arrive at this model through a modification of the standard regression model, enabling us to derive a probabilistic version of the well-known statistical regression technique of backfitting. Using the Expectation- Maximization algorithm, along with variational approximation methods to overcome intractability, we extend our algorithm to include automatic relevance detection of the input features. This Variational Bayesian Least Squares (VBLS) approach retains its simplicity as a linear model, but offers a novel statistically robust â??black- boxâ? approach to generalized linear regression with high-dimensional inputs. It can be easily extended to nonlinear regression and classification problems. In particular, we derive the framework of sparse Bayesian learning, e.g., the Relevance Vector Machine, with VBLS at its core, offering significant computational and robustness advantages for this class of methods. We evaluate our algorithm on synthetic and neurophysiological data sets, as well as on standard regression and classification benchmark data sets, comparing it with other competitive statistical approaches and demonstrating its suitability as a drop-in replacement for other generalized linear regression techniques.

Author(s): Ting, J. and D’Souza, A. and Vijayakumar, S. and Schaal, S.
Book Title: Neural Computation
Volume: 22
Pages: 831-886
Year: 2010

Department(s): Autonome Motorik
Bibtex Type: Article (article)

Cross Ref: p10408
Note: clmc
URL: http://www-clmc.usc.edu/publications/T/ting-NC2010.pdf


  title = {Efficient learning and feature detection in high dimensional regression},
  author = {Ting, J. and D'Souza, A. and Vijayakumar, S. and Schaal, S.},
  booktitle = {Neural Computation},
  volume = {22},
  pages = {831-886},
  year = {2010},
  note = {clmc},
  crossref = {p10408},
  url = {http://www-clmc.usc.edu/publications/T/ting-NC2010.pdf}