Statistical Learning Theory: Models, Concepts, and Results


Book Chapter


Statistical learning theory provides the theoretical basis for many of today's machine learning algorithms and is arguably one of the most beautifully developed branches of artificial intelligence in general. It originated in Russia in the 1960s and gained wide popularity in the 1990s following the development of the so-called Support Vector Machine (SVM), which has become a standard tool for pattern recognition in a variety of domains ranging from computer vision to computational biology. Providing the basis of new learning algorithms, however, was not the only motivation for developing statistical learning theory. It was just as much a philosophical one, attempting to answer the question of what it is that allows us to draw valid conclusions from empirical data. In this article we attempt to give a gentle, non-technical overview over the key ideas and insights of statistical learning theory. We do not assume that the reader has a deep background in mathematics, statistics, or computer science. Given the nature of the subject matter, however, some familiarity with mathematical concepts and notations and some intuitive understanding of basic probability is required. There exist many excellent references to more technical surveys of the mathematics of statistical learning theory: the monographs by one of the founders of statistical learning theory ([Vapnik, 1995], [Vapnik, 1998]), a brief overview over statistical learning theory in Section 5 of [Sch{\"o}lkopf and Smola, 2002], more technical overview papers such as [Bousquet et al., 2003], [Mendelson, 2003], [Boucheron et al., 2005], [Herbrich and Williamson, 2002], and the monograph [Devroye et al., 1996].

Author(s): von Luxburg, U. and Schölkopf, B.
Book Title: Handbook of the History of Logic, Vol. 10: Inductive Logic
Volume: 10
Pages: 651-706
Year: 2011
Month: May
Day: 0
Editors: Gabbay, D. M., Hartmann, S. and Woods, J. H.
Publisher: Elsevier North Holland

Department(s): Empirical Inference
Bibtex Type: Book Chapter (inbook)

Address: Amsterdam, Netherlands
DOI: 10.1016/B978-0-444-52936-7.50016-1
ISBN: 978-0-444-52936-7
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF


  title = {Statistical Learning Theory: Models, Concepts, and Results},
  author = {von Luxburg, U. and Sch{\"o}lkopf, B.},
  booktitle = {Handbook of the History of Logic, Vol. 10: Inductive Logic},
  volume = {10},
  pages = {651-706},
  editors = {Gabbay, D. M., Hartmann, S. and Woods, J. H.},
  publisher = {Elsevier North Holland},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Amsterdam, Netherlands},
  month = may,
  year = {2011},
  month_numeric = {5}