Header logo is


2019


no image
Limitations of the empirical Fisher approximation for natural gradient descent

Kunstner, F., Hennig, P., Balles, L.

Advances in Neural Information Processing Systems 32, pages: 4158-4169, (Editors: H. Wallach and H. Larochelle and A. Beygelzimer and F. d’Alché-Buc and E. Fox and R. Garnett), Curran Associates, Inc., 33rd Annual Conference on Neural Information Processing Systems, December 2019 (conference)

ei pn

link (url) [BibTex]

2019


link (url) [BibTex]


no image
Convergence Guarantees for Adaptive Bayesian Quadrature Methods

Kanagawa, M., Hennig, P.

Advances in Neural Information Processing Systems 32, pages: 6234-6245, (Editors: H. Wallach and H. Larochelle and A. Beygelzimer and F. d’Alché-Buc and E. Fox and R. Garnett), Curran Associates, Inc., 33rd Annual Conference on Neural Information Processing Systems, December 2019 (conference)

ei pn

link (url) [BibTex]

link (url) [BibTex]


no image
DeepOBS: A Deep Learning Optimizer Benchmark Suite

Schneider, F., Balles, L., Hennig, P.

7th International Conference on Learning Representations (ICLR), May 2019 (conference)

ei pn

link (url) [BibTex]

link (url) [BibTex]


no image
Fast and Robust Shortest Paths on Manifolds Learned from Data

Arvanitidis, G., Hauberg, S., Hennig, P., Schober, M.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1506-1515, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

ei pn

PDF link (url) [BibTex]

PDF link (url) [BibTex]


Active Probabilistic Inference on Matrices for Pre-Conditioning in Stochastic Optimization
Active Probabilistic Inference on Matrices for Pre-Conditioning in Stochastic Optimization

de Roos, F., Hennig, P.

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, pages: 1448-1457, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (conference)

Abstract
Pre-conditioning is a well-known concept that can significantly improve the convergence of optimization algorithms. For noise-free problems, where good pre-conditioners are not known a priori, iterative linear algebra methods offer one way to efficiently construct them. For the stochastic optimization problems that dominate contemporary machine learning, however, this approach is not readily available. We propose an iterative algorithm inspired by classic iterative linear solvers that uses a probabilistic model to actively infer a pre-conditioner in situations where Hessian-projections can only be constructed with strong Gaussian noise. The algorithm is empirically demonstrated to efficiently construct effective pre-conditioners for stochastic gradient descent and its variants. Experiments on problems of comparably low dimensionality show improved convergence. In very high-dimensional problems, such as those encountered in deep learning, the pre-conditioner effectively becomes an automatic learning-rate adaptation scheme, which we also empirically show to work well.

ei pn

PDF link (url) [BibTex]

PDF link (url) [BibTex]


no image
Ferromagnetic colloids in liquid crystal solvents

Zarubin, G.

Universität Stuttgart, Stuttgart, 2019 (phdthesis)

icm

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Fluctuating interface with a pinning potential

Pranjić, Daniel

Universität Stuttgart, Stuttgart, 2019 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Controlling pattern formation in the confined Schnakenberg model

Beyer, David Bernhard

Universität Stuttgart, Stuttgart, 2019 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Interfaces in fluids of ionic liquid crystals

Bartsch, H.

Universität Stuttgart, Stuttgart, 2019 (phdthesis)

icm

link (url) DOI [BibTex]

link (url) DOI [BibTex]

2017


On the Design of {LQR} Kernels for Efficient Controller Learning
On the Design of LQR Kernels for Efficient Controller Learning

Marco, A., Hennig, P., Schaal, S., Trimpe, S.

Proceedings of the 56th IEEE Annual Conference on Decision and Control (CDC), pages: 5193-5200, IEEE, IEEE Conference on Decision and Control, December 2017 (conference)

Abstract
Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic description of the latent objective function, typically a Gaussian process (GP). As is shown herein, GPs with a common kernel choice can, however, lead to poor learning outcomes on standard quadratic control problems. For a first-order system, we construct two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning. Simulations of uncertain linear and nonlinear systems demonstrate that the LQR kernels yield superior learning performance.

am ics pn

arXiv PDF On the Design of LQR Kernels for Efficient Controller Learning - CDC presentation DOI Project Page [BibTex]

2017


arXiv PDF On the Design of LQR Kernels for Efficient Controller Learning - CDC presentation DOI Project Page [BibTex]


Coupling Adaptive Batch Sizes with Learning Rates
Coupling Adaptive Batch Sizes with Learning Rates

Balles, L., Romero, J., Hennig, P.

In Proceedings Conference on Uncertainty in Artificial Intelligence (UAI) 2017, pages: 410-419, (Editors: Gal Elidan and Kristian Kersting), Association for Uncertainty in Artificial Intelligence (AUAI), Conference on Uncertainty in Artificial Intelligence (UAI), August 2017 (inproceedings)

Abstract
Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These methods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behavior of the stochastic optimization algorithm, though, since it determines the variance of the gradient estimates. This variance also changes over the optimization process; when using a constant batch size, stability and convergence is thus often enforced by means of a (manually tuned) decreasing learning rate schedule. We propose a practical method for dynamic batch size adaptation. It estimates the variance of the stochastic gradients and adapts the batch size to decrease the variance proportionally to the value of the objective function, removing the need for the aforementioned learning rate decrease. In contrast to recent related work, our algorithm couples the batch size to the learning rate, directly reflecting the known relationship between the two. On three image classification benchmarks, our batch size adaptation yields faster optimization convergence, while simultaneously simplifying learning rate tuning. A TensorFlow implementation is available.

ps pn

Code link (url) Project Page [BibTex]

Code link (url) Project Page [BibTex]


no image
Dynamic Time-of-Flight

Schober, M., Adam, A., Yair, O., Mazor, S., Nowozin, S.

Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pages: 170-179, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (conference)

ei pn

DOI [BibTex]

DOI [BibTex]


Virtual vs. {R}eal: Trading Off Simulations and Physical Experiments in Reinforcement Learning with {B}ayesian Optimization
Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization

Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A. P., Krause, A., Schaal, S., Trimpe, S.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages: 1557-1563, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

am ics pn

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]


Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.

Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 54, pages: 528-536, Proceedings of Machine Learning Research, (Editors: Sign, Aarti and Zhu, Jerry), PMLR, April 2017 (conference)

pn

pdf link (url) Project Page [BibTex]

pdf link (url) Project Page [BibTex]


no image
Stationary and time-dependent heat transfer in paradigmatic many-body geometries

Asheichyk, Kiryl

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Non-equilibrium forces after temperature quenches in ideal fluids with conserved density

Hölzl, Christian

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Numerical studies of active colloids at fluid interfaces

Peter, Toni

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
New Directions for Learning with Kernels and Gaussian Processes (Dagstuhl Seminar 16481)

Gretton, A., Hennig, P., Rasmussen, C., Schölkopf, B.

Dagstuhl Reports, 6(11):142-167, 2017 (book)

ei pn

DOI [BibTex]

DOI [BibTex]


no image
Collective dynamics of laterally confined active particles near fluid-fluid interfaces

Kistner, Irina

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Self-diffusion of DNA grafted functional colloids in a crowded environment

Werner, M.

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Electrostatic interaction between non-identical charged particles at an electrolyte interface

Schmetzer, Timo

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Microscopic investigation of the Marangoni effect

Pöhnl, Matthias

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Interfacial structure of a catalytic surface

Lipp, Melanie

Universität Stuttgart, Stuttgart, 2017 (mastersthesis)

icm

[BibTex]

[BibTex]

2007


no image
Phase behavior and effective interactions in colloidal suspensions

Grodon, C.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

icm

link (url) [BibTex]

2007


link (url) [BibTex]


no image
The effect of capillary forces on adhesion of biological and artificial attachment devices

De Souza, E. J.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

icm

link (url) [BibTex]

link (url) [BibTex]


no image
Brownian dynamics in near surface flows

Almenar Egea, L.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Single lamella crystals of polyethylene accessible by catalytic emulsion-polymerization

Weber, C. H. M., Chiche, A., Krausch, G., Rosenfeldt, S., Ballauff, M., Goettker-Schnetmann, I., Tong, Q., Mecking, S., Harnau, L.

In Polymeric Materials: Science \& Engineering, 97, pages: 133-134, 2007 (inproceedings)

icm

[BibTex]

[BibTex]

2003


no image
Coexisting Phases in Binary Platelet Mixtures

Bier, M.

Universität Stuttgart, Stuttgart, 2003 (mastersthesis)

icm

[BibTex]

2003


[BibTex]


no image
Capillary forces between structured substrates

De Souza, E. J.

Universität Stuttgart, Stuttgart, 2003 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Statistical physics of stochastic geometries

Brodatzki, U.

Universität Wuppertal, Wuppertal, 2003 (phdthesis)

icm

[BibTex]

[BibTex]


no image
Colloidal Particles in Critical Fluids

Schlesener, F.

Universität Stuttgart, Stuttgart, 2003 (phdthesis)

icm

[BibTex]

[BibTex]


no image
Structure and Solvation Forces in Binary Hard-Sphere Mixtures

Grodon, C.

Universität Stuttgart, Stuttgart, 2003 (mastersthesis)

icm

[BibTex]

[BibTex]

2000


no image
Selbst- und Fremddiffusion in amorphem Si28C36N36 und Si3N4

Matics, S.

Universität Stuttgart, Stuttgart, 2000 (phdthesis)

icm

link (url) [BibTex]

2000


link (url) [BibTex]