Header logo is


2018


no image
Kernel Recursive ABC: Point Estimation with Intractable Likelihood

Kajihara, T., Kanagawa, M., Yamazaki, K., Fukumizu, K.

Proceedings of the 35th International Conference on Machine Learning, pages: 2405-2414, PMLR, July 2018 (conference)

Abstract
We propose a novel approach to parameter estimation for simulator-based statistical models with intractable likelihood. Our proposed method involves recursive application of kernel ABC and kernel herding to the same observed data. We provide a theoretical explanation regarding why the approach works, showing (for the population setting) that, under a certain assumption, point estimates obtained with this method converge to the true parameter, as recursion proceeds. We have conducted a variety of numerical experiments, including parameter estimation for a real-world pedestrian flow simulator, and show that in most cases our method outperforms existing approaches.

pn

Paper [BibTex]

2018


Paper [BibTex]


Thumb xl tease pic
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients

Balles, L., Hennig, P.

In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018 (inproceedings) Accepted

Abstract
The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a combination of two aspects: for each weight, the update direction is determined by the sign of stochastic gradients, whereas the update magnitude is determined by an estimate of their relative variance. We disentangle these two aspects and analyze them in isolation, gaining insight into the mechanisms underlying ADAM. This analysis also extends recent results on adverse effects of ADAM on generalization, isolating the sign aspect as the problematic one. Transferring the variance adaptation to SGD gives rise to a novel method, completing the practitioner's toolbox for problems where ADAM fails.

pn

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


no image
Direct observations of sub-100 nm spin wave propagation in magnonic wave-guides

Träger, N., Gruszecki, P., Lisiecki, F., Förster, J., Weigand, M., Kuswik, P., Dubowik, J., Schütz, G., Krawczyk, M., Gräfe, J.

In 2018 IEEE International Magnetics Conference (INTERMAG 2018), IEEE, Singapore, 2018 (inproceedings)

mms

DOI [BibTex]

DOI [BibTex]


no image
Interpreting FORC diagrams beyond the Preisach model: an experimental permalloy micro array investigation

Gross, F., Ilse, S., Schütz, G., Gräfe, J., Goering, E.

In 2018 IEEE International Magnetics Conference (INTERMAG 2018), IEEE, Singapore, 2018 (inproceedings)

mms

DOI [BibTex]

DOI [BibTex]

2017


Thumb xl fig toyex lqr1kernel 1
On the Design of LQR Kernels for Efficient Controller Learning

Marco, A., Hennig, P., Schaal, S., Trimpe, S.

Proceedings of the 56th IEEE Annual Conference on Decision and Control (CDC), pages: 5193-5200, IEEE, IEEE Conference on Decision and Control, December 2017 (conference)

Abstract
Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic description of the latent objective function, typically a Gaussian process (GP). As is shown herein, GPs with a common kernel choice can, however, lead to poor learning outcomes on standard quadratic control problems. For a first-order system, we construct two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning. Simulations of uncertain linear and nonlinear systems demonstrate that the LQR kernels yield superior learning performance.

am ics pn

arXiv PDF On the Design of LQR Kernels for Efficient Controller Learning - CDC presentation DOI Project Page [BibTex]

2017


arXiv PDF On the Design of LQR Kernels for Efficient Controller Learning - CDC presentation DOI Project Page [BibTex]


Thumb xl teaser
Coupling Adaptive Batch Sizes with Learning Rates

Balles, L., Romero, J., Hennig, P.

In Proceedings Conference on Uncertainty in Artificial Intelligence (UAI) 2017, pages: 410-419, (Editors: Gal Elidan and Kristian Kersting), Association for Uncertainty in Artificial Intelligence (AUAI), Conference on Uncertainty in Artificial Intelligence (UAI), August 2017 (inproceedings)

Abstract
Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These methods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behavior of the stochastic optimization algorithm, though, since it determines the variance of the gradient estimates. This variance also changes over the optimization process; when using a constant batch size, stability and convergence is thus often enforced by means of a (manually tuned) decreasing learning rate schedule. We propose a practical method for dynamic batch size adaptation. It estimates the variance of the stochastic gradients and adapts the batch size to decrease the variance proportionally to the value of the objective function, removing the need for the aforementioned learning rate decrease. In contrast to recent related work, our algorithm couples the batch size to the learning rate, directly reflecting the known relationship between the two. On three image classification benchmarks, our batch size adaptation yields faster optimization convergence, while simultaneously simplifying learning rate tuning. A TensorFlow implementation is available.

ps pn

Code link (url) Project Page [BibTex]

Code link (url) Project Page [BibTex]


no image
Dynamic Time-of-Flight

Schober, M., Adam, A., Yair, O., Mazor, S., Nowozin, S.

Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pages: 170-179, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (conference)

ei pn

DOI [BibTex]

DOI [BibTex]


Thumb xl this one
Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization

Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A. P., Krause, A., Schaal, S., Trimpe, S.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages: 1557-1563, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

am ics pn

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]

PDF arXiv ICRA 2017 Spotlight presentation Virtual vs. Real - Video explanation DOI Project Page [BibTex]


Thumb xl screen shot 2017 07 20 at 12.31.00 pm
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.

Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 54, pages: 528-536, Proceedings of Machine Learning Research, (Editors: Sign, Aarti and Zhu, Jerry), PMLR, April 2017 (conference)

pn

pdf link (url) Project Page [BibTex]

pdf link (url) Project Page [BibTex]

2012


Thumb xl thumb hennigk2012
Quasi-Newton Methods: A New Direction

Hennig, P., Kiefel, M.

In Proceedings of the 29th International Conference on Machine Learning, pages: 25-32, ICML ’12, (Editors: John Langford and Joelle Pineau), Omnipress, New York, NY, USA, ICML, July 2012 (inproceedings)

Abstract
Four decades after their invention, quasi- Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression under varying prior assumptions. This new notion elucidates some shortcomings of classical algorithms, and lights the way to a novel nonparametric quasi-Newton method, which is able to make more efficient use of available information at computational cost similar to its predecessors.

ei ps pn

website+code pdf link (url) [BibTex]

2012


website+code pdf link (url) [BibTex]


no image
Learning Tracking Control with Forward Models

Bócsi, B., Hennig, P., Csató, L., Peters, J.

In pages: 259 -264, IEEE International Conference on Robotics and Automation (ICRA), May 2012 (inproceedings)

Abstract
Performing task-space tracking control on redundant robot manipulators is a difficult problem. When the physical model of the robot is too complex or not available, standard methods fail and machine learning algorithms can have advantages. We propose an adaptive learning algorithm for tracking control of underactuated or non-rigid robots where the physical model of the robot is unavailable. The control method is based on the fact that forward models are relatively straightforward to learn and local inversions can be obtained via local optimization. We use sparse online Gaussian process inference to obtain a flexible probabilistic forward model and second order optimization to find the inverse mapping. Physical experiments indicate that this approach can outperform state-of-the-art tracking control algorithms in this context.

ei pn

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Approximate Gaussian Integration using Expectation Propagation

Cunningham, J., Hennig, P., Lacoste-Julien, S.

In pages: 1-11, -, January 2012 (inproceedings) Submitted

Abstract
While Gaussian probability densities are omnipresent in applied mathematics, Gaussian cumulative probabilities are hard to calculate in any but the univariate case. We offer here an empirical study of the utility of Expectation Propagation (EP) as an approximate integration method for this problem. For rectangular integration regions, the approximation is highly accurate. We also extend the derivations to the more general case of polyhedral integration regions. However, we find that in this polyhedral case, EP's answer, though often accurate, can be almost arbitrarily wrong. These unexpected results elucidate an interesting and non-obvious feature of EP not yet studied in detail, both for the problem of Gaussian probabilities and for EP more generally.

ei pn

Web [BibTex]

Web [BibTex]


no image
Kernel Topic Models

Hennig, P., Stern, D., Herbrich, R., Graepel, T.

In Fifteenth International Conference on Artificial Intelligence and Statistics, 22, pages: 511-519, JMLR Proceedings, (Editors: Lawrence, N. D. and Girolami, M.), JMLR.org, AISTATS , 2012 (inproceedings)

Abstract
Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.

ei pn

PDF Web [BibTex]

PDF Web [BibTex]


no image
Spin wave mediated magnetic vortex core reversal

Stoll, H.

In 8461, San Diego, California, USA, 2012 (inproceedings)

mms

DOI [BibTex]

DOI [BibTex]

2001


no image
Computational micromagnetism of magnetic structures and magnetization processes in thin plantelets and small particles

Kronmüller, H., Hertel, R.

In Magnetic Storage Sstems Beyond 2000, 41, pages: 345-362, Nato Science Series II: Mathematics, Physics and Chemistry, Kluwer Academic Publishers, Rhodos, Greece, 2001 (inproceedings)

mms

[BibTex]

2001


[BibTex]


no image
Hydrogen storage in mechanically treated single wall carbon nanotrubes

Haluska, M., Hulman, M., Hirscher, M., Becher, M., Roth, S., Stepanek, I., Bernier, P.

In Electronic Properties of Molecular Nanostructures: XV International Winterschool/Euroconference, 591, pages: 603-608, American Institute of Physics Conference Proceedings, AIP, Kirchberg [Austria], 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Isotopic mass and lattice constant of Si and Ge: X-Ray standing wave measurements

Zegenhagen, J., Kazimirov, A., Cao, L. X., Konuma, M., Sozontov, E., Plachke, D., Carstanjen, H. D., Bilger, G., Haller, E., Kohn, V., Cardona, M.

In Proceedings of the 25th Conference on the Physics of Semiconductors, 87, pages: 125-127, Springer proceedings in physics, Springer, Osaka, Japan, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Positron Annihilation Studies on Stable and Undercooled Metal Melts at the Stuttgart Pelletron

Stoll, H., Siegle, A., Major, J.

In Application of Accelerators in Research and Industry, 576, pages: 749-752, AIP Conference Proceedings, Denton, Texas, USA, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Submicrometer spatially resolved measurements of mechanical properties and correlation to microstructure and composition

Kunert, M., Baretzky, B., Baker, S. P., Mittemeijer, E. J.

In Fundamentals of Nanoindentation and Nanotribology II, 649, pages: Q3.2.1-Q3.2.6, Materials Research Society Symposium Proceedings, MRS, Boston, MA, USA, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
The six-jump diffusion cycles in B2-compounds

Drautz, R., Meyer, B., Fähnle, M.

In Proceedings of DIMAT 2000, the Fifth International Conference on Diffusion in Materials, pages: 417-422, Defect and Diffusion Forum, Scitec Publications Ltd., Paris, France, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Ionic nitriding of austenitic and ferritic steel with the aid of a high aperture hall current accelerator

Straumal, B. B., Vershinin, N. F., Friesel, M., Ishenko, S. A., Gust, W.

In Diffusion in Materials DIMAT2000, 194, pages: 1457-1462, Defect and Diffusion Forum, Trans Tech, Paris, France, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
First proof of slow trapping of positronium in polymers by an Age-Momentum-Correlation (AMOC) experiment

Dauwe, C., Balcaen, N., van Waeyenberge, B., van Petegem, S., Stoll, H.

In Positron Annihilation. Proceedings of the 12th International Conference on Positron Annihilation, 363/365, pages: 254-256, Materials Science Forum, Trans Tech Publications Ltd., München, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Positron-age-momentum correlation

Stoll, H., Bandzuch, P., Siegle, A.

In Positron Annihilation: Proceedings of the 12th International Conference on Positron Annihilation, 363-365, pages: 547-551, Materials Science Forum, Trans Tech Publications Ltd., München, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Nanocrystalline and nanostructured high-performance permanent magnets

Goll, D., Hadjipanayis, G. C., Kronmüller, H.

In Applications of Ferromagnetic and Optical Materials, Storage and Magnetoelectronics, 674, pages: U2.4.1-U2.4.12, Materials Research Society Symposium Proceedings, MRS, San Francisco, Calif., 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Ion beam analysis with monolayer depth resolution using the electrostatic spectrometer at the MPI Stuttgart

Plachke, D., Blohm, G., Fischer, T., Khellaf, A., Kruse, O., Stoll, H., Carstanjen, H. D.

In Proceedings of the 16th International Conference on Applications of Accelerators in Research and Industry, 576, pages: 458-462, American Institute of Physics Conference Proceedings, AIP, Denton, Texas, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
From the electronic structure to the macroscopiy behavior: A multi-scale analysis of plasticity in intermetallic compounds

Fähnle, M., Kohlhammer, S., Bester, G.

In Influences of Interface and Dislocation Behavior on Microstructure Evolution, 652, pages: Y4.5.1.-Y4.5.12, Materials Research Society Symposium Proceedings, MRS, Boston, Mass., USA, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Influence of the microstructure on the magnetic properties of giant-magnetostrictive TbDyFe films

Hirscher, M., Winzek, B., Fischer, S. F., Kronmüller, H.

In Smart Materials. Proceedings of the 1st Caesarium, pages: 23-37, Springer, Bonn, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Materials analysis with monolayer depth resolution using MeV ion beams

Carstanjen, H. D.

In 117, Las Vegas, USA, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Flux-line pinning in low-angle grain boundaries.

Albrecht, J., Leonhardt, S., Kronmüller, H.

In Proceedings 10th International Workshop on Critical Currents (IWCC 2001), pages: 41-43, Göttingen, Germany, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Measurement of the low-temperature self-diffusivity of lithium by elastic recoil detection analysis

Wieland, O., Carstanjen, H. D.

In Proceedings of DIMAT 2000, the Fifth International Conference on Diffusion in Materials, 194/199, pages: 35-41, Defect and Diffusion Forum, Scitec Publications Ltd., Paris, France, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
From the electronic structure to the macroscopic behaviour: a multi-scale analysis of plasticity in intermetallic compounds

Fähnle, M., Kohlhammer, S., Bester, G.

In Influences of Interface and Dislocation Behavior on Microstructure Evolution, 652, pages: Y.4.5.1-Y.4.5.12, Materials Research Society Symposium Proceedings, MRS, Boston, Mass., 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Enhancement of the critical current density of YBa2Cu3O7-8-films by substracte irradiation

Leonhardt, S., Albrecht, J., Warthmann, R., Kronmüller, H.

In High-Tc Superconductors and Related Applications: Materials Science, Fundamental Properties, and Some Future Electronic Applications. Proceedings of the NATO Advanced Study Institute, 86, pages: 529-534, NATO Science Series 3. High Technology, Kluwer Academic Publishers, Albena, Bulgaria, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
AMOC studies of positronium in fine MgO powder

van Waeyenberge, B., Dauwe, C., Stoll, H.

In Positron Annihilation. Proceedings of the 12th International Conference on Positron Annihilation, 363/365, pages: 401-403, Materials Science Forum, Trans Tech Publications Ltd., München, 2001 (inproceedings)

mms

[BibTex]

[BibTex]


no image
Atomic defects and electronic structure of B2-FeAl, CoAl and NiAl

Fähnle, M., Meyer, B., Bester, G., Majer, J., Börnsen, N.

In Proceedings of DIMAT 2000, the Fifth International Conference on Diffusion in Materials, 194/199, pages: 279-285, Defect and Diffusion Forum, Scitec Publications Ltd., Paris, France, 2001 (inproceedings)

mms

[BibTex]

[BibTex]