The ability to predict how an environment changes based on forces applied to it is fundamental for a robot to achieve specific goals. Traditionally in robotics, this problem is addressed through the use of pre-specified models or physics simulators, taking advantage of prior knowledge of the problem structure. While these models are general and have broad applicability, they depend on accurate estimation of model parameters such as object shape, mass, friction etc. On the other hand, learning based methods such as Predictive State Representations or more recent deep learning approaches have looked at learning these models directly from raw perceptual information in a model-free manner. These methods operate on raw data without any intermediate parameter estimation, but lack the structure and generality of model-based techniques. In this talk, I will present some work that tries to bridge the gap between these two paradigms by proposing a specific class of deep visual dynamics models (SE3-Nets) that explicitly encode strong physical and 3D geometric priors (specifically, rigid body dynamics) in their structure. As opposed to traditional deep models that reason about dynamics/motion a pixel level, we show that the physical priors implicit in our network architectures enable them to reason about dynamics at the object level - our network learns to identify objects in the scene and to predict rigid body rotation and translation per object. I will present results on applying our deep architectures to two specific problems: 1) Modeling scene dynamics where the task is to predict future depth observations given the current observation and an applied action and 2) Real-time visuomotor control of a Baxter manipulator based only on raw depth data. We show that: 1) Our proposed architectures significantly outperform baseline deep models on dynamics modelling and 2) Our architectures perform comparably or better than baseline models for visuomotor control while operating at camera rates (30Hz) and relying on far less information.
Organizers: Franzi Meier
Machine learning has become a popular application domain for modern optimization techniques, pushing its algorithmic frontier. The need for large scale optimization algorithms which can handle millions of dimensions or data points, typical for the big data era, have brought a resurgence of interest for first order algorithms, making us revisit the venerable stochastic gradient method [Robbins-Monro 1951] as well as the Frank-Wolfe algorithm [Frank-Wolfe 1956]. In this talk, I will review recent improvements on these algorithms which can exploit the structure of modern machine learning approaches. I will explain why the Frank-Wolfe algorithm has become so popular lately; and present a surprising tweak on the stochastic gradient method which yields a fast linear convergence rate. Motivating applications will include weakly supervised video analysis and structured prediction problems.
Organizers: Philipp Hennig
This talk will look at hardware-based means of assembling, controlling and driving systems at the smallest of scales, including those that can become autonomous. I will show that insights from physics, chemistry and material engineering can be used to permit the simplification and miniaturization of otherwise bulky systems and that this can give rise to new technologies. One of the technologies we have invented may also permit the development of new imaging devices.
In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, that required that annotated training data must be available for all tasks, I will talk about a new setting, in which for some tasks, potentially most of them, only unlabeled training data is available. Consequently, to solve all tasks, information must be transfered between tasks with labels and tasks without labels. Focussing on an instance-based transfer method I will consider two variants of this setting: when the set of labeled tasks is fixed, and when it can be actively selected by the learner. I will discuss a generalization bound that covers both scenarios and an algorithm, that follows from it, for making the choice of labeled tasks (in the active case) and for transferring information between the tasks in a principled way. I will also show results of some experiments that illustrate the effectiveness of the algorithm.
Organizers: Georg Martius
This talk draws three parallels between classical algebraic quadrature rules, that are exact for polynomials of low degree, and kernel (or Bayesian) quadrature rules: i) Computational efficiency. Construction of scalable multivariate algebraic quadrature rules is challenging whereas kernel quadrature necessitates solving a linear system of equations, quickly becoming computationally prohibitive. Fully symmetric sets and Smolyak sparse grids can be used to solve both problems. ii) Derivatives and optimal rules. Algebraic degree of a Gaussian quadrature rule cannot be improved by adding derivative evaluations of the integrand. This holds for optimal kernel quadrature rules in the sense that derivatives are of no help in minimising the worst-case error (or posterior integral variance). iii) Positivity of the weights. Essentially as a consequence of the preceding property, both the Gaussian and optimal kernel quadrature rules have positive weights (i.e., they are positive linear functionals).
Organizers: Alexandra Gessner
Standard methods of causal discovery take as input a statistical data set of measurements of well-defined causal variables. The goal is then to determine the causal relations among these variables. But how are these causal variables identified or constructed in the first place? Often we have sensor level data but assume that the relevant causal interactions occur at a higher scale of aggregation. Sometimes we only have aggregate measurements of causal interactions at a finer scale. I will motivate the general problem of causal discovery and present recent work on a framework and method for the construction and identification of causal macro-variables that ensures that the resulting causal variables have well-defined intervention distributions. Time permitting, I will show an application of this approach to large scale climate data, for which we were able to identify the macro-phenomenon of El Nino using an unsupervised method on micro-level measurements of the sea surface temperature and wind speeds over the equatorial Pacific.
Organizers: Sebastian Weichwald
This work investigates the development of the sense of agency and of object permanence in humanoid robots. Based on findings from developmental psychology and from neuroscience, development of sense of object permanence is linked to development of sense of agency and to processes of internal simulation of sensor activity. In the course of the work, two sets of experiments will be presented, in the first set a humanoid robot has to learn the forward relationship between its movements and their sensory consequences perceived from the visual input. In particular, a self-monitoring mechanism was implemented that allows the robot to distinguish between self-generated movements and those generated by external events. In a second experiment, once having learned this mapping, the self-monitoring mechanism is exploited to suppress the predicted visual consequences of intended movements. The speculation is made that this process can allow for the development of sense of object permanence. It will be shown, that using these predictions, the robot maintains an enhanced simulated image where an object occluded by the movement of the robot arm is still visible, due to sensory attenuation processes.
In robotics, it is often practically and theoretically convenient to design motion planners for approximate simple robot and environment models first, and then adapt such reference planners to more accurate complex settings. In this talk, I will introduce a new approach to extend the applicability of motion planners of simple settings to more complex settings using reference governors. Reference governors are add-on control schemes for closed-loop dynamical systems to enforce constraint satisfaction while maintaining stability, and offers a systematic way of separating the issues of stability and constraint enforcement. I will demonstrate example applications of reference governors for sensor-based navigation in environments cluttered with convex obstacles and for smooth extensions of low-order (e.g., position- or velocity-controlled) feedback motion planners to high-order (e.g., force/torque controlled) robot models, while retaining stability and collision avoidance properties.
Growth of the internet and social media has spurred the sharing and dissemination of personal data at large scale. At the same time, recent developments in computer vision has enabled unseen effectiveness and efficiency in automated recognition. It is clear that visual data contains private information that can be mined, yet the privacy implications of sharing such data have been less studied in computer vision community. In the talk, I will present some key results from our study of the implications of the development of computer vision on the identifiability in social media, and an analysis of existing and new anonymisation techniques. In particular, we show that adversarial image perturbations (AIP) introduce human invisible perturbations on the input image that effectively misleads a recogniser. They are far more aesthetic and effective compared to e.g. face blurring. The core limitation, however, is that AIPs are usually generated against specific target recogniser(s), and it is hard to guarantee the performance against uncertain, potentially adaptive recognisers. As a first step towards dealing with the uncertainty, we have introduced a game theoretical framework to obtain the user’s privacy guarantee independent of the randomly chosen recogniser (within some fixed set).
Organizers: Siyu Tang
In the recent years, commodity 3D sensors have become easily and widely available. These advances in sensing technology have spawned significant interest in using captured 3D data for mapping and semantic understanding of 3D environments. In this talk, I will give an overview of our latest research in the context of 3D reconstruction of indoor environments. I will further talk about the use of 3D data in the context of modern machine learning techniques. Specifically, I will highlight the importance of training data, and how can we efficiently obtain labeled and self-supervised ground truth training datasets from captured 3D content. Finally, I will show a selection of state-of-the-art deep learning approaches, including discriminative semantic labeling of 3D scenes and generative reconstruction techniques.
Organizers: Despoina Paschalidou
Autonomous systems rely on learning from experience to automatically refine their strategy and adapt to their environment, and thereby have huge advantages over traditional hand engineered systems. At PROWLER.io we use reinforcement learning (RL) for sequential decision making under uncertainty to develop intelligent agents capable of acting in dynamic and unknown environments. In this talk we first give a general overview of the goals and the research conducted at PROWLER.io. Then, we will talk about two specific research topics. The first is Information-Theoretic Model Uncertainty which deals with the problem of making robust decisions that take into account unspecified models of the environment. The second is Deep Model-Based Reinforcement Learning which deals with the problem of learning the transition and the reward function of a Markov Decision Process in order to use it for data-efficient learning.
Organizers: Michel Besserve