Institute Talks

Dynamic Scene Analysis Using CrowdCam Data

Talk
  • 24 May 2017 • 11:00 12:00
  • Yael Moses
  • Greenhouse (PS)

Dynamic events such as family gatherings, concerts or sports events are often photographed by a group of people. The set of still images obtained this way is rich in dynamic content. We consider the question of whether such a set of still images, rather the traditional video sequences, can be used for analyzing the dynamic content of the scene. This talk will describe several instances of this problem, their solutions and directions for future studies. In particular, we will present a method to extend epipolar geometry to predict location of a moving feature in CrowdCam images. The method assumes that the temporal order of the set of images, namely photo-sequencing, is given. We will briefly describe our method to compute photo-sequencing using geometric considerations and rank aggregation. We will also present a method for identifying the moving regions in a scene, which is a basic component in dynamic scene analysis. Finally, we will consider a new vision of developing collaborative CrowdCam, and a first step toward this goal.

Organizers: Jonas Wulff

Geometry of Neural Networks

Talk
  • 24 May 2017 • 2:30 3:30
  • Guido Montúfar
  • N4.022 (EI Dept. meeting room / 4th floor, north building)

Deep Learning is one of the most successful machine learning approaches to artificial intelligence. In this talk I discuss the geometry of neural networks as a way to study the success of Deep Learning at a mathematical level and to develop a theoretical basis for making further advances, especially in situations with limited amounts of data and challenging problems in reinforcement learning. I present a few recent results on the representational power of neural networks and then demonstrate how to align this with structures from perception-action problems in order to obtain more efficient learning systems.

Organizers: Jane Walters

The Perceptual Advantage of Symmetry for Scene Perception

Talk
  • 29 May 2017 • 14:00 15:00
  • Sven Dickinson
  • Green-House (PS)

Human observers can classify photographs of real-world scenes after only a very brief exposure to the image (Potter & Levy, 1969; Thorpe, Fize, Marlot, et al., 1996; VanRullen & Thorpe, 2001). Line drawings of natural scenes have been shown to capture essential structural information required for successful scene categorization (Walther et al., 2011). Here, we investigate how the spatial relationships between lines and line segments in the line drawings affect scene classification. In one experiment, we tested the effect of removing either the junctions or the middle segments between junctions. Surprisingly, participants performed better when shown the middle segments (47.5%) than when shown the junctions (42.2%). It appeared as if the images with middle segments tended to maintain the most parallel/locally symmetric portions of the contours. In order to test this hypothesis, in a second experiment, we either removed the most symmetric half of the contour pixels or the least symmetric half of the contour pixels using a novel method of measuring the local symmetry of each contour pixel in the image. Participants were much better at categorizing images containing the most symmetric contour pixels (49.7%) than the least symmetric (38.2%).  Thus, results from both experiments demonstrate that local contour symmetry is a crucial organizing principle in complex real-world scenes. Joint work with John Wilder (UofT CS, Psych), Morteza Rezanejad (McGill CS), Kaleem Siddiqi (McGill CS), Allan Jepson (UofT CS), and Dirk Bernhardt-Walther (UofT Psych), to be presented at VSS 2017.

Organizers: Ahmed Osman

Frederick Eberhardt - TBA

IS Colloquium
  • 03 July 2017 • 11:15 12:15
  • Frederick Eberhardt
  • Max Planck House Lecture Hall

Organizers: Sebastian Weichwald

  • Felix Berkenkamp
  • AMD Seminar Room (Paul-Ehrlich-Str. 15, 1rst floor)

Bayesian optimization is a powerful tool that has been successfully used to automatically optimize the parameters of a fixed control policy. It has many desirable properties, such as data-efficiently and being able to handle noisy measurements. However, standard Bayesian optimization does not consider any constraints imposed by the real system, which limits its applications to highly controlled environments. In this talk, I will introduce an extension of this framework, which additionally considers multiple safety constraints during the optimization process. This method enables safe parameter optimization by only evaluating parameters that fulfill all safety constraints with high probability. I will show several experiments on a quadrotor vehicle which demonstrate the method. Lastly, I will briefly talk about how the ideas behind safe Bayesian optimization can be used to safely explore unknown environments (MDPs).

Organizers: Sebastian Trimpe


  • Aldo Faisal
  • MPH Lecture Hall

Our research questions are centred on a basic characteristic of human brains: variability in their behaviour and their underlying meaning for cognitive mechanisms. Such variability is emerging as a key ingredient in understanding biological principles (Faisal, Selen & Wolpert, 2008, Nature Rev Neurosci) and yet lacks adequate quantitative and computational methods for description and analysis. Crucially, we find that biological and behavioural variability contains important information that our brain and our technology can make us of (instead of just averaging it away): Using advanced body sensor networks, we measured eye-movements, full-body and hand kinematics of humans living in a studio flat and are going to present some insightful results on motor control and visual attention that suggest that the control of behaviour "in-the-wild" is predictably different ways than what we measure "in-the-lab". The results have implications for robotics, prosthetics and neuroscience.

Organizers: Matthias Hohmann


Probabilistic Numerics for Differential Equations

IS Colloquium
  • 11 January 2016 • 11:15 12:15
  • Tim Sullivan

Beginning with a seminal paper of Diaconis (1988), the aim of so-called "probabilistic numerics" is to compute probabilistic solutions to deterministic problems arising in numerical analysis by casting them as statistical inference problems. For example, numerical integration of a deterministic function can be seen as the integration of an unknown/random function, with evaluations of the integrand at the integration nodes proving partial information about the integrand. Advantages offered by this viewpoint include: access to the Bayesian representation of prior and posterior uncertainties; better propagation of uncertainty through hierarchical systems than simple worst-case error bounds; and appropriate accounting for numerical truncation and round-off error in inverse problems, so that the replicability of deterministic simulations is not confused with their accuracy, thereby yielding an inappropriately concentrated Bayesian posterior. This talk will describe recent work on probabilistic numerical solvers for ordinary and partial differential equations, including their theoretical construction, convergence rates, and applications to forward and inverse problems. Joint work with Andrew Stuart (Warwick).

Organizers: Philipp Hennig


  • Jun Nakanishi
  • TTR, AMD Seminar Room (first floor)

Understanding the principles of natural movement generation has been and continues to be one of the most interesting and important open problems in the fields of robotics and neural control of movement. In this talk, I introduce an overview of our previous work on the control of dynamic movements in robotic systems towards the goal of control design principles and understanding of motion generation. Our research has focused in the fields of dynamical systems theory, adaptive and optimal control and statistical learning, and their application to robotics towards achieving dynamically dexterous behavior in robotic systems. First, our studies on dynamical systems based task encoding in robot brachiation, movement primitives for imitation learning, and oscillator based biped locomotion control will be presented. Then, our recent work on optimal control of robotic systems with variable stiffness actuation will be introduced towards the aim of achieving highly dynamic movements by exploiting the natural dynamics of the system. Finally, our new humanoid robot H-1 at TUM-ICS will be introduced.

Organizers: Ludovic Righetti


  • Alexander Sprowitz
  • TTR, AMD Seminar Room (first floor)

The current performance gap between legged animals and legged robots is large. Animals can reach high locomotion speed in complex terrain, or run at a low cost of transport. They are able to rapidly sense their environment, process sensor data, learn and plan locomotion strategies, and execute feedforward and feedback controlled locomotion patterns fluently on the fly. Animals use hardware that has, compared to the latest man-made actuators, electronics, and processors, relatively low bandwidth, medium power density, and low speed. The most common approach to legged robot locomotion is still assuming rigid linkage hardware, high torque actuators, and model based control algorithms with high bandwidth and high gain feedback mechanisms. State of the art robotic demonstrations such as the 2015 DARPA challenge showed that seemingly trivial locomotion tasks such as level walking, or walking over soft sand still stops most of our biped and quadruped robots. This talk focuses on an alternative class of legged robots and control algorithms designed and implemented on several quadruped and biped platforms, for a new generation of legged robotic systems. Biomechanical blueprints inspired by nature, and mechanisms from locomotion neurocontrol were designed, tested, and can be compared to their biological counterparts. We focus on hardware and controllers that allow comparably cheap robotics, in terms of computation, control, and mechanical complexity. Our goal are highly dynamic, robust legged systems with low weight and inertia, relatively low mechanical complexity and cost of transport, and little computational demands for standard locomotion tasks. Ideally, such system can also be used as testing platforms to explain not yet understood biomechanical and neurocontrol aspects of animals.

Organizers: Ludovic Righetti


  • Gernot Müller-Putz
  • MPH Lecture Hall

More than half of the persons with spinal cord injuries (SCI) are suffering from impairments of both hands, which results in a tremendous decrease of quality of life and represents a major barrier for inclusion in society. Functional restoration is possible with neuroprostheses (NPs) based on functional electrical stimulation (FES). A Brain-Computer Interface provides a means of control for such neuroprosthetics since users have limited abilities to use traditional assistive devices. This talk presents our early research on BCI-based NP control based on motor imagery, discusses hybrid BCI solutions and shows our work and effort on movement trajectory decoding. An outlook to future BCI applications will conclude this talk.

Organizers: Moritz Grosse-Wentrup


Making Robots Learn

IS Colloquium
  • 13 November 2015 • 11:30 12:30
  • Prof. Pieter Abbeel
  • Max Planck House Tübingen, Lecture Hall

Programming robots remains notoriously difficult. Equipping robots with the ability to learn would by-pass the need for what often ends up being time-consuming task specific programming. In this talk I will describe the ideas behind two promising types of robot learning: First I will discuss apprenticeship learning, in which robots learn from human demonstrations, and which has enabled autonomous helicopter aerobatics, knot tying, basic suturing, and cloth manipulation. Then I will discuss deep reinforcement learning, in which robots learn through their own trial and error, and which has enabled learning locomotion as well as a range of assembly and manipulation tasks.

Organizers: Stefan Schaal


Understanding Plants and Animals

Talk
  • 10 November 2015 • 11:00 12:00
  • Prof. David W. Jacobs
  • MRZ seminar room

I will describe a series of work that aims to automatically understand images of animals and plants. I will begin by describing recent work that uses Bounded Distortion matching to model pose variation in animals. Using a generic 3D model of an animal and multiple images of different individuals in various poses, we construct a model that captures the way in which the animal articulates. This is done by solving for the pose of the template that matches each image while simultaneously solving for the stiffness of each tetrahedron of the model. We minimize an L1 norm on stiffness, producing a model that bends easily at joints, but that captures the rigidity of other parts of the animal. We show that this model can determine the pose of animals such as cats in a wide range of positions. Bounded distortion forms a core part of the matching between 3D model and 2D images. I will also show that Bounded Distortion can be used for 2D matching. We use it to find corresponding features in images very robustly, optimizing an L0 distance to maximize the number of matched features, while bounding the amount of non-rigid variation between the images. We demonstrate the use of this approach in matching non-rigid objects and in wide-baseline matching of features. I will also give an overview of a method for identifying the parts of animals in images, to produce an automatic correspondence between images of animals. Building on these correspondences we develop methods for recognizing the species of a bird, or the breed of a dog. We use these recognition algorithms to construct electronic field guides. I will describe three field guides that we have published, Birdsnap, Dogsnap, and Leafsnap. Leafsnap identifies the species of trees using shape-based matching to compare images of leaves. Leafsnap has been downloaded by over 1.5 million users, and has been used in schools and in biodiversity studies. This work has been done in collaboration with many University of Maryland students and with groups at Columbia University, the Smithsonian Institution National Museum of Natural History, and the Weizmann Institute.

Organizers: Stephan Streuber


Understanding Plants and Animals

Talk
  • 10 November 2015 • 11:00 12:00
  • Prof. David W. Jacobs
  • MRZ seminar room

I will describe a series of work that aims to automatically understand images of animals and plants. I will begin by describing recent work that uses Bounded Distortion matching to model pose variation in animals. Using a generic 3D model of an animal and multiple images of different individuals in various poses, we construct a model that captures the way in which the animal articulates. This is done by solving for the pose of the template that matches each image while simultaneously solving for the stiffness of each tetrahedron of the model. We minimize an L1 norm on stiffness, producing a model that bends easily at joints, but that captures the rigidity of other parts of the animal. We show that this model can determine the pose of animals such as cats in a wide range of positions. Bounded distortion forms a core part of the matching between 3D model and 2D images. I will also show that Bounded Distortion can be used for 2D matching. We use it to find corresponding features in images very robustly, optimizing an L0 distance to maximize the number of matched features, while bounding the amount of non-rigid variation between the images. We demonstrate the use of this approach in matching non-rigid objects and in wide-baseline matching of features. I will also give an overview of a method for identifying the parts of animals in images, to produce an automatic correspondence between images of animals. Building on these correspondences we develop methods for recognizing the species of a bird, or the breed of a dog. We use these recognition algorithms to construct electronic field guides. I will describe three field guides that we have published, Birdsnap, Dogsnap, and Leafsnap. Leafsnap identifies the species of trees using shape-based matching to compare images of leaves. Leafsnap has been downloaded by over 1.5 million users, and has been used in schools and in biodiversity studies. This work has been done in collaboration with many University of Maryland students and with groups at Columbia University, the Smithsonian Institution National Museum of Natural History, and the Weizmann Institute.

Organizers: Stephan Streuber


  • Olga Diamanti
  • MRZ Seminar room

The design of tangent vector fields on discrete surfaces is a basic building block for many geometry processing applications, such as surface remeshing, parameterization and architectural geometric design. Many applications require the design of multiple vector fields (vector sets) coupled in a nontrivial way; for example, sets of more than two vectors are used for meshing of triangular, quadrilateral and hexagonal meshes. In this talk, a new, polynomial-based representation for general unordered vector sets will be presented. Using this representation we can efficiently interpolate user provided vector constraints to design vector set fields. Our interpolation scheme will require neither integer period jumps, nor explicit pairings of vectors between adjacent sets on a manifold, as is common in field design literature. Several extensions to the basic interpolation scheme are possible, which make our representation applicable in various scenarios; in this talk, we will focus on generating vector set fields particularly suited for mesh parameterization and show applications in architectural modeling.

Organizers: Gerard Pons-Moll