Header logo is
Institute Talks

Robust Discrimination and Generation of Faces using Compact, Disentangled Embeddings

Talk
  • 22 August 2019 • 15:00 16:00
  • Björn Browatzki
  • PS Aquarium

Current solutions to discriminative and generative tasks in computer vision exist separately and often lack interpretability and explainability. Using faces as our application domain, here we present an architecture that is based around two core ideas that address these issues: first, our framework learns an unsupervised, low-dimensional embedding of faces using an adversarial autoencoder that is able to synthesize high-quality face images. Second, a supervised disentanglement splits the low-dimensional embedding vector into four sub-vectors, each of which contains separated information about one of four major face attributes (pose, identity, expression, and style) that can be used both for discriminative tasks and for manipulating all four attributes in an explicit manner. The resulting architecture achieves state-of-the-art image quality, good discrimination and face retrieval results on each of the four attributes, and supports various face editing tasks using a face representation of only 99 dimensions. Finally, we apply the architecture's robust image synthesis capabilities to visually debug label-quality issues in an existing face dataset.

Organizers: Timo Bolkart

  • Ingmar H. Riedel-Kruse
  • Max-Planck-Institute for Intelligent Systems, Heisenbergstraße 3, Stuttgart, Room 2P4

I will share my vision that microbiological systems should be as programmable, interactive, accessible, constructible, and useful as our personal electronic devices. Natural multi-cellular organisms and symbiotic systems achieve complex tasks through division of labor among cells. Such systems transcend current electronics and robotics in many ways, e.g., they synthesize chemicals, generate active physical forms, and self-replicate. Harnessing these features promises significant impact for manufacturing (bioelectronics / smart materials /swarm robotics), health (tissue engineering), chemistry (pathway modularization), ecology (bioremediation), biodesign (art), and more. My lab takes a synergistic bottom-up / top-down approach to achieve such transformative applications: (1) We utilize synthetic biology and biophysics approaches to engineer and understand multi-cell bacterial assemblies. We developed the first synthetic cell-cell adhesion toolbox [1] and optogenetic cell-surface adhesion toolbox (‘Biofilm Lithography’) [2]. Integration with standard synthetic biology components (e.g., for signaling, differentiation, logic) now enables a new intelligent materials paradigm that rests on versatile, modular, and composable smart particles (i.e., cells). (2) We pioneered ‘Interactive Biotechnology’ that enables humans to directly interact with living multi-cell assemblies in real-time. I will provide the rational for this interactivity, demonstrate multiple applications using phototactic Euglena cells (e.g., tangible museum exhibits [3], biology cloud experimentation labs [4], biotic video games [5]), and show how this technology aided the discovery of new microswimmer phototaxis control strategies [6]. Finally, I discuss architecture and swarm programming languages for future bio-electronic devices (i.e., ‘Biotic Processing Units’ – BPUs) [7,8]. REFs: [1] Glass, Cell ’18; [2] Jin, PNAS ’18; [3] Lee, CHI ACM ’15; [4] Hossain, Nature Biotech ‘16; [5] Cira, PLoS Biology ‘15; [6] Tsang, Nature Physics ’18; [7] Lam LOC ‘17; [8] Washington, PNAS ‘19.


Perceptual and Affective Characteristics of Tactile Stimuli

Talk
  • 14 February 2019 • 15:00 16:00
  • Yongjae Yoo
  • 2P4 in Heisenbergstr. 3

With the advent of technology, tactile stimuli are adopted widely in many human-computer interactions. However, their perceptual and emotional characteristics are not much studied yet. In this talk, to help in understanding these characteristics, I will introduce my perception and emotion studies, as well as my future research plan. For perceptual characteristics, I will introduce an estimation method for perceived intensity of superimposed vibrations, verbal expressions for vibrotactile stimuli, and adjectival magnitude functions. Then, I will present a vibrotactile authoring tool that utilizes the adjectival magnitude functions as an application. For affective characteristics, I will introduce my emotion studies that investigate the effects of physical parameters of vibrotactile and thermal stimuli on the emotional responses using the valence-arousal space (V-A space). Then, as an application, I will present an emotion augmenting method that changes the emotion of visual stimuli in mobile devices using tactile stimuli.

Organizers: Katherine J. Kuchenbecker


Neural networks discovering quantum error correction strategies

IS Colloquium
  • 28 January 2019 • 11:15 12:15
  • Florian Marquardt
  • MPI IS building, Max Planck Ring 4, N0.002 (main seminar room)

Machine learning with artificial neural networks is revolutionizing science. The most advanced challenges require discovering answers autonomously. In the domain of reinforcement learning, control strategies are improved according to a reward function. The power of neural-network-based reinforcement learning has been highlighted by spectacular recent successes such as playing Go, but its benefits for physics are yet to be demonstrated. Here, we show how a network-based "agent" can discover complete quantum-error-correction strategies, protecting a collection of qubits against noise. These strategies require feedback adapted to measurement outcomes. Finding them from scratch without human guidance and tailored to different hardware resources is a formidable challenge due to the combinatorially large search space. To solve this challenge, we develop two ideas: two-stage learning with teacher and student networks and a reward quantifying the capability to recover the quantum information stored in a multiqubit system. Beyond its immediate impact on quantum computation, our work more generally demonstrates the promise of neural-network-based reinforcement learning in physics.


References: Reinforcement Learning with Neural Networks for Quantum Feedback Thomas Fösel, Petru Tighineanu, Talitha Weiss, Florian Marquardt Physical Review X 8(3) (2018)

Organizers: Matthias Bauer


  • Yuliang Xiu
  • PS Aquarium

Multi-person articulated pose tracking is an important while challenging problem in human behavior understanding. In this talk, going along the road of top-down approaches, I will introduce a decent and efficient pose tracker based on pose flows. This approach can achieve real-time pose tracking without loss of accuracy. Besides, to better understand human activities in visual contents, clothes texture and geometric details also play indispensable roles. However, extrapolating them from a single image is much more difficult than rigid objects due to its large variations in pose, shape, and cloth. I will present a two-stage pipeline to predict human bodies and synthesize human novel views from one single-view image.

Organizers: Siyu Tang


Mind Games

IS Colloquium
  • 21 December 2018 • 11:00 12:00
  • Peter Dayan
  • IS Lecture Hall

Much existing work in reinforcement learning involves environments that are either intentionally neutral, lacking a role for cooperation and competition, or intentionally simple, when agents need imagine nothing more than that they are playing versions of themselves. Richer game theoretic notions become important as these constraints are relaxed. For humans, this encompasses issues that concern utility, such as envy and guilt, and that concern inference, such as recursive modeling of other players, I will discuss studies treating a paradigmatic game of trust as an interactive partially-observable Markov decision process, and will illustrate the solution concepts with evidence from interactions between various groups of subjects, including those diagnosed with borderline and anti-social personality disorders.


  • Yao Feng
  • PS Aquarium

In this talk, I will present my understanding on 3D face reconstruction, modelling and applications from a deep learning perspective. In the first part of my talk, I will discuss the relationship between representations (point clouds, meshes, etc) and network layers (CNN, GCN, etc) on face reconstruction task, then present my ECCV work PRN which proposed a new representation to help achieve state-of-the-art performance on face reconstruction and dense alignment tasks. I will also introduce my open source project face3d that provides examples for generating different 3D face representations. In the second part of the talk, I will talk some publications in integrating 3D techniques into deep networks, then introduce my upcoming work which implements this. In the third part, I will present how related tasks could promote each other in deep learning, including face recognition for face reconstruction task and face reconstruction for face anti-spoofing task. Finally, with such understanding of these three parts, I will present my plans on 3D face modelling and applications.

Organizers: Timo Bolkart


Generating Faces & Heads: Texture, Shape and Beyond.

Talk
  • 17 December 2018 • 11:00 12:00
  • Stefanos Zafeiriou
  • PS Aquarium

The past few years with the advent of Deep Convolutional Neural Networks (DCNNs), as well as the availability of visual data it was shown that it is possible to produce excellent results in very challenging tasks, such as visual object recognition, detection, tracking etc. Nevertheless, in certain tasks such as fine-grain object recognition (e.g., face recognition) it is very difficult to collect the amount of data that are needed. In this talk, I will show how, using DCNNs, we can generate highly realistic faces and heads and use them for training algorithms such as face and facial expression recognition. Next, I will reverse the problem and demonstrate how by having trained a very powerful face recognition network it can be used to perform very accurate 3D shape and texture reconstruction of faces from a single image. Finally, I will demonstrate how to create very lightweight networks for representing 3D face texture and shape structure by capitalising upon intrinsic mesh convolutions.

Organizers: Dimitrios Tzionas


  • Prof. Dr. Björn Ommer
  • PS Aquarium

Understanding objects and their behavior from images and videos is a difficult inverse problem. It requires learning a metric in image space that reflects object relations in real world. This metric learning problem calls for large volumes of training data. While images and videos are easily available, labels are not, thus motivating self-supervised metric and representation learning. Furthermore, I will present a widely applicable strategy based on deep reinforcement learning to improve the surrogate tasks underlying self-supervision. Thereafter, the talk will cover the learning of disentangled representations that explicitly separate different object characteristics. Our approach is based on an analysis-by-synthesis paradigm and can generate novel object instances with flexible changes to individual characteristics such as their appearance and pose. It nicely addresses diverse applications in human and animal behavior analysis, a topic we have intensive collaboration on with neuroscientists. Time permitting, I will discuss the disentangling of representations from a wider perspective including novel strategies to image stylization and new strategies for regularization of the latent space of generator networks.

Organizers: Joel Janai


  • Yanxi Liu
  • Aquarium (N3.022)

Human pose stability analysis is the key to understanding locomotion and control of body equilibrium, with numerous applications in the fields of Kinesiology, Medicine and Robotics. We propose and validate a novel approach to learn dynamics from kinematics of a human body to aid stability analysis. More specifically, we propose an end-to-end deep learning architecture to regress foot pressure from a human pose derived from video. We have collected and utilized a set of long (5min +) choreographed Taiji (Tai Chi) sequences of multiple subjects with synchronized motion capture, foot pressure and video data. The derived human pose data and corresponding foot pressure maps are used jointly in training a convolutional neural network with residual architecture, named “PressNET”. Cross validation results show promising performance of PressNet, significantly outperforming the baseline method under reasonable sensor noise ranges.

Organizers: Nadine Rueegg


Physical Reasoning and Robot Manipulation

Talk
  • 11 December 2018 • 15:00 16:00
  • Marc Toussaint
  • 2R4 Werner Köster lecture hall

Animals and humans are excellent in conceiving of solutions to physical and geometric problems, for instance in using tools, coming up with creative constructions, or eventually inventing novel mechanisms and machines. Cognitive scientists coined the term intuitive physics in this context. It is a shame we do not yet have good computational models of such capabilities. A main stream of current robotics research focusses on training robots for narrow manipulation skills - often using massive data from physical simulators. Complementary to that we should also try to understand how basic principles underlying physics can directly be used to enable general purpose physical reasoning in robots, rather than sampling data from physical simulations. In this talk I will discuss an approach called Logic-Geometric Programming, which builds a bridge between control theory, AI planning and robot manipulation. It demonstrates strong performance on sequential manipulation problems, but also raises a number of highly interesting fundamental problems, including its probabilistic formulation, reactive execution and learning.

Organizers: Katherine J. Kuchenbecker Ildikó Papp-Wiedmann Barbara Kettemann Matthias Tröndle