Header logo is


2023


no image
Efficient Sampling from Differentiable Matrix Elements

Kofler, A.

Technical University of Munich, Germany, September 2023 (mastersthesis)

ei

[BibTex]

2023


[BibTex]


no image
Intrinsic complexity and mechanisms of expressivity of cortical neurons

Spieler, A. M.

University of Tübingen, Germany, March 2023 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Towards Generative Machine Teaching

Qui, Z.

Technical University of Munich, Germany, February 2023 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Generation and Quantification of Spin in Robot Table Tennis

Dittrich, A.

University of Stuttgart, Germany, January 2023 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Natural Language Processing for Policymaking

Jin, Z., Mihalcea, R.

In Handbook of Computational Social Science for Policy, pages: 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (inbook)

ei

DOI [BibTex]

DOI [BibTex]


Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80
Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80

Berenz, V., Widmaier, F., Guist, S., Schölkopf, B., Büchler, D.

Robot Software Architectures Workshop (RSA) 2023, ICRA, 2023 (techreport)

Abstract
Robotic applications require the integration of various modalities, encompassing perception, control of real robots and possibly the control of simulated environments. While the state-of-the-art robotic software solutions such as ROS 2 provide most of the required features, flexible synchronization between algorithms, data streams and control loops can be tedious. o80 is a versatile C++ framework for robotics which provides a shared memory model and a command framework for real-time critical systems. It enables expert users to set up complex robotic systems and generate Python bindings for scientists. o80's unique feature is its flexible synchronization between processes, including the traditional blocking commands and the novel ``bursting mode'', which allows user code to control the execution of the lower process control loop. This makes it particularly useful for setups that mix real and simulated environments.

ei

arxiv poster link (url) [BibTex]


no image
Challenging Common Assumptions in Multi-task Learning

Elich, C., Kirchdorfer, L., Köhler, J. M., Schott, L.

abs/2311.04698, CoRR/arxiv, 2023 (techreport)

ev

paper link (url) [BibTex]

paper link (url) [BibTex]

2022


Reconstructing Expressive {3D} Humans from {RGB} Images
Reconstructing Expressive 3D Humans from RGB Images

Choutas, V.

ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

Abstract
To interact with our environment, we need to adapt our body posture and grasp objects with our hands. During a conversation our facial expressions and hand gestures convey important non-verbal cues about our emotional state and intentions towards our fellow speakers. Thus, modeling and capturing 3D full-body shape and pose, hand articulation and facial expressions are necessary to create realistic human avatars for augmented and virtual reality. This is a complex task, due to the large number of degrees of freedom for articulation, body shape variance, occlusions from objects and self-occlusions from body parts, e.g. crossing our hands, and subject appearance. The community has thus far relied on expensive and cumbersome equipment, such as multi-view cameras or motion capture markers, to capture the 3D human body. While this approach is effective, it is limited to a small number of subjects and indoor scenarios. Using monocular RGB cameras would greatly simplify the avatar creation process, thanks to their lower cost and ease of use. These advantages come at a price though, since RGB capture methods need to deal with occlusions, perspective ambiguity and large variations in subject appearance, in addition to all the challenges posed by full-body capture. In an attempt to simplify the problem, researchers generally adopt a divide-and-conquer strategy, estimating the body, face and hands with distinct methods using part-specific datasets and benchmarks. However, the hands and face constrain the body and vice-versa, e.g. the position of the wrist depends on the elbow, shoulder, etc.; the divide-and-conquer approach can not utilize this constraint. In this thesis, we aim to reconstruct the full 3D human body, using only readily accessible monocular RGB images. In a first step, we introduce a parametric 3D body model, called SMPL-X, that can represent full-body shape and pose, hand articulation and facial expression. Next, we present an iterative optimization method, named SMPLify-X, that fits SMPL-X to 2D image keypoints. While SMPLify-X can produce plausible results if the 2D observations are sufficiently reliable, it is slow and susceptible to initialization. To overcome these limitations, we introduce ExPose, a neural network regressor, that predicts SMPL-X parameters from an image using body-driven attention, i.e. by zooming in on the hands and face, after predicting the body. From the zoomed-in part images, dedicated part networks predict the hand and face parameters. ExPose combines the independent body, hand, and face estimates by trusting them equally. This approach though does not fully exploit the correlation between parts and fails in the presence of challenges such as occlusion or motion blur. Thus, we need a better mechanism to aggregate information from the full body and part images. PIXIE uses neural networks called moderators that learn to fuse information from these two image sets before predicting the final part parameters. Overall, the addition of the hands and face leads to noticeably more natural and expressive reconstructions. Creating high fidelity avatars from RGB images requires accurate estimation of 3D body shape. Although existing methods are effective at predicting body pose, they struggle with body shape. We identify the lack of proper training data as the cause. To overcome this obstacle, we propose to collect internet images from fashion models websites, together with anthropometric measurements. At the same time, we ask human annotators to rate images and meshes according to a pre-defined set of linguistic attributes. We then define mappings between measurements, linguistic shape attributes and 3D body shape. Equipped with these mappings, we train a neural network regressor, SHAPY, that predicts accurate 3D body shapes from a single RGB image. We observe that existing 3D shape benchmarks lack subject variety and/or ground-truth shape. Thus, we introduce a new benchmark, Human Bodies in the Wild (HBW), which contains images of humans and their corresponding 3D ground-truth body shape. SHAPY shows how we can overcome the lack of in-the-wild images with 3D shape annotations through easy-to-obtain anthropometric measurements and linguistic shape attributes. Regressors that estimate 3D model parameters are robust and accurate, but often fail to tightly fit the observations. Optimization-based approaches tightly fit the data, by minimizing an energy function composed of a data term that penalizes deviations from the observations and priors that encode our knowledge of the problem. Finding the balance between these terms and implementing a performant version of the solver is a time-consuming and non-trivial task. Machine-learned continuous optimizers combine the benefits of both regression and optimization approaches. They learn the priors directly from data, avoiding the need for hand-crafted heuristics and loss term balancing, and benefit from optimized neural network frameworks for fast inference. Inspired from the classic Levenberg-Marquardt algorithm, we propose a neural optimizer that outperforms classic optimization, regression and hybrid optimization-regression approaches. Our proposed update rule uses a weighted combination of gradient descent and a network-predicted update. To show the versatility of the proposed method, we apply it on three other problems, namely full body estimation from (i) 2D keypoints, (ii) head and hand location from a head-mounted device and (iii) face tracking from dense 2D landmarks. Our method can easily be applied to new model fitting problems and offers a competitive alternative to well-tuned traditional model fitting pipelines, both in terms of accuracy and speed. To summarize, we propose a new and richer representation of the human body, SMPL-X, that is able to jointly model the 3D human body pose and shape, facial expressions and hand articulation. We propose methods, SMPLify-X, ExPose and PIXIE that estimate SMPL-X parameters from monocular RGB images, progressively improving the accuracy and realism of the predictions. To further improve reconstruction fidelity, we demonstrate how we can use easy-to-collect internet data and human annotations to overcome the lack of 3D shape data and train a model, SHAPY, that predicts accurate 3D body shape from a single RGB image. Finally, we propose a flexible learnable update rule for parametric human model fitting that outperforms both classic optimization and neural network approaches. This approach is easily applicable to a variety of problems, unlocking new applications in AR/VR scenarios.

ps

pdf [BibTex]

2022


pdf [BibTex]


no image
Life Improvement Science

Lieder, F., Prentice, M.

In Encyclopedia of Quality of Life and Well-Being Research, Springer, November 2022 (inbook)

re

[BibTex]

[BibTex]


no image
Investigating Independent Mechanisms in Neural Networks

Liang, W.

Université Paris-Saclay, France, October 2022 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Causality, causal digital twins, and their applications

Schölkopf, B.

Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling (Dagstuhl Seminar 22382), (Editors: Berens, Philipp and Cranmer, Kyle and Lawrence, Neil D. and von Luxburg, Ulrike and Montgomery, Jessica), September 2022 (talk)

ei

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Multi-Target Multi-Object Manipulation using Relational Deep Reinforcement Learning

Feil, M.

Technnical University Munich, Germany, September 2022 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Independent Mechanism Analysis for High Dimensions

Sliwa, J.

University of Tübingen, Germany, September 2022, (Graduate Training Centre of Neuroscience) (mastersthesis)

ei

[BibTex]

[BibTex]


no image
On the Adversarial Robustness of Causal Algorithmic Recourse

Dominguez-Olmedo, R.

University of Tübingen, Germany, August 2022 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Independent Mechanism Analysis in High-Dimensional Observation Spaces

Ghosh, S.

ETH Zurich, Switzerland, June 2022 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models

Li, H., Stueckler, J.

abs/2204.06651, CoRR/arxiv, 2022 (techreport)

Abstract
In this paper, we analyze the observability of the visual-inertial odometry (VIO) using stereo cameras with a velocity-control based kinematic motion model. Previous work shows that in general case the global position and yaw are unobservable in VIO system, additionally the roll and pitch become also unobservable if there is no rotation. We prove that by integrating a planar motion constraint roll and pitch become observable. We also show that the parameters of the motion model are observable.

ev

link (url) [BibTex]


no image
Causal Models for Dynamical Systems

Peters, J., Bauer, S., Pfister, N.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 671-690, 1, Association for Computing Machinery, 2022 (inbook)

ei

arXiv DOI [BibTex]

arXiv DOI [BibTex]


no image
Towards Causal Algorithmic Recourse

Karimi, A. H., von Kügelgen, J., Schölkopf, B., Valera, I.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 139-166, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei plg

DOI [BibTex]

DOI [BibTex]


no image
Causality for Machine Learning

Schölkopf, B.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 765-804, 1, Association for Computing Machinery, New York, NY, USA, 2022 (inbook)

ei

arXiv DOI [BibTex]

arXiv DOI [BibTex]


no image
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations

Salewski, L., Koepke, A. S., Lensch, H. P. A., Akata, Z.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 69-88, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei

DOI [BibTex]

DOI [BibTex]


no image
Voltage dependent investigations on the spin polarization of layered heterostructues

Miller, M.

Universität Stuttgart, Stuttgart, 2022 (mastersthesis)

mms

[BibTex]

[BibTex]

2021


Magnetic Micro-/Nanopropellers  for Biomedicine
Magnetic Micro-/Nanopropellers for Biomedicine

Qiu, T., Jeong, M., Goyal, R., Kadiri, V., Sachs, J., Fischer, P.

In Field-Driven Micro and Nanorobots for Biology and Medicine, pages: 389-410, 16, (Editors: Sun, Y. and Wang, X. and Yu, J.), Springer Nature, November 2021 (inbook)

Abstract
In nature, many bacteria swim by rotating their helical flagella. A particularly promising class of artificial micro- and nano-robots mimic this propeller-like propulsion mechanism to move through fluids and tissues for applications in minimally-invasive medicine. Several fundamental challenges have to be overcome in order to build micro-machines that move similar to bacteria for in vivo applications. Here, we review recent advances of magnetically-powered micro-/nano-propellers. Four important aspects of the propellers – the geometrical shape, the fabrication method, the generation of magnetic fields for actuation, and the choice of biocompatible magnetic materials – are highlighted. First, the fundamental requirements are elucidated that arise due to hydrodynamics at low Reynolds (Re) number. We discuss the role that the propellers’ shape and symmetry play in realizing effective propulsion at low Re. Second, the additive nano-fabrication method Glancing Angle Deposition is discussed as a versatile technique to quickly grow large numbers of designer nano-helices. Third, systems to generate rotating magnetic fields via permanent magnets or electromagnetic coils are presented. And finally, the biocompatibility of the magnetic materials is discussed. Iron-platinum is highlighted due to its biocompatibility and its superior magnetic properties, which is promising for targeted delivery, minimally-invasive magnetic nano-devices and biomedical applications.

pf

link (url) DOI [BibTex]

2021


link (url) DOI [BibTex]


no image
Learning Neural Causal Models with Active Interventions

Scherrer, N.

ETH Zurich, Switzerland, November 2021 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Study of the Interventional Consistency of Autoencoders

Lanzillotta, G.

ETH Zurich, Switzerland, October 2021 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Physically Plausible Tracking & Reconstruction of Dynamic Objects

Strecke, M., Stückler, J.

KIT Science Week Scientific Conference & DGR-Days 2021, October 2021 (talk)

ev

[BibTex]

[BibTex]


no image
Electriflow: Augmenting Books With Tangible Animation Using Soft Electrohydraulic Actuators

Purnendu, , Novack, S., Acome, E., Alistar, M., Keplinger, C., Gross, M. D., Bruns, C., Leithinger, D.

In ACM SIGGRAPH 2021 Labs, pages: 1-2, Association for Computing Machinery, SIGGRAPH 2021, August 2021 (inbook)

Abstract
We present Electriflow: a method of augmenting books with tangible animation employing soft electrohydraulic actuators. These actuators are compact, silent and fast in operation, and can be fabricated with commodity materials. They generate an immediate hydraulic force upon electrostatic activation without an external fluid supply source, enabling a simple and self-contained design. Electriflow actuators produce an immediate shape transition from flat to folded state which enabled their seamless integration into books. For the Emerging Technologies exhibit, we will demonstrate the prototype of a book augmented with the capability of tangible animation.

rm

Supplemental Material link (url) DOI [BibTex]

Supplemental Material link (url) DOI [BibTex]


Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning
Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning

Heindrich, L., Consul, S., Stojcheski, J., Lieder, F.

Tübingen, Germany, The first edition of Life Improvement Science Conference, June 2021 (talk) Accepted

Abstract
The discovery of decision strategies is an essential part of creating effective cognitive tutors that teach planning and decision-making skills to humans. In the context of bounded rationality, this requires weighing the benefits of different planning operations compared to their computational costs. For small decision problems, it has already been shown that near-optimal decision strategies can be discovered automatically and that the discovered strategies can be taught to humans to increase their performance. Unfortunately, these near-optimal strategy discovery algorithms have not been able to scale well to larger problems due to their computational complexity. In this talk, we will present recent work at the Rationality Enhancement Group to overcome the computational bottleneck of existing strategy discovery algorithms. Our approach makes use of the hierarchical structure of human behavior by decomposing sequential decision problems into two sub-problems: setting a goal and planning how to achieve it. An additional metacontroller component is introduced to switch the current goal when it becomes beneficial. The hierarchical decomposition enables us to discover near-optimal strategies for human planning in larger and more complex tasks than previously possible. We then show in online experiments that teaching the discovered strategies to humans improves their performance in complex sequential decision-making tasks.

re

Project Page [BibTex]

Project Page [BibTex]


Toward a Science of Effective Well-Doing
Toward a Science of Effective Well-Doing

Lieder, F., Prentice, M., Corwin-Renner, E.

May 2021 (techreport)

Abstract
Well-doing, broadly construed, encompasses acting and thinking in ways that contribute to humanity’s flourishing in the long run. This often takes the form of setting a prosocial goal and pursuing it over an extended period of time. To set and pursue goals in a way that is extremely beneficial for humanity (effective well-doing), people often have to employ critical thinking and far-sighted, rational decision-making in the service of the greater good. To promote effective well-doing, we need to better understand its determinants and psychological mechanisms, as well as the barriers to effective well-doing and how they can be overcome. In this article, we introduce a taxonomy of different forms of well-doing and introduce a conceptual model of the cognitive mechanisms of effective well-doing. We view effective well-doing as the upper end of a moral continuum whose lower half comprises behaviors that are harmful to humanity (ill-doing), and we argue that the capacity for effective well-doing has to be developed through personal growth (e.g., learning how to pursue goals effectively). Research on these phenomena has so far been scattered across numerous disconnected literatures from multiple disciplines. To bring these communities together, we call for the establishment of a transdisciplinary research field focussed on understanding and promoting effective well-doing and personal growth as well as understanding and reducing ill-doing. We define this research field in terms of its goals and questions. We review what is already known about these questions in different disciplines and argue that laying the scientific foundation for promoting effective well-doing is one of the most valuable contributions that the behavioral sciences can make in the 21st century.

re

Preprint Project Page [BibTex]


no image
Robotic Surgery Training in AR: Multimodal Record and Replay

Krauthausen, F.

pages: 1-147, University of Stuttgart, Stuttgart, May 2021, Study Program in Software Engineering (mastersthesis)

hi

[BibTex]

[BibTex]


no image
Direct detection of spin Hall effect induced torques in platinum/ferromagnetic bilayer systems

Alten, F.

Universität Stuttgart, Stuttgart, January 2021 (mastersthesis)

mms

[BibTex]


no image
Turbulence Modulation and Energy Transfer in Turbulent Channel Flow Coupled with One-Side Porous Media

Chu, X., Wang, W., Müller, J., Schöning, H. V., Liu, Y., Weigand, B.

In High Performance Computing in Science and Engineering’20, pages: 373-386, Springer, 2021 (incollection)

minibot

[BibTex]

[BibTex]

2020


no image
Voltage dependent interfacial magnetism in multilayer systems

Nacke, R.

Universität Stuttgart, Stuttgart, December 2020 (thesis)

mms

[BibTex]

2020


[BibTex]


no image
Hydromagnonics: Manipulation of magnonic systems with hydrogen

Sauter, R.

Universität Stuttgart, Stuttgart, December 2020 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Ahmed, O.

ETH Zurich, Switzerland, October 2020 (mastersthesis)

ei

[BibTex]

[BibTex]


Optimal To-Do List Gamification
Optimal To-Do List Gamification

Stojcheski, J., Felso, V., Lieder, F.

ArXiv Preprint, 2020 (techreport)

Abstract
What should I work on first? What can wait until later? Which projects should I prioritize and which tasks are not worth my time? These are challenging questions that many people face every day. People’s intuitive strategy is to prioritize their immediate experience over the long-term consequences. This leads to procrastination and the neglect of important long-term projects in favor of seemingly urgent tasks that are less important. Optimal gamification strives to help people overcome these problems by incentivizing each task by a number of points that communicates how valuable it is in the long-run. Unfortunately, computing the optimal number of points with standard dynamic programming methods quickly becomes intractable as the number of a person’s projects and the number of tasks required by each project increase. Here, we introduce and evaluate a scalable method for identifying which tasks are most important in the long run and incentivizing each task according to its long-term value. Our method makes it possible to create to-do list gamification apps that can handle the size and complexity of people’s to-do lists in the real world.

re

link (url) DOI Project Page [BibTex]


no image
Deep learning for the parameter estimation of tight-binding Hamiltonians

Cacioppo, A.

University of Roma, La Sapienza, Italy, May 2020 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Learning Algorithms, Invariances, and the Real World

Zecevic, M.

Technical University of Darmstadt, Germany, April 2020 (mastersthesis)

ei

[BibTex]

[BibTex]


no image
Interaction of hydrogen isotopes with flexible metal-organic frameworks

Bondorf, L.

Universität Stuttgart, Stuttgart, February 2020 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures

Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D.

In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer International Publishing, 2020 (inbook)

ev

link (url) [BibTex]

link (url) [BibTex]


no image
Developing new methods for routing and optimal transport on networks

Lonardi, A.

Università degli studi di Padova, 2020 (mastersthesis)

pio

pdf [BibTex]

pdf [BibTex]