Header logo is

667 results (View BibTeX file of all listed publications)

2024

no image
Identifiable Causal Representation Learning

von Kügelgen, J.

University of Cambridge, UK, Cambridge, February 2024, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

2024

ei von Kügelgen, J. Identifiable Causal Representation Learning University of Cambridge, UK, Cambridge, February 2024, (Cambridge-Tübingen-Fellowship) (phdthesis)

[BibTex]

Creating a Haptic Empathetic Robot Animal That Feels Touch and Emotion
Creating a Haptic Empathetic Robot Animal That Feels Touch and Emotion

Burns, R.

University of Tübingen, Tübingen, Germany, February 2024, Department of Computer Science (phdthesis)

Abstract
Social touch, such as a hug or a poke on the shoulder, is an essential aspect of everyday interaction. Humans use social touch to gain attention, communicate needs, express emotions, and build social bonds. Despite its importance, touch sensing is very limited in most commercially available robots. By endowing robots with social-touch perception, one can unlock a myriad of new interaction possibilities. In this thesis, I present my work on creating a Haptic Empathetic Robot Animal (HERA), a koala-like robot for children with autism. I demonstrate the importance of establishing design guidelines based on one's target audience, which we investigated through interviews with autism specialists. I share our work on creating full-body tactile sensing for the NAO robot using low-cost, do-it-yourself (DIY) methods, and I introduce an approach to model long-term robot emotions using second-order dynamics.

hi

Project Page [BibTex]

hi Burns, R. Creating a Haptic Empathetic Robot Animal That Feels Touch and Emotion University of Tübingen, Tübingen, Germany, February 2024, Department of Computer Science (phdthesis)

Project Page [BibTex]

2023

Hydraulically Amplified Self-healing Electrostatic Actuators
Hydraulically Amplified Self-healing Electrostatic Actuators

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K.

(US Patent 11795979B2), October 2023 (patent)

Abstract
An electro-hydraulic actuator includes a deformable shell defining an enclosed internal cavity and containing a liquid dielectric, first and second electrodes on first and second sides, respectively, of the enclosed internal cavity. An electrostatic force between the first and second electrodes upon application of a voltage to one of the electrodes draws the electrodes towards each other to displace the liquid dielectric within the enclosed internal cavity. The shell includes active and inactive areas such that the electrostatic forces between the first and second electrodes displaces the liquid dielectric within the enclosed internal cavity from the active area of the shell to the inactive area of the shell. The first and second electrodes, the deformable shell, and the liquid dielectric cooperate to form a self-healing capacitor, and the liquid dielectric is configured for automatically filling breaches in the liquid dielectric resulting from dielectric breakdown.

rm

link (url) [BibTex]

2023

rm Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K. Hydraulically Amplified Self-healing Electrostatic Actuators (US Patent 11795979B2), October 2023 (patent)

link (url) [BibTex]

no image
Gesture-Based Nonverbal Interaction for Exercise Robots

Mohan, M.

University of Tübingen, Tübingen, Germany, October 2023, Department of Computer Science (phdthesis)

Abstract
When teaching or coaching, humans augment their words with carefully timed hand gestures, head and body movements, and facial expressions to provide feedback to their students. Robots, however, rarely utilize these nuanced cues. A minimally supervised social robot equipped with these abilities could support people in exercising, physical therapy, and learning new activities. This thesis examines how the intuitive power of human gestures can be harnessed to enhance human-robot interaction. To address this question, this research explores gesture-based interactions to expand the capabilities of a socially assistive robotic exercise coach, investigating the perspectives of both novice users and exercise-therapy experts. This thesis begins by concentrating on the user's engagement with the robot, analyzing the feasibility of minimally supervised gesture-based interactions. This exploration seeks to establish a framework in which robots can interact with users in a more intuitive and responsive manner. The investigation then shifts its focus toward the professionals who are integral to the success of these innovative technologies: the exercise-therapy experts. Roboticists face the challenge of translating the knowledge of these experts into robotic interactions. We address this challenge by developing a teleoperation algorithm that can enable exercise therapists to create customized gesture-based interactions for a robot. Thus, this thesis lays the groundwork for dynamic gesture-based interactions in minimally supervised environments, with implications for not only exercise-coach robots but also broader applications in human-robot interaction.

hi

Project Page [BibTex]

hi Mohan, M. Gesture-Based Nonverbal Interaction for Exercise Robots University of Tübingen, Tübingen, Germany, October 2023, Department of Computer Science (phdthesis)

Project Page [BibTex]

High Strain Peano Hydraulically Amplified Self-Healing Electrostatic (HASEL) Transducers
High Strain Peano Hydraulically Amplified Self-Healing Electrostatic (HASEL) Transducers

Keplinger, C. M., Wang, X., Mitchell, S. K.

(US Patent App. 18/138,621), August 2023 (patent)

Abstract
High strain hydraulically amplified self-healing electrostatic transducers having increased maximum theoretical and practical strains are disclosed. In particular, the actuators include electrode configurations having a zipping front created by the attraction of the electrodes that is configured orthogonally to a strain axis along which the actuators. This configuration produces increased strains. In turn, various form factors for the actuator configuration are presented including an artificial circular muscle and a strain amplifying pulley system. Other actuator configurations are contemplated that include independent and opposed electrode pairs to create cyclic activation, hybrid electrode configurations, and use of strain limiting layers for controlled deflection of the actuator.

rm

link (url) [BibTex]

no image
Learning and Testing Powerful Hypotheses

Kübler, J. M.

University of Tübingen, Germany, July 2023 (phdthesis)

ei

[BibTex]

ei Kübler, J. M. Learning and Testing Powerful Hypotheses University of Tübingen, Germany, July 2023 (phdthesis)

[BibTex]

Capacitive Self-Sensing for Electrostatic Transducers with High Voltage Isolation
Capacitive Self-Sensing for Electrostatic Transducers with High Voltage Isolation

Correll, N., Ly, K. D., Kellaris, N. A., Keplinger, C. M.

(US Patent App. 17/928,453), June 2023 (patent)

Abstract
Transducer systems disclosed herein include self-sensing capabilities. In particular, electrostatic transducers include a low voltage electrode and a high voltage electrode. A low voltage sensing unit is coupled with the low voltage electrode of the electrostatic transducer. The low voltage sensing unit is configured to measure a capacitance of the electrostatic transducer, from which displacement of the electrostatic transducer may be calculated. High voltage drive signals received by the high voltage electrode during actuation may be isolated from the low voltage sensing unit. The isolation may be provided by dielectric material of the electrostatic transducer, a voltage suppression component, and/or a voltage suppression module comprising a low impedance ground path. In the event of an electrical failure of the transducer, the low voltage sensing unit may be isolated from high voltages.

rm

link (url) [BibTex]

rm Correll, N., Ly, K. D., Kellaris, N. A., Keplinger, C. M. Capacitive Self-Sensing for Electrostatic Transducers with High Voltage Isolation (US Patent App. 17/928,453), June 2023 (patent)

link (url) [BibTex]

no image
Learning Identifiable Representations: Independent Influences and Multiple Views

Gresele, L.

University of Tübingen, Germany, June 2023 (phdthesis)

ei

[BibTex]

no image
Learning with and for discrete optimization

Paulus, M.

(ETH Zurich, Switzerland), May 2023, CLS PhD Program (phdthesis)

ei

[BibTex]

ei Paulus, M. Learning with and for discrete optimization (ETH Zurich, Switzerland), May 2023, CLS PhD Program (phdthesis)

[BibTex]

High Strain Peano Hydraulically Amplified Self-healing Electrostatic (HASEL) Transducers
High Strain Peano Hydraulically Amplified Self-healing Electrostatic (HASEL) Transducers

Keplinger, C. M., Wang, X., Mitchell, S. K.

(US Patent 11635094), April 2023 (patent)

Abstract
High strain hydraulically amplified self-healing electrostatic transducers having increased maximum theoretical and practical strains are disclosed. In particular, the actuators include electrode configurations having a zipping front created by the attraction of the electrodes that is configured orthogonally to a strain axis along which the actuators. This configuration produces increased strains. In turn, various form factors for the actuator configuration are presented including an artificial circular muscle and a strain amplifying pulley system. Other actuator configurations are contemplated that include independent and opposed electrode pairs to create cyclic activation, hybrid electrode configurations, and use of strain limiting layers for controlled deflection of the actuator.

rm

link (url) [BibTex]

An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment
An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment

Sarvestani, A., Ruppert, F., Badri-Spröwitz, A.

2023 (unpublished) Submitted

Abstract
Ground reaction force sensing is one of the key components of gait analysis in legged locomotion research. To measure continuous force data during locomotion, we present a novel compound instrumented treadmill design. The treadmill is 1.7 m long, with a natural frequency of 170 Hz and an adjustable range that can be used for humans and small robots alike. Here, we present the treadmill’s design methodology and characterize it in its natural frequency, noise behavior and real-life performance. Additionally, we apply an ISO 376 norm conform calibration procedure for all spatial force directions and center of pressure position. We achieve a force accuracy of ≤ 5.6 N for the ground reaction forces and ≤ 13 mm in center of pressure position.

dlg

arXiv link (url) DOI [BibTex]

no image
Natural Language Processing for Policymaking

Jin, Z., Mihalcea, R.

In Handbook of Computational Social Science for Policy, pages: 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (inbook)

ei

DOI [BibTex]

ei Jin, Z., Mihalcea, R. Natural Language Processing for Policymaking In Handbook of Computational Social Science for Policy, pages: 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (inbook)

DOI [BibTex]

no image
Object-Level Dynamic Scene Reconstruction With Physical Plausibility From RGB-D Images

Strecke, M. F.

Eberhard Karls Universität Tübingen, Tübingen, 2023 (phdthesis)

Abstract
Humans have the remarkable ability to perceive and interact with objects in the world around them. They can easily segment objects from visual data and have an intuitive understanding of how physics influences objects. By contrast, robots are so far often constrained to tailored environments for a specific task, due to their inability to reconstruct a versatile and accurate scene representation. In this thesis, we combine RGB-D video data with background knowledge of real-world physics to develop such a representation for robots.

Our contributions can be separated into two main parts: a dynamic object tracking tool and optimization frameworks that allow for improving shape reconstructions based on physical plausibility. The dynamic object tracking tool "EM-Fusion" detects, segments, reconstructs, and tracks objects from RGB-D video data. We propose a probabilistic data association approach for attributing the image pixels to the different moving objects in the scene. This allows us to track and reconstruct moving objects and the background scene with state-of-the art accuracy and robustness towards occlusions.

We investigate two ways of further optimizing the reconstructed shapes of moving objects based on physical plausibility. The first of these, "Co-Section", includes physical plausibility by reasoning about the empty space around an object. We observe that no two objects can occupy the same space at the same time and that the depth images in the input video provide an estimate of observed empty space. Based on these observations, we propose intersection and hull constraints, which we combine with the observed surfaces in a global optimization approach. Compared to EM-Fusion, which only reconstructs the observed surface, Co-Section optimizes watertight shapes. These watertight shapes provide a rough estimate of unseen surfaces and could be useful as initialization for further refinement, e.g., by interactive perception. In the second optimization approach, "DiffSDFSim", we reason about object shapes based on physically plausible object motion. We observe that object trajectories after collisions depend on the object's shape, and extend a differentiable physics simulation for optimizing object shapes together with other physical properties (e.g., forces, masses, friction) based on the motion of the objects and their interactions. Our key contributions are using signed distance function models for representing shapes and a novel method for computing gradients that models the dependency of the time of contact on object shapes. We demonstrate that our approach recovers target shapes well by fitting to target trajectories and depth observations. Further, the ground-truth trajectories are recovered well in simulation using the resulting shape and physical properties. This enables predictions about the future motion of objects by physical simulation.

We anticipate that our contributions can be useful building blocks in the development of 3D environment perception for robots. The reconstruction of individual objects as in EM-Fusion is a key ingredient required for interactions with objects. Completed shapes as the ones provided by Co-Section provide useful cues for planning interactions like grasping of objects. Finally, the recovery of shape and other physical parameters using differentiable simulation as in DiffSDFSim allows simulating objects and thus predicting the effects of interactions. Future work might extend the presented works for interactive perception of dynamic environments by comparing these predictions with observed real-world interactions to further improve the reconstructions and physical parameter estimations.

ev

link (url) DOI [BibTex]

no image
Microfibers with mushroom-shaped tips for optimal adhesion

Sitti, M., Aksak, B.

2023, US Patent 11,613,674 (patent)

pi

[BibTex]

pi Sitti, M., Aksak, B. Microfibers with mushroom-shaped tips for optimal adhesion 2023, US Patent 11,613,674 (patent)

[BibTex]

Magnetic trap system and method of navigating a microscopic device
Magnetic trap system and method of navigating a microscopic device

Son, D., Ugurlu, M., Bluemer, P., Sitti, M.

2023, US Patent App. 17/871,598 (patent)

pi

[BibTex]

no image
Static and dynamic investigation of magnonic systems: materials, applications and modeling

Schulz, Frank Martin Ernst

Universität Stuttgart, Stuttgart, 2023 (phdthesis)

mms

link (url) DOI [BibTex]

mms Schulz, Frank Martin Ernst Static and dynamic investigation of magnonic systems: materials, applications and modeling Universität Stuttgart, Stuttgart, 2023 (phdthesis)

link (url) DOI [BibTex]

2022

no image
DRY ADHESIVES AND METHODS FOR MAKING DRY ADHESIVES

Metin Sitti, Michael Murphy, Burak Aksak

December 2022, US Patent App. 17/895,334, 2022 (patent)

pi

[BibTex]

2022

pi Metin Sitti, Michael Murphy, Burak Aksak DRY ADHESIVES AND METHODS FOR MAKING DRY ADHESIVES December 2022, US Patent App. 17/895,334, 2022 (patent)

[BibTex]

Reconstructing Expressive {3D} Humans from {RGB} Images
Reconstructing Expressive 3D Humans from RGB Images

Choutas, V.

ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

Abstract
To interact with our environment, we need to adapt our body posture and grasp objects with our hands. During a conversation our facial expressions and hand gestures convey important non-verbal cues about our emotional state and intentions towards our fellow speakers. Thus, modeling and capturing 3D full-body shape and pose, hand articulation and facial expressions are necessary to create realistic human avatars for augmented and virtual reality. This is a complex task, due to the large number of degrees of freedom for articulation, body shape variance, occlusions from objects and self-occlusions from body parts, e.g. crossing our hands, and subject appearance. The community has thus far relied on expensive and cumbersome equipment, such as multi-view cameras or motion capture markers, to capture the 3D human body. While this approach is effective, it is limited to a small number of subjects and indoor scenarios. Using monocular RGB cameras would greatly simplify the avatar creation process, thanks to their lower cost and ease of use. These advantages come at a price though, since RGB capture methods need to deal with occlusions, perspective ambiguity and large variations in subject appearance, in addition to all the challenges posed by full-body capture. In an attempt to simplify the problem, researchers generally adopt a divide-and-conquer strategy, estimating the body, face and hands with distinct methods using part-specific datasets and benchmarks. However, the hands and face constrain the body and vice-versa, e.g. the position of the wrist depends on the elbow, shoulder, etc.; the divide-and-conquer approach can not utilize this constraint. In this thesis, we aim to reconstruct the full 3D human body, using only readily accessible monocular RGB images. In a first step, we introduce a parametric 3D body model, called SMPL-X, that can represent full-body shape and pose, hand articulation and facial expression. Next, we present an iterative optimization method, named SMPLify-X, that fits SMPL-X to 2D image keypoints. While SMPLify-X can produce plausible results if the 2D observations are sufficiently reliable, it is slow and susceptible to initialization. To overcome these limitations, we introduce ExPose, a neural network regressor, that predicts SMPL-X parameters from an image using body-driven attention, i.e. by zooming in on the hands and face, after predicting the body. From the zoomed-in part images, dedicated part networks predict the hand and face parameters. ExPose combines the independent body, hand, and face estimates by trusting them equally. This approach though does not fully exploit the correlation between parts and fails in the presence of challenges such as occlusion or motion blur. Thus, we need a better mechanism to aggregate information from the full body and part images. PIXIE uses neural networks called moderators that learn to fuse information from these two image sets before predicting the final part parameters. Overall, the addition of the hands and face leads to noticeably more natural and expressive reconstructions. Creating high fidelity avatars from RGB images requires accurate estimation of 3D body shape. Although existing methods are effective at predicting body pose, they struggle with body shape. We identify the lack of proper training data as the cause. To overcome this obstacle, we propose to collect internet images from fashion models websites, together with anthropometric measurements. At the same time, we ask human annotators to rate images and meshes according to a pre-defined set of linguistic attributes. We then define mappings between measurements, linguistic shape attributes and 3D body shape. Equipped with these mappings, we train a neural network regressor, SHAPY, that predicts accurate 3D body shapes from a single RGB image. We observe that existing 3D shape benchmarks lack subject variety and/or ground-truth shape. Thus, we introduce a new benchmark, Human Bodies in the Wild (HBW), which contains images of humans and their corresponding 3D ground-truth body shape. SHAPY shows how we can overcome the lack of in-the-wild images with 3D shape annotations through easy-to-obtain anthropometric measurements and linguistic shape attributes. Regressors that estimate 3D model parameters are robust and accurate, but often fail to tightly fit the observations. Optimization-based approaches tightly fit the data, by minimizing an energy function composed of a data term that penalizes deviations from the observations and priors that encode our knowledge of the problem. Finding the balance between these terms and implementing a performant version of the solver is a time-consuming and non-trivial task. Machine-learned continuous optimizers combine the benefits of both regression and optimization approaches. They learn the priors directly from data, avoiding the need for hand-crafted heuristics and loss term balancing, and benefit from optimized neural network frameworks for fast inference. Inspired from the classic Levenberg-Marquardt algorithm, we propose a neural optimizer that outperforms classic optimization, regression and hybrid optimization-regression approaches. Our proposed update rule uses a weighted combination of gradient descent and a network-predicted update. To show the versatility of the proposed method, we apply it on three other problems, namely full body estimation from (i) 2D keypoints, (ii) head and hand location from a head-mounted device and (iii) face tracking from dense 2D landmarks. Our method can easily be applied to new model fitting problems and offers a competitive alternative to well-tuned traditional model fitting pipelines, both in terms of accuracy and speed. To summarize, we propose a new and richer representation of the human body, SMPL-X, that is able to jointly model the 3D human body pose and shape, facial expressions and hand articulation. We propose methods, SMPLify-X, ExPose and PIXIE that estimate SMPL-X parameters from monocular RGB images, progressively improving the accuracy and realism of the predictions. To further improve reconstruction fidelity, we demonstrate how we can use easy-to-collect internet data and human annotations to overcome the lack of 3D shape data and train a model, SHAPY, that predicts accurate 3D body shape from a single RGB image. Finally, we propose a flexible learnable update rule for parametric human model fitting that outperforms both classic optimization and neural network approaches. This approach is easily applicable to a variety of problems, unlocking new applications in AR/VR scenarios.

ps

pdf [BibTex]

ps Choutas, V. Reconstructing Expressive 3D Humans from RGB Images ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

pdf [BibTex]

no image
Multi-Timescale Representation Learning of Human and Robot Haptic Interactions

Richardson, B.

University of Stuttgart, Stuttgart, Germany, December 2022, Faculty of Computer Science, Electrical Engineering and Information Technology (phdthesis)

Abstract
The sense of touch is one of the most crucial components of the human sensory system. It allows us to safely and intelligently interact with the physical objects and environment around us. By simply touching or dexterously manipulating an object, we can quickly infer a multitude of its properties. For more than fifty years, researchers have studied how humans physically explore and form perceptual representations of objects. Some of these works proposed the paradigm through which human haptic exploration is presently understood: humans use a particular set of exploratory procedures to elicit specific semantic attributes from objects. Others have sought to understand how physically measured object properties correspond to human perception of semantic attributes. Few, however, have investigated how specific explorations are perceived. As robots become increasingly advanced and more ubiquitous in daily life, they are beginning to be equipped with haptic sensing capabilities and algorithms for processing and structuring haptic information. Traditional haptics research has so far strongly influenced the introduction of haptic sensation and perception into robots but has not proven sufficient to give robots the necessary tools to become intelligent autonomous agents. The work presented in this thesis seeks to understand how single and sequential haptic interactions are perceived by both humans and robots. In our first study, we depart from the more traditional methods of studying human haptic perception and investigate how the physical sensations felt during single explorations are perceived by individual people. We treat interactions as probability distributions over a haptic feature space and train a model to predict how similarly a pair of surfaces is rated, predicting perceived similarity with a reasonable degree of accuracy. Our novel method also allows us to evaluate how individual people weigh different surface properties when they make perceptual judgments. The method is highly versatile and presents many opportunities for further studies into how humans form perceptual representations of specific explorations. Our next body of work explores how to improve robotic haptic perception of single interactions. We use unsupervised feature-learning methods to derive powerful features from raw robot sensor data and classify robot explorations into numerous haptic semantic property labels that were assigned from human ratings. Additionally, we provide robots with more nuanced perception by learning to predict graded ratings of a subset of properties. Our methods outperform previous attempts that all used hand-crafted features, demonstrating the limitations of such traditional approaches. To push robot haptic perception beyond evaluation of single explorations, our final work introduces and evaluates a method to give robots the ability to accumulate information over many sequential actions; our approach essentially takes advantage of object permanence by conditionally and recursively updating the representation of an object as it is sequentially explored. We implement our method on a robotic gripper platform that performs multiple exploratory procedures on each of many objects. As the robot explores objects with new procedures, it gains confidence in its internal representations and classification of object properties, thus moving closer to the marvelous haptic capabilities of humans and providing a solid foundation for future research in this domain.

hi

link (url) Project Page [BibTex]

hi Richardson, B. Multi-Timescale Representation Learning of Human and Robot Haptic Interactions University of Stuttgart, Stuttgart, Germany, December 2022, Faculty of Computer Science, Electrical Engineering and Information Technology (phdthesis)

link (url) Project Page [BibTex]

Hydraulically Amplified Self-healing Electrostatic Transducers Harnessing Zipping Mechanism
Hydraulically Amplified Self-healing Electrostatic Transducers Harnessing Zipping Mechanism

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K., Morrissey, T. G.

(US Patent 11486421B2), November 2022 (patent)

Abstract
Hydraulically-amplified, self-healing, electrostatic transducers that harness electrostatic and hydraulic forces to achieve various actuation modes. Electrostatic forces between electrode pairs of the transducers generated upon application of a voltage to the electrode pairs draws the electrodes in each pair towards each other to displace a liquid dielectric contained within an enclosed internal cavity of the transducers to drive actuation in various manners. The electrodes and the liquid dielectric form a self-healing capacitor whereby the liquid dielectric automatically fills breaches in the liquid dielectric resulting from dielectric breakdown. Due to the resting shape of the cavity, a zipping-mechanism allows for selectively actuating the electrodes to a desired extent by controlling the voltage supplied.

rm

link (url) [BibTex]

rm Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K., Morrissey, T. G. Hydraulically Amplified Self-healing Electrostatic Transducers Harnessing Zipping Mechanism (US Patent 11486421B2), November 2022 (patent)

link (url) [BibTex]

no image
Towards learning mechanistic models at the right level of abstraction

Neitz, A.

University of Tübingen, Germany, November 2022 (phdthesis)

ei

[BibTex]

ei Neitz, A. Towards learning mechanistic models at the right level of abstraction University of Tübingen, Germany, November 2022 (phdthesis)

[BibTex]

no image
Life Improvement Science

Lieder, F., Prentice, M.

In Encyclopedia of Quality of Life and Well-Being Research, Springer, November 2022 (inbook)

re

[BibTex]

re Lieder, F., Prentice, M. Life Improvement Science In Encyclopedia of Quality of Life and Well-Being Research, Springer, November 2022 (inbook)

[BibTex]

Hydraulically Amplified Self-Healing Electrostatic (HASEL) Pumps
Hydraulically Amplified Self-Healing Electrostatic (HASEL) Pumps

Mitchell, S. K., Acome, E. L., Keplinger, C. M.

(US Patent App. 17/635,339), October 2022 (patent)

Abstract
A pumping system includes a conduit with an inlet region and an outlet region and a first pump coupled with the conduit between the inlet region and the outlet region. The first pump includes a first actuator chamber configured to house at least a first actuator, a first pump chamber aligned along a longitudinal axis of the conduit, wherein the first pump chamber is in fluid communication with the inlet region and the outlet region, and a first flexible diaphragm separating the first actuator chamber from the first pump chamber. Methods for operating the pumping system are also disclosed.

rm

link (url) [BibTex]

rm Mitchell, S. K., Acome, E. L., Keplinger, C. M. Hydraulically Amplified Self-Healing Electrostatic (HASEL) Pumps (US Patent App. 17/635,339), October 2022 (patent)

link (url) [BibTex]

no image
Learning Causal Representations for Generalization and Adaptation in Supervised, Imitation, and Reinforcement Learning

Lu, C.

University of Cambridge, UK, Cambridge, October 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

ei Lu, C. Learning Causal Representations for Generalization and Adaptation in Supervised, Imitation, and Reinforcement Learning University of Cambridge, UK, Cambridge, October 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

[BibTex]

no image
Understanding the Influence of Moisture on Fingerpad-Surface Interactions

Nam, S.

University of Tübingen, Tübingen, Germany, October 2022, Department of Computer Science (phdthesis)

Abstract
People frequently touch objects with their fingers. The physical deformation of a finger pressing an object surface stimulates mechanoreceptors, resulting in a perceptual experience. Through interactions between perceptual sensations and motor control, humans naturally acquire the ability to manage friction under various contact conditions. Many researchers have advanced our understanding of human fingers to this point, but their complex structure and the variations in friction they experience due to continuously changing contact conditions necessitate additional study. Moisture is a primary factor that influences many aspects of the finger. In particular, sweat excreted from the numerous sweat pores on the fingerprints modifies the finger's material properties and the contact conditions between the finger and a surface. Measuring changes of the finger's moisture over time and in response to external stimuli presents a challenge for researchers, as commercial moisture sensors do not provide continuous measurements. This dissertation investigates the influence of moisture on fingerpad-surface interactions from diverse perspectives. First, we examine the extent to which moisture on the finger contributes to the sensation of stickiness during contact with glass. Second, we investigate the representative material properties of a finger at three distinct moisture levels, since the softness of human skin varies significantly with moisture. The third perspective is friction; we examine how the contact conditions, including the moisture of a finger, determine the available friction force opposing lateral sliding on glass. Fourth, we have invented and prototyped a transparent in vivo moisture sensor for the continuous measurement of finger hydration. In the first part of this dissertation, we explore how the perceptual intensity of light stickiness relates to the physical interaction between the skin and the surface. We conducted a psychophysical experiment in which nine participants actively pressed their index finger on a flat glass plate with a normal force close to 1.5 N and then detached it after a few seconds. A custom-designed apparatus recorded the contact force vector and the finger contact area during each interaction as well as pre- and post-trial finger moisture. After detaching their finger, participants judged the stickiness of the glass using a nine-point scale. We explored how sixteen physical variables derived from the recorded data correlate with each other and with the stickiness judgments of each participant. These analyses indicate that stickiness perception mainly depends on the pre-detachment pressing duration, the time taken for the finger to detach, and the impulse in the normal direction after the normal force changes sign; finger-surface adhesion seems to build with pressing time, causing a larger normal impulse during detachment and thus a more intense stickiness sensation. We additionally found a strong between-subjects correlation between maximum real contact area and peak pull-off force, as well as between finger moisture and impulse. When a fingerpad presses into a hard surface, the development of the contact area depends on the pressing force and speed. Importantly, it also varies with the finger's moisture, presumably because hydration changes the tissue's material properties. Therefore, for the second part of this dissertation, we collected data from one finger repeatedly pressing a glass plate under three moisture conditions, and we constructed a finite element model that we optimized to simulate the same three scenarios. We controlled the moisture of the subject's finger to be dry, natural, or moist and recorded 15 pressing trials in each condition. The measurements include normal force over time plus finger-contact images that are processed to yield gross contact area. We defined the axially symmetric 3D model's lumped parameters to include an SLS-Kelvin model (spring in series with parallel spring and damper) for the bulk tissue, plus an elastic epidermal layer. Particle swarm optimization was used to find the parameter values that cause the simulation to best match the trials recorded in each moisture condition. The results show that the softness of the bulk tissue reduces as the finger becomes more hydrated. The epidermis of the moist finger model is softest, while the natural finger model has the highest viscosity. In the third part of this dissertation, we focused on friction between the fingerpad and the surface. The magnitude of finger-surface friction available at the onset of full slip is crucial for understanding how the human hand can grip and manipulate objects. Related studies revealed the significance of moisture and contact time in enhancing friction. Recent research additionally indicated that surface temperature may also affect friction. However, previously reported friction coefficients have been measured only in dynamic contact conditions, where the finger is already sliding across the surface. In this study, we repeatedly measured the initial friction before full slip under eight contact conditions with low and high finger moisture, pressing time, and surface temperature. Moisture and pressing time both independently increased finger-surface friction across our population of twelve participants, and the effect of surface temperature depended on the contact conditions. Furthermore, detailed analysis of the recorded measurements indicates that micro stick-slip during the partial-slip phase contributes to enhanced friction. For the fourth and final part of this dissertation, we designed a transparent moisture sensor for continuous measurement of fingerpad hydration. Because various stimuli cause the sweat pores on fingerprints to excrete sweat, many researchers want to quantify the flow and assess its impact on the formation of the contact area. Unfortunately, the most popular sensor for skin hydration is opaque and does not offer continuous measurements. Our capacitive moisture sensor consists of a pair of inter-digital electrodes covered by an insulating layer, enabling impedance measurements across a wide frequency range. This proposed sensor is made entirely of transparent materials, which allows us to simultaneously measure the finger's contact area. Electrochemical impedance spectroscopy identifies the equivalent electrical circuit and the electrical component parameters that are affected by the amount of moisture present on the surface of the sensor. Most notably, the impedance at 1 kHz seems to best reflect the relative amount of sweat.

hi

DOI Project Page [BibTex]

hi Nam, S. Understanding the Influence of Moisture on Fingerpad-Surface Interactions University of Tübingen, Tübingen, Germany, October 2022, Department of Computer Science (phdthesis)

DOI Project Page [BibTex]

no image
Causality, causal digital twins, and their applications

Schölkopf, B.

Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling (Dagstuhl Seminar 22382), (Editors: Berens, Philipp and Cranmer, Kyle and Lawrence, Neil D. and von Luxburg, Ulrike and Montgomery, Jessica), September 2022 (talk)

ei

link (url) DOI [BibTex]

ei Schölkopf, B. Causality, causal digital twins, and their applications Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling (Dagstuhl Seminar 22382), (Editors: Berens, Philipp and Cranmer, Kyle and Lawrence, Neil D. and von Luxburg, Ulrike and Montgomery, Jessica), September 2022 (talk)

link (url) DOI [BibTex]

no image
Methods for Minimizing the Spread of Misinformation on the Web

Tabibian, B.

University of Tübingen, Germany, September 2022 (phdthesis)

ei

[BibTex]

ei Tabibian, B. Methods for Minimizing the Spread of Misinformation on the Web University of Tübingen, Germany, September 2022 (phdthesis)

[BibTex]

Hydraulically Amplified Self-healing Electrostatic Actuators
Hydraulically Amplified Self-healing Electrostatic Actuators

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K.

(US Patent 11408452), August 2022 (patent)

Abstract
An electro-hydraulic actuator includes a deformable shell defining an enclosed internal cavity and containing a liquid dielectric, first and second electrodes on first and second sides, respectively, of the enclosed internal cavity. An electrostatic force between the first and second electrodes upon application of a voltage to one of the electrodes draws the electrodes towards each other to displace the liquid dielectric within the enclosed internal cavity. The shell includes active and inactive areas such that the electrostatic forces between the first and second electrodes displaces the liquid dielectric within the enclosed internal cavity from the active area of the shell to the inactive area of the shell. The first and second electrodes, the deformable shell, and the liquid dielectric cooperate to form a self-healing capacitor, and the liquid dielectric is configured for automatically filling breaches in the liquid dielectric resulting from dielectric breakdown.

rm

link (url) [BibTex]

rm Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K. Hydraulically Amplified Self-healing Electrostatic Actuators (US Patent 11408452), August 2022 (patent)

link (url) [BibTex]

no image
Learning and Using Causal Knowledge: A Further Step Towards a Higher-Level Intelligence

Huang, B.

Carnegie Mellon University, Pittsburgh, USA, July 2022 (phdthesis)

ei

[BibTex]

ei Huang, B. Learning and Using Causal Knowledge: A Further Step Towards a Higher-Level Intelligence Carnegie Mellon University, Pittsburgh, USA, July 2022 (phdthesis)

[BibTex]

Composite Layering of Hydraulically Amplified Self-Healing Electrostatic Transducers
Composite Layering of Hydraulically Amplified Self-Healing Electrostatic Transducers

Keplinger, C. M., Mitchell, S. K., Kellaris, N. A., Rothemund, P.

(US Patent App. 17436455), May 2022 (patent)

Abstract
A hydraulically amplified self-healing electrostatic (HASEL) transducer includes a composite, multi-layered structure. In an example, a HASEL transducer includes a dielectric layer including at least one fluid dielectric layer. The dielectric layer includes a first side and a second side opposing the first side. The HASEL transducer further includes a first electrode disposed at the first side of the dielectric layer, a second electrode disposed at the second side of the dielectric layer, a first outer layer disposed at the first electrode opposite the dielectric layer, and a second outer layer disposed at the second electrode opposite the dielectric layer. The first outer layer and second outer layer exhibit different mechanical and electrical properties from the dielectric layer.

rm

link (url) [BibTex]

rm Keplinger, C. M., Mitchell, S. K., Kellaris, N. A., Rothemund, P. Composite Layering of Hydraulically Amplified Self-Healing Electrostatic Transducers (US Patent App. 17436455), May 2022 (patent)

link (url) [BibTex]

no image
Proceedings of the First Conference on Causal Learning and Reasoning (CLeaR 2022)

Schölkopf, B., Uhler, C., Zhang, K.

177, Proceedings of Machine Learning Research, PMLR, April 2022 (proceedings)

ei

link (url) [BibTex]

ei Schölkopf, B., Uhler, C., Zhang, K. Proceedings of the First Conference on Causal Learning and Reasoning (CLeaR 2022) 177, Proceedings of Machine Learning Research, PMLR, April 2022 (proceedings)

link (url) [BibTex]

no image
Variational Inference in Dynamical Systems

Ialongo, A.

University of Cambridge, UK, Cambridge, February 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

ei Ialongo, A. Variational Inference in Dynamical Systems University of Cambridge, UK, Cambridge, February 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

[BibTex]

no image
Entwicklung von Methoden und Bausteinen zur Realisierung Komplexer Magnonischer Systeme

Groß, F.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), 2022 (phdthesis)

mms

link (url) [BibTex]

mms Groß, F. Entwicklung von Methoden und Bausteinen zur Realisierung Komplexer Magnonischer Systeme Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), 2022 (phdthesis)

link (url) [BibTex]

no image
Machine-Learning-Driven Haptic Sensor Design

Sun, H.

University of Tuebingen, Library, 2022 (phdthesis)

Abstract
Similar to biological systems, robots may need skin-like sensing ability to perceive interactions in complex, changing, and human-involved environments. Current skin-like sensing technologies are still far behind their biological counterparts when considering resolution, dynamics range, robustness, and surface coverage together. One key challenge is the wiring of sensing elements. During my Ph.D. study, I explore how machine learning can enable the design of a new kind of haptic sensors to deal with such a challenge. On the one hand, I propose super-resolution-oriented tactile skins, reducing the number of physical sensing elements while achieving high spatial accuracy. On the other hand, I explore vision-based haptic sensor designs. In this thesis, I present four types of machine-learning-driven haptic sensors that I designed for coarse and fine robotic applications, varying from large surface (robot limbs) to small surface sensing (robot fingers). Moreover, I propose a super-resolution theory to guide sensor designs at all levels ranging from hardware design (material/structure/transduction), data collection (real/simulated), and signal processing methods (analytical/data-driven). I investigate two designs for large-scale coarse-resolution sensing, e.g., robotic limbs. HapDef sparsely attaches a few strain gauges on a large curved surface internally to measure the deformation over the whole surface. ERT-DNN wraps a large surface with a piece of multi-layered conductive fabric, which varies its conductivity upon contacts exerted. I also conceive two approaches for small-scale fine-resolution sensing, e.g., robotic fingertips. BaroDome sparsely embeds a few barometers inside a soft elastomer to measure internal pressure changes caused by external contact. Insight encloses a high-resolution camera to view a soft shell from within. Generically, an inverse problem needs to be solved when trying to obtain high-resolution sensing with a few physical sensing elements. I develop machine-learning frameworks suitable for solving this inverse problem. They process various raw sensor data and extract useful haptic information in practice. Machine learning methods rely on data collected by an automated robotic stimulation device or synthesized using finite element methods. I build several physical testbeds and finite element models to collect copious data. I propose machine learning frameworks to combine data from different sources that are good enough to deal with the noise in real data and generalize well from seen to unseen situations. While developing my prototype sensors, I have faced reoccurring design choices. To help my developments and guide future research, I propose a unified theory with the concept of taxel-value-isolines. It captures the physical effects required for super-resolution, ties them to all parts of the sensor design, and allows us to assess them quantitatively. The theory offers an explanation about physically achievable accuracies for localizing and quantifying contact based on uncertainties introduced by measurement noise in sensor elements. The theoretical analysis aims to predict the best performance before a physical prototype is built and helps to evaluate the hardware design, data collection, and data processing methods during implementation. This thesis presents a new perspective on haptic sensor design. Using machine learning to substitute the entire data-processing pipeline, I present several haptic sensor designs for applications ranging from large-surface skins to high-resolution tactile fingertip sensors. The developed theory for obtaining optimal super-resolution can guide future sensor designs.

al

link (url) [BibTex]

al Sun, H. Machine-Learning-Driven Haptic Sensor Design University of Tuebingen, Library, 2022 (phdthesis)

link (url) [BibTex]

no image
Causal Models for Dynamical Systems

Peters, J., Bauer, S., Pfister, N.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 671-690, 1, Association for Computing Machinery, 2022 (inbook)

ei

arXiv DOI [BibTex]

ei Peters, J., Bauer, S., Pfister, N. Causal Models for Dynamical Systems In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 671-690, 1, Association for Computing Machinery, 2022 (inbook)

arXiv DOI [BibTex]

no image
Towards Causal Algorithmic Recourse

Karimi, A. H., von Kügelgen, J., Schölkopf, B., Valera, I.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 139-166, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei plg

DOI [BibTex]

ei plg Karimi, A. H., von Kügelgen, J., Schölkopf, B., Valera, I. Towards Causal Algorithmic Recourse In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 139-166, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

DOI [BibTex]

no image
Causality for Machine Learning

Schölkopf, B.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 765-804, 1, Association for Computing Machinery, New York, NY, USA, 2022 (inbook)

ei

arXiv DOI [BibTex]

ei Schölkopf, B. Causality for Machine Learning In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 765-804, 1, Association for Computing Machinery, New York, NY, USA, 2022 (inbook)

arXiv DOI [BibTex]

no image
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations

Salewski, L., Koepke, A. S., Lensch, H. P. A., Akata, Z.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 69-88, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei

DOI [BibTex]

ei Salewski, L., Koepke, A. S., Lensch, H. P. A., Akata, Z. CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 69-88, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

DOI [BibTex]

2021

Magnetic Micro-/Nanopropellers for Biomedicine
Magnetic Micro-/Nanopropellers for Biomedicine

Qiu, T., Jeong, M., Goyal, R., Kadiri, V., Sachs, J., Fischer, P.

In Field-Driven Micro and Nanorobots for Biology and Medicine, pages: 389-410, 16, (Editors: Sun, Y. and Wang, X. and Yu, J.), Springer Nature, November 2021 (inbook)

Abstract
In nature, many bacteria swim by rotating their helical flagella. A particularly promising class of artificial micro- and nano-robots mimic this propeller-like propulsion mechanism to move through fluids and tissues for applications in minimally-invasive medicine. Several fundamental challenges have to be overcome in order to build micro-machines that move similar to bacteria for in vivo applications. Here, we review recent advances of magnetically-powered micro-/nano-propellers. Four important aspects of the propellers – the geometrical shape, the fabrication method, the generation of magnetic fields for actuation, and the choice of biocompatible magnetic materials – are highlighted. First, the fundamental requirements are elucidated that arise due to hydrodynamics at low Reynolds (Re) number. We discuss the role that the propellers’ shape and symmetry play in realizing effective propulsion at low Re. Second, the additive nano-fabrication method Glancing Angle Deposition is discussed as a versatile technique to quickly grow large numbers of designer nano-helices. Third, systems to generate rotating magnetic fields via permanent magnets or electromagnetic coils are presented. And finally, the biocompatibility of the magnetic materials is discussed. Iron-platinum is highlighted due to its biocompatibility and its superior magnetic properties, which is promising for targeted delivery, minimally-invasive magnetic nano-devices and biomedical applications.

pf

link (url) DOI [BibTex]

2021

pf Qiu, T., Jeong, M., Goyal, R., Kadiri, V., Sachs, J., Fischer, P. Magnetic Micro-/Nanopropellers for Biomedicine In Field-Driven Micro and Nanorobots for Biology and Medicine, pages: 389-410, 16, (Editors: Sun, Y. and Wang, X. and Yu, J.), Springer Nature, November 2021 (inbook)

link (url) DOI [BibTex]

no image
Models for Data-Efficient Reinforcement Learning on Real-World Applications

Doerr, A.

University of Stuttgart, Stuttgart, October 2021 (phdthesis)

ics

DOI [BibTex]

ics Doerr, A. Models for Data-Efficient Reinforcement Learning on Real-World Applications University of Stuttgart, Stuttgart, October 2021 (phdthesis)

DOI [BibTex]

no image
Physically Plausible Tracking & Reconstruction of Dynamic Objects

Strecke, M., Stückler, J.

KIT Science Week Scientific Conference & DGR-Days 2021, October 2021 (talk)

ev

[BibTex]

ev Strecke, M., Stückler, J. Physically Plausible Tracking & Reconstruction of Dynamic Objects KIT Science Week Scientific Conference & DGR-Days 2021, October 2021 (talk)

[BibTex]

no image
Dynamics of Learning and Learning of Dynamics

Mehrjou, A.

ETH Zürich, Zürich, October 2021 (phdthesis)

ei

DOI [BibTex]

ei Mehrjou, A. Dynamics of Learning and Learning of Dynamics ETH Zürich, Zürich, October 2021 (phdthesis)

DOI [BibTex]

no image
A Large Scale Brain-Computer Interface for Patients with Neurological Diseases

Hohmann, M.

University of Tübingen, Germany, September 2021 (phdthesis)

ei

[BibTex]

ei Hohmann, M. A Large Scale Brain-Computer Interface for Patients with Neurological Diseases University of Tübingen, Germany, September 2021 (phdthesis)

[BibTex]

no image
Deep Learning Beyond The Training Distribution

Parascandolo, G.

ETH Zürich, Switzerland, Zürich, September 2021, (CLS Fellowship Program) (phdthesis)

ei

DOI [BibTex]

ei Parascandolo, G. Deep Learning Beyond The Training Distribution ETH Zürich, Switzerland, Zürich, September 2021, (CLS Fellowship Program) (phdthesis)

DOI [BibTex]

Skinned multi-infant linear body model
Skinned multi-infant linear body model

Hesse, N., Pujades, S., Romero, J., Black, M.

(US Patent 11,127,163, 2021), September 2021 (patent)

Abstract
A computer-implemented method for automatically obtaining pose and shape parameters of a human body. The method includes obtaining a sequence of digital 3D images of the body, recorded by at least one depth camera; automatically obtaining pose and shape parameters of the body, based on images of the sequence and a statistical body model; and outputting the pose and shape parameters. The body may be an infant body.

ps

[BibTex]

ps Hesse, N., Pujades, S., Romero, J., Black, M. Skinned multi-infant linear body model (US Patent 11,127,163, 2021), September 2021 (patent)

[BibTex]