Header logo is


2024


no image
Identifiable Causal Representation Learning

von Kügelgen, J.

University of Cambridge, UK, Cambridge, February 2024, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

2024


[BibTex]


Creating a Haptic Empathetic Robot Animal That Feels Touch and Emotion
Creating a Haptic Empathetic Robot Animal That Feels Touch and Emotion

Burns, R.

University of Tübingen, Tübingen, Germany, February 2024, Department of Computer Science (phdthesis)

Abstract
Social touch, such as a hug or a poke on the shoulder, is an essential aspect of everyday interaction. Humans use social touch to gain attention, communicate needs, express emotions, and build social bonds. Despite its importance, touch sensing is very limited in most commercially available robots. By endowing robots with social-touch perception, one can unlock a myriad of new interaction possibilities. In this thesis, I present my work on creating a Haptic Empathetic Robot Animal (HERA), a koala-like robot for children with autism. I demonstrate the importance of establishing design guidelines based on one's target audience, which we investigated through interviews with autism specialists. I share our work on creating full-body tactile sensing for the NAO robot using low-cost, do-it-yourself (DIY) methods, and I introduce an approach to model long-term robot emotions using second-order dynamics.

hi

Project Page [BibTex]

Project Page [BibTex]

2023


Hydraulically Amplified Self-healing Electrostatic Actuators
Hydraulically Amplified Self-healing Electrostatic Actuators

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K.

(US Patent 11795979B2), October 2023 (patent)

Abstract
An electro-hydraulic actuator includes a deformable shell defining an enclosed internal cavity and containing a liquid dielectric, first and second electrodes on first and second sides, respectively, of the enclosed internal cavity. An electrostatic force between the first and second electrodes upon application of a voltage to one of the electrodes draws the electrodes towards each other to displace the liquid dielectric within the enclosed internal cavity. The shell includes active and inactive areas such that the electrostatic forces between the first and second electrodes displaces the liquid dielectric within the enclosed internal cavity from the active area of the shell to the inactive area of the shell. The first and second electrodes, the deformable shell, and the liquid dielectric cooperate to form a self-healing capacitor, and the liquid dielectric is configured for automatically filling breaches in the liquid dielectric resulting from dielectric breakdown.

rm

link (url) [BibTex]

2023


link (url) [BibTex]


no image
Gesture-Based Nonverbal Interaction for Exercise Robots

Mohan, M.

University of Tübingen, Tübingen, Germany, October 2023, Department of Computer Science (phdthesis)

Abstract
When teaching or coaching, humans augment their words with carefully timed hand gestures, head and body movements, and facial expressions to provide feedback to their students. Robots, however, rarely utilize these nuanced cues. A minimally supervised social robot equipped with these abilities could support people in exercising, physical therapy, and learning new activities. This thesis examines how the intuitive power of human gestures can be harnessed to enhance human-robot interaction. To address this question, this research explores gesture-based interactions to expand the capabilities of a socially assistive robotic exercise coach, investigating the perspectives of both novice users and exercise-therapy experts. This thesis begins by concentrating on the user's engagement with the robot, analyzing the feasibility of minimally supervised gesture-based interactions. This exploration seeks to establish a framework in which robots can interact with users in a more intuitive and responsive manner. The investigation then shifts its focus toward the professionals who are integral to the success of these innovative technologies: the exercise-therapy experts. Roboticists face the challenge of translating the knowledge of these experts into robotic interactions. We address this challenge by developing a teleoperation algorithm that can enable exercise therapists to create customized gesture-based interactions for a robot. Thus, this thesis lays the groundwork for dynamic gesture-based interactions in minimally supervised environments, with implications for not only exercise-coach robots but also broader applications in human-robot interaction.

hi

Project Page [BibTex]

Project Page [BibTex]


High Strain Peano Hydraulically Amplified Self-Healing Electrostatic (HASEL) Transducers
High Strain Peano Hydraulically Amplified Self-Healing Electrostatic (HASEL) Transducers

Keplinger, C. M., Wang, X., Mitchell, S. K.

(US Patent App. 18/138,621), August 2023 (patent)

Abstract
High strain hydraulically amplified self-healing electrostatic transducers having increased maximum theoretical and practical strains are disclosed. In particular, the actuators include electrode configurations having a zipping front created by the attraction of the electrodes that is configured orthogonally to a strain axis along which the actuators. This configuration produces increased strains. In turn, various form factors for the actuator configuration are presented including an artificial circular muscle and a strain amplifying pulley system. Other actuator configurations are contemplated that include independent and opposed electrode pairs to create cyclic activation, hybrid electrode configurations, and use of strain limiting layers for controlled deflection of the actuator.

rm

link (url) [BibTex]


no image
Learning and Testing Powerful Hypotheses

Kübler, J. M.

University of Tübingen, Germany, July 2023 (phdthesis)

ei

[BibTex]

[BibTex]


Capacitive Self-Sensing for Electrostatic Transducers with High Voltage Isolation
Capacitive Self-Sensing for Electrostatic Transducers with High Voltage Isolation

Correll, N., Ly, K. D., Kellaris, N. A., Keplinger, C. M.

(US Patent App. 17/928,453), June 2023 (patent)

Abstract
Transducer systems disclosed herein include self-sensing capabilities. In particular, electrostatic transducers include a low voltage electrode and a high voltage electrode. A low voltage sensing unit is coupled with the low voltage electrode of the electrostatic transducer. The low voltage sensing unit is configured to measure a capacitance of the electrostatic transducer, from which displacement of the electrostatic transducer may be calculated. High voltage drive signals received by the high voltage electrode during actuation may be isolated from the low voltage sensing unit. The isolation may be provided by dielectric material of the electrostatic transducer, a voltage suppression component, and/or a voltage suppression module comprising a low impedance ground path. In the event of an electrical failure of the transducer, the low voltage sensing unit may be isolated from high voltages.

rm

link (url) [BibTex]

link (url) [BibTex]


no image
Learning Identifiable Representations: Independent Influences and Multiple Views

Gresele, L.

University of Tübingen, Germany, June 2023 (phdthesis)

ei

[BibTex]


no image
Learning with and for discrete optimization

Paulus, M.

(ETH Zurich, Switzerland), May 2023, CLS PhD Program (phdthesis)

ei

[BibTex]

[BibTex]


High Strain Peano Hydraulically Amplified Self-healing Electrostatic (HASEL) Transducers
High Strain Peano Hydraulically Amplified Self-healing Electrostatic (HASEL) Transducers

Keplinger, C. M., Wang, X., Mitchell, S. K.

(US Patent 11635094), April 2023 (patent)

Abstract
High strain hydraulically amplified self-healing electrostatic transducers having increased maximum theoretical and practical strains are disclosed. In particular, the actuators include electrode configurations having a zipping front created by the attraction of the electrodes that is configured orthogonally to a strain axis along which the actuators. This configuration produces increased strains. In turn, various form factors for the actuator configuration are presented including an artificial circular muscle and a strain amplifying pulley system. Other actuator configurations are contemplated that include independent and opposed electrode pairs to create cyclic activation, hybrid electrode configurations, and use of strain limiting layers for controlled deflection of the actuator.

rm

link (url) [BibTex]


no image
Object-Level Dynamic Scene Reconstruction With Physical Plausibility From RGB-D Images

Strecke, M. F.

Eberhard Karls Universität Tübingen, Tübingen, 2023 (phdthesis)

Abstract
Humans have the remarkable ability to perceive and interact with objects in the world around them. They can easily segment objects from visual data and have an intuitive understanding of how physics influences objects. By contrast, robots are so far often constrained to tailored environments for a specific task, due to their inability to reconstruct a versatile and accurate scene representation. In this thesis, we combine RGB-D video data with background knowledge of real-world physics to develop such a representation for robots.

Our contributions can be separated into two main parts: a dynamic object tracking tool and optimization frameworks that allow for improving shape reconstructions based on physical plausibility. The dynamic object tracking tool "EM-Fusion" detects, segments, reconstructs, and tracks objects from RGB-D video data. We propose a probabilistic data association approach for attributing the image pixels to the different moving objects in the scene. This allows us to track and reconstruct moving objects and the background scene with state-of-the art accuracy and robustness towards occlusions.

We investigate two ways of further optimizing the reconstructed shapes of moving objects based on physical plausibility. The first of these, "Co-Section", includes physical plausibility by reasoning about the empty space around an object. We observe that no two objects can occupy the same space at the same time and that the depth images in the input video provide an estimate of observed empty space. Based on these observations, we propose intersection and hull constraints, which we combine with the observed surfaces in a global optimization approach. Compared to EM-Fusion, which only reconstructs the observed surface, Co-Section optimizes watertight shapes. These watertight shapes provide a rough estimate of unseen surfaces and could be useful as initialization for further refinement, e.g., by interactive perception. In the second optimization approach, "DiffSDFSim", we reason about object shapes based on physically plausible object motion. We observe that object trajectories after collisions depend on the object's shape, and extend a differentiable physics simulation for optimizing object shapes together with other physical properties (e.g., forces, masses, friction) based on the motion of the objects and their interactions. Our key contributions are using signed distance function models for representing shapes and a novel method for computing gradients that models the dependency of the time of contact on object shapes. We demonstrate that our approach recovers target shapes well by fitting to target trajectories and depth observations. Further, the ground-truth trajectories are recovered well in simulation using the resulting shape and physical properties. This enables predictions about the future motion of objects by physical simulation.

We anticipate that our contributions can be useful building blocks in the development of 3D environment perception for robots. The reconstruction of individual objects as in EM-Fusion is a key ingredient required for interactions with objects. Completed shapes as the ones provided by Co-Section provide useful cues for planning interactions like grasping of objects. Finally, the recovery of shape and other physical parameters using differentiable simulation as in DiffSDFSim allows simulating objects and thus predicting the effects of interactions. Future work might extend the presented works for interactive perception of dynamic environments by comparing these predictions with observed real-world interactions to further improve the reconstructions and physical parameter estimations.

ev

link (url) DOI [BibTex]


no image
Microfibers with mushroom-shaped tips for optimal adhesion

Sitti, M., Aksak, B.

2023, US Patent 11,613,674 (patent)

pi

[BibTex]

[BibTex]


Magnetic trap system and method of navigating a microscopic device
Magnetic trap system and method of navigating a microscopic device

Son, D., Ugurlu, M., Bluemer, P., Sitti, M.

2023, US Patent App. 17/871,598 (patent)

pi

[BibTex]


no image
Static and dynamic investigation of magnonic systems: materials, applications and modeling

Schulz, Frank Martin Ernst

Universität Stuttgart, Stuttgart, 2023 (phdthesis)

mms

link (url) DOI [BibTex]

link (url) DOI [BibTex]

2022


no image
DRY ADHESIVES AND METHODS FOR MAKING DRY ADHESIVES

Metin Sitti, Michael Murphy, Burak Aksak

December 2022, US Patent App. 17/895,334, 2022 (patent)

pi

[BibTex]

2022


[BibTex]


Reconstructing Expressive {3D} Humans from {RGB} Images
Reconstructing Expressive 3D Humans from RGB Images

Choutas, V.

ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

Abstract
To interact with our environment, we need to adapt our body posture and grasp objects with our hands. During a conversation our facial expressions and hand gestures convey important non-verbal cues about our emotional state and intentions towards our fellow speakers. Thus, modeling and capturing 3D full-body shape and pose, hand articulation and facial expressions are necessary to create realistic human avatars for augmented and virtual reality. This is a complex task, due to the large number of degrees of freedom for articulation, body shape variance, occlusions from objects and self-occlusions from body parts, e.g. crossing our hands, and subject appearance. The community has thus far relied on expensive and cumbersome equipment, such as multi-view cameras or motion capture markers, to capture the 3D human body. While this approach is effective, it is limited to a small number of subjects and indoor scenarios. Using monocular RGB cameras would greatly simplify the avatar creation process, thanks to their lower cost and ease of use. These advantages come at a price though, since RGB capture methods need to deal with occlusions, perspective ambiguity and large variations in subject appearance, in addition to all the challenges posed by full-body capture. In an attempt to simplify the problem, researchers generally adopt a divide-and-conquer strategy, estimating the body, face and hands with distinct methods using part-specific datasets and benchmarks. However, the hands and face constrain the body and vice-versa, e.g. the position of the wrist depends on the elbow, shoulder, etc.; the divide-and-conquer approach can not utilize this constraint. In this thesis, we aim to reconstruct the full 3D human body, using only readily accessible monocular RGB images. In a first step, we introduce a parametric 3D body model, called SMPL-X, that can represent full-body shape and pose, hand articulation and facial expression. Next, we present an iterative optimization method, named SMPLify-X, that fits SMPL-X to 2D image keypoints. While SMPLify-X can produce plausible results if the 2D observations are sufficiently reliable, it is slow and susceptible to initialization. To overcome these limitations, we introduce ExPose, a neural network regressor, that predicts SMPL-X parameters from an image using body-driven attention, i.e. by zooming in on the hands and face, after predicting the body. From the zoomed-in part images, dedicated part networks predict the hand and face parameters. ExPose combines the independent body, hand, and face estimates by trusting them equally. This approach though does not fully exploit the correlation between parts and fails in the presence of challenges such as occlusion or motion blur. Thus, we need a better mechanism to aggregate information from the full body and part images. PIXIE uses neural networks called moderators that learn to fuse information from these two image sets before predicting the final part parameters. Overall, the addition of the hands and face leads to noticeably more natural and expressive reconstructions. Creating high fidelity avatars from RGB images requires accurate estimation of 3D body shape. Although existing methods are effective at predicting body pose, they struggle with body shape. We identify the lack of proper training data as the cause. To overcome this obstacle, we propose to collect internet images from fashion models websites, together with anthropometric measurements. At the same time, we ask human annotators to rate images and meshes according to a pre-defined set of linguistic attributes. We then define mappings between measurements, linguistic shape attributes and 3D body shape. Equipped with these mappings, we train a neural network regressor, SHAPY, that predicts accurate 3D body shapes from a single RGB image. We observe that existing 3D shape benchmarks lack subject variety and/or ground-truth shape. Thus, we introduce a new benchmark, Human Bodies in the Wild (HBW), which contains images of humans and their corresponding 3D ground-truth body shape. SHAPY shows how we can overcome the lack of in-the-wild images with 3D shape annotations through easy-to-obtain anthropometric measurements and linguistic shape attributes. Regressors that estimate 3D model parameters are robust and accurate, but often fail to tightly fit the observations. Optimization-based approaches tightly fit the data, by minimizing an energy function composed of a data term that penalizes deviations from the observations and priors that encode our knowledge of the problem. Finding the balance between these terms and implementing a performant version of the solver is a time-consuming and non-trivial task. Machine-learned continuous optimizers combine the benefits of both regression and optimization approaches. They learn the priors directly from data, avoiding the need for hand-crafted heuristics and loss term balancing, and benefit from optimized neural network frameworks for fast inference. Inspired from the classic Levenberg-Marquardt algorithm, we propose a neural optimizer that outperforms classic optimization, regression and hybrid optimization-regression approaches. Our proposed update rule uses a weighted combination of gradient descent and a network-predicted update. To show the versatility of the proposed method, we apply it on three other problems, namely full body estimation from (i) 2D keypoints, (ii) head and hand location from a head-mounted device and (iii) face tracking from dense 2D landmarks. Our method can easily be applied to new model fitting problems and offers a competitive alternative to well-tuned traditional model fitting pipelines, both in terms of accuracy and speed. To summarize, we propose a new and richer representation of the human body, SMPL-X, that is able to jointly model the 3D human body pose and shape, facial expressions and hand articulation. We propose methods, SMPLify-X, ExPose and PIXIE that estimate SMPL-X parameters from monocular RGB images, progressively improving the accuracy and realism of the predictions. To further improve reconstruction fidelity, we demonstrate how we can use easy-to-collect internet data and human annotations to overcome the lack of 3D shape data and train a model, SHAPY, that predicts accurate 3D body shape from a single RGB image. Finally, we propose a flexible learnable update rule for parametric human model fitting that outperforms both classic optimization and neural network approaches. This approach is easily applicable to a variety of problems, unlocking new applications in AR/VR scenarios.

ps

pdf [BibTex]

pdf [BibTex]


no image
Multi-Timescale Representation Learning of Human and Robot Haptic Interactions

Richardson, B.

University of Stuttgart, Stuttgart, Germany, December 2022, Faculty of Computer Science, Electrical Engineering and Information Technology (phdthesis)

Abstract
The sense of touch is one of the most crucial components of the human sensory system. It allows us to safely and intelligently interact with the physical objects and environment around us. By simply touching or dexterously manipulating an object, we can quickly infer a multitude of its properties. For more than fifty years, researchers have studied how humans physically explore and form perceptual representations of objects. Some of these works proposed the paradigm through which human haptic exploration is presently understood: humans use a particular set of exploratory procedures to elicit specific semantic attributes from objects. Others have sought to understand how physically measured object properties correspond to human perception of semantic attributes. Few, however, have investigated how specific explorations are perceived. As robots become increasingly advanced and more ubiquitous in daily life, they are beginning to be equipped with haptic sensing capabilities and algorithms for processing and structuring haptic information. Traditional haptics research has so far strongly influenced the introduction of haptic sensation and perception into robots but has not proven sufficient to give robots the necessary tools to become intelligent autonomous agents. The work presented in this thesis seeks to understand how single and sequential haptic interactions are perceived by both humans and robots. In our first study, we depart from the more traditional methods of studying human haptic perception and investigate how the physical sensations felt during single explorations are perceived by individual people. We treat interactions as probability distributions over a haptic feature space and train a model to predict how similarly a pair of surfaces is rated, predicting perceived similarity with a reasonable degree of accuracy. Our novel method also allows us to evaluate how individual people weigh different surface properties when they make perceptual judgments. The method is highly versatile and presents many opportunities for further studies into how humans form perceptual representations of specific explorations. Our next body of work explores how to improve robotic haptic perception of single interactions. We use unsupervised feature-learning methods to derive powerful features from raw robot sensor data and classify robot explorations into numerous haptic semantic property labels that were assigned from human ratings. Additionally, we provide robots with more nuanced perception by learning to predict graded ratings of a subset of properties. Our methods outperform previous attempts that all used hand-crafted features, demonstrating the limitations of such traditional approaches. To push robot haptic perception beyond evaluation of single explorations, our final work introduces and evaluates a method to give robots the ability to accumulate information over many sequential actions; our approach essentially takes advantage of object permanence by conditionally and recursively updating the representation of an object as it is sequentially explored. We implement our method on a robotic gripper platform that performs multiple exploratory procedures on each of many objects. As the robot explores objects with new procedures, it gains confidence in its internal representations and classification of object properties, thus moving closer to the marvelous haptic capabilities of humans and providing a solid foundation for future research in this domain.

hi

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


Hydraulically Amplified Self-healing Electrostatic Transducers Harnessing Zipping Mechanism
Hydraulically Amplified Self-healing Electrostatic Transducers Harnessing Zipping Mechanism

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K., Morrissey, T. G.

(US Patent 11486421B2), November 2022 (patent)

Abstract
Hydraulically-amplified, self-healing, electrostatic transducers that harness electrostatic and hydraulic forces to achieve various actuation modes. Electrostatic forces between electrode pairs of the transducers generated upon application of a voltage to the electrode pairs draws the electrodes in each pair towards each other to displace a liquid dielectric contained within an enclosed internal cavity of the transducers to drive actuation in various manners. The electrodes and the liquid dielectric form a self-healing capacitor whereby the liquid dielectric automatically fills breaches in the liquid dielectric resulting from dielectric breakdown. Due to the resting shape of the cavity, a zipping-mechanism allows for selectively actuating the electrodes to a desired extent by controlling the voltage supplied.

rm

link (url) [BibTex]

link (url) [BibTex]


no image
Towards learning mechanistic models at the right level of abstraction

Neitz, A.

University of Tübingen, Germany, November 2022 (phdthesis)

ei

[BibTex]

[BibTex]


Hydraulically Amplified Self-Healing Electrostatic (HASEL) Pumps
Hydraulically Amplified Self-Healing Electrostatic (HASEL) Pumps

Mitchell, S. K., Acome, E. L., Keplinger, C. M.

(US Patent App. 17/635,339), October 2022 (patent)

Abstract
A pumping system includes a conduit with an inlet region and an outlet region and a first pump coupled with the conduit between the inlet region and the outlet region. The first pump includes a first actuator chamber configured to house at least a first actuator, a first pump chamber aligned along a longitudinal axis of the conduit, wherein the first pump chamber is in fluid communication with the inlet region and the outlet region, and a first flexible diaphragm separating the first actuator chamber from the first pump chamber. Methods for operating the pumping system are also disclosed.

rm

link (url) [BibTex]

link (url) [BibTex]


no image
Learning Causal Representations for Generalization and Adaptation in Supervised, Imitation, and Reinforcement Learning

Lu, C.

University of Cambridge, UK, Cambridge, October 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

[BibTex]


no image
Understanding the Influence of Moisture on Fingerpad-Surface Interactions

Nam, S.

University of Tübingen, Tübingen, Germany, October 2022, Department of Computer Science (phdthesis)

Abstract
People frequently touch objects with their fingers. The physical deformation of a finger pressing an object surface stimulates mechanoreceptors, resulting in a perceptual experience. Through interactions between perceptual sensations and motor control, humans naturally acquire the ability to manage friction under various contact conditions. Many researchers have advanced our understanding of human fingers to this point, but their complex structure and the variations in friction they experience due to continuously changing contact conditions necessitate additional study. Moisture is a primary factor that influences many aspects of the finger. In particular, sweat excreted from the numerous sweat pores on the fingerprints modifies the finger's material properties and the contact conditions between the finger and a surface. Measuring changes of the finger's moisture over time and in response to external stimuli presents a challenge for researchers, as commercial moisture sensors do not provide continuous measurements. This dissertation investigates the influence of moisture on fingerpad-surface interactions from diverse perspectives. First, we examine the extent to which moisture on the finger contributes to the sensation of stickiness during contact with glass. Second, we investigate the representative material properties of a finger at three distinct moisture levels, since the softness of human skin varies significantly with moisture. The third perspective is friction; we examine how the contact conditions, including the moisture of a finger, determine the available friction force opposing lateral sliding on glass. Fourth, we have invented and prototyped a transparent in vivo moisture sensor for the continuous measurement of finger hydration. In the first part of this dissertation, we explore how the perceptual intensity of light stickiness relates to the physical interaction between the skin and the surface. We conducted a psychophysical experiment in which nine participants actively pressed their index finger on a flat glass plate with a normal force close to 1.5 N and then detached it after a few seconds. A custom-designed apparatus recorded the contact force vector and the finger contact area during each interaction as well as pre- and post-trial finger moisture. After detaching their finger, participants judged the stickiness of the glass using a nine-point scale. We explored how sixteen physical variables derived from the recorded data correlate with each other and with the stickiness judgments of each participant. These analyses indicate that stickiness perception mainly depends on the pre-detachment pressing duration, the time taken for the finger to detach, and the impulse in the normal direction after the normal force changes sign; finger-surface adhesion seems to build with pressing time, causing a larger normal impulse during detachment and thus a more intense stickiness sensation. We additionally found a strong between-subjects correlation between maximum real contact area and peak pull-off force, as well as between finger moisture and impulse. When a fingerpad presses into a hard surface, the development of the contact area depends on the pressing force and speed. Importantly, it also varies with the finger's moisture, presumably because hydration changes the tissue's material properties. Therefore, for the second part of this dissertation, we collected data from one finger repeatedly pressing a glass plate under three moisture conditions, and we constructed a finite element model that we optimized to simulate the same three scenarios. We controlled the moisture of the subject's finger to be dry, natural, or moist and recorded 15 pressing trials in each condition. The measurements include normal force over time plus finger-contact images that are processed to yield gross contact area. We defined the axially symmetric 3D model's lumped parameters to include an SLS-Kelvin model (spring in series with parallel spring and damper) for the bulk tissue, plus an elastic epidermal layer. Particle swarm optimization was used to find the parameter values that cause the simulation to best match the trials recorded in each moisture condition. The results show that the softness of the bulk tissue reduces as the finger becomes more hydrated. The epidermis of the moist finger model is softest, while the natural finger model has the highest viscosity. In the third part of this dissertation, we focused on friction between the fingerpad and the surface. The magnitude of finger-surface friction available at the onset of full slip is crucial for understanding how the human hand can grip and manipulate objects. Related studies revealed the significance of moisture and contact time in enhancing friction. Recent research additionally indicated that surface temperature may also affect friction. However, previously reported friction coefficients have been measured only in dynamic contact conditions, where the finger is already sliding across the surface. In this study, we repeatedly measured the initial friction before full slip under eight contact conditions with low and high finger moisture, pressing time, and surface temperature. Moisture and pressing time both independently increased finger-surface friction across our population of twelve participants, and the effect of surface temperature depended on the contact conditions. Furthermore, detailed analysis of the recorded measurements indicates that micro stick-slip during the partial-slip phase contributes to enhanced friction. For the fourth and final part of this dissertation, we designed a transparent moisture sensor for continuous measurement of fingerpad hydration. Because various stimuli cause the sweat pores on fingerprints to excrete sweat, many researchers want to quantify the flow and assess its impact on the formation of the contact area. Unfortunately, the most popular sensor for skin hydration is opaque and does not offer continuous measurements. Our capacitive moisture sensor consists of a pair of inter-digital electrodes covered by an insulating layer, enabling impedance measurements across a wide frequency range. This proposed sensor is made entirely of transparent materials, which allows us to simultaneously measure the finger's contact area. Electrochemical impedance spectroscopy identifies the equivalent electrical circuit and the electrical component parameters that are affected by the amount of moisture present on the surface of the sensor. Most notably, the impedance at 1 kHz seems to best reflect the relative amount of sweat.

hi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Does deliberate prospection help students set better goals?

Jähnichen, S., Weber, F., Prentice, M., Lieder, F.

KogWis 2022 "Understanding Minds", September 2022 (poster) Accepted

re

link (url) [BibTex]

link (url) [BibTex]


no image
Methods for Minimizing the Spread of Misinformation on the Web

Tabibian, B.

University of Tübingen, Germany, September 2022 (phdthesis)

ei

[BibTex]

[BibTex]


Hydraulically Amplified Self-healing Electrostatic Actuators
Hydraulically Amplified Self-healing Electrostatic Actuators

Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K.

(US Patent 11408452), August 2022 (patent)

Abstract
An electro-hydraulic actuator includes a deformable shell defining an enclosed internal cavity and containing a liquid dielectric, first and second electrodes on first and second sides, respectively, of the enclosed internal cavity. An electrostatic force between the first and second electrodes upon application of a voltage to one of the electrodes draws the electrodes towards each other to displace the liquid dielectric within the enclosed internal cavity. The shell includes active and inactive areas such that the electrostatic forces between the first and second electrodes displaces the liquid dielectric within the enclosed internal cavity from the active area of the shell to the inactive area of the shell. The first and second electrodes, the deformable shell, and the liquid dielectric cooperate to form a self-healing capacitor, and the liquid dielectric is configured for automatically filling breaches in the liquid dielectric resulting from dielectric breakdown.

rm

link (url) [BibTex]

link (url) [BibTex]


no image
Learning and Using Causal Knowledge: A Further Step Towards a Higher-Level Intelligence

Huang, B.

Carnegie Mellon University, Pittsburgh, USA, July 2022 (phdthesis)

ei

[BibTex]

[BibTex]


Composite Layering of Hydraulically Amplified Self-Healing Electrostatic Transducers
Composite Layering of Hydraulically Amplified Self-Healing Electrostatic Transducers

Keplinger, C. M., Mitchell, S. K., Kellaris, N. A., Rothemund, P.

(US Patent App. 17436455), May 2022 (patent)

Abstract
A hydraulically amplified self-healing electrostatic (HASEL) transducer includes a composite, multi-layered structure. In an example, a HASEL transducer includes a dielectric layer including at least one fluid dielectric layer. The dielectric layer includes a first side and a second side opposing the first side. The HASEL transducer further includes a first electrode disposed at the first side of the dielectric layer, a second electrode disposed at the second side of the dielectric layer, a first outer layer disposed at the first electrode opposite the dielectric layer, and a second outer layer disposed at the second electrode opposite the dielectric layer. The first outer layer and second outer layer exhibit different mechanical and electrical properties from the dielectric layer.

rm

link (url) [BibTex]

link (url) [BibTex]


no image
Variational Inference in Dynamical Systems

Ialongo, A.

University of Cambridge, UK, Cambridge, February 2022, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

[BibTex]

[BibTex]


no image
Entwicklung von Methoden und Bausteinen zur Realisierung Komplexer Magnonischer Systeme

Groß, F.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), 2022 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Machine-Learning-Driven Haptic Sensor Design

Sun, H.

University of Tuebingen, Library, 2022 (phdthesis)

Abstract
Similar to biological systems, robots may need skin-like sensing ability to perceive interactions in complex, changing, and human-involved environments. Current skin-like sensing technologies are still far behind their biological counterparts when considering resolution, dynamics range, robustness, and surface coverage together. One key challenge is the wiring of sensing elements. During my Ph.D. study, I explore how machine learning can enable the design of a new kind of haptic sensors to deal with such a challenge. On the one hand, I propose super-resolution-oriented tactile skins, reducing the number of physical sensing elements while achieving high spatial accuracy. On the other hand, I explore vision-based haptic sensor designs. In this thesis, I present four types of machine-learning-driven haptic sensors that I designed for coarse and fine robotic applications, varying from large surface (robot limbs) to small surface sensing (robot fingers). Moreover, I propose a super-resolution theory to guide sensor designs at all levels ranging from hardware design (material/structure/transduction), data collection (real/simulated), and signal processing methods (analytical/data-driven). I investigate two designs for large-scale coarse-resolution sensing, e.g., robotic limbs. HapDef sparsely attaches a few strain gauges on a large curved surface internally to measure the deformation over the whole surface. ERT-DNN wraps a large surface with a piece of multi-layered conductive fabric, which varies its conductivity upon contacts exerted. I also conceive two approaches for small-scale fine-resolution sensing, e.g., robotic fingertips. BaroDome sparsely embeds a few barometers inside a soft elastomer to measure internal pressure changes caused by external contact. Insight encloses a high-resolution camera to view a soft shell from within. Generically, an inverse problem needs to be solved when trying to obtain high-resolution sensing with a few physical sensing elements. I develop machine-learning frameworks suitable for solving this inverse problem. They process various raw sensor data and extract useful haptic information in practice. Machine learning methods rely on data collected by an automated robotic stimulation device or synthesized using finite element methods. I build several physical testbeds and finite element models to collect copious data. I propose machine learning frameworks to combine data from different sources that are good enough to deal with the noise in real data and generalize well from seen to unseen situations. While developing my prototype sensors, I have faced reoccurring design choices. To help my developments and guide future research, I propose a unified theory with the concept of taxel-value-isolines. It captures the physical effects required for super-resolution, ties them to all parts of the sensor design, and allows us to assess them quantitatively. The theory offers an explanation about physically achievable accuracies for localizing and quantifying contact based on uncertainties introduced by measurement noise in sensor elements. The theoretical analysis aims to predict the best performance before a physical prototype is built and helps to evaluate the hardware design, data collection, and data processing methods during implementation. This thesis presents a new perspective on haptic sensor design. Using machine learning to substitute the entire data-processing pipeline, I present several haptic sensor designs for applications ranging from large-surface skins to high-resolution tactile fingertip sensors. The developed theory for obtaining optimal super-resolution can guide future sensor designs.

al

link (url) [BibTex]

link (url) [BibTex]

2021


no image
Models for Data-Efficient Reinforcement Learning on Real-World Applications

Doerr, A.

University of Stuttgart, Stuttgart, October 2021 (phdthesis)

ics

DOI [BibTex]

2021


DOI [BibTex]


no image
Dynamics of Learning and Learning of Dynamics

Mehrjou, A.

ETH Zürich, Zürich, October 2021 (phdthesis)

ei

DOI [BibTex]

DOI [BibTex]


no image
A Large Scale Brain-Computer Interface for Patients with Neurological Diseases

Hohmann, M.

University of Tübingen, Germany, September 2021 (phdthesis)

ei

[BibTex]

[BibTex]


no image
Deep Learning Beyond The Training Distribution

Parascandolo, G.

ETH Zürich, Switzerland, Zürich, September 2021, (CLS Fellowship Program) (phdthesis)

ei

DOI [BibTex]

DOI [BibTex]


Skinned multi-infant linear body model
Skinned multi-infant linear body model

Hesse, N., Pujades, S., Romero, J., Black, M.

(US Patent 11,127,163, 2021), September 2021 (patent)

Abstract
A computer-implemented method for automatically obtaining pose and shape parameters of a human body. The method includes obtaining a sequence of digital 3D images of the body, recorded by at least one depth camera; automatically obtaining pose and shape parameters of the body, based on images of the sequence and a statistical body model; and outputting the pose and shape parameters. The body may be an infant body.

ps

[BibTex]

[BibTex]


Huggie{B}ot: An Interactive Hugging Robot With Visual and Haptic Perception
HuggieBot: An Interactive Hugging Robot With Visual and Haptic Perception

Block, A. E.

ETH Zürich, Zürich, August 2021, Department of Computer Science (phdthesis)

Abstract
Hugs are one of the first forms of contact and affection humans experience. Receiving a hug is one of the best ways to feel socially supported, and the lack of social touch can have severe adverse effects on an individual's well-being. Due to the prevalence and health benefits of hugging, roboticists are interested in creating robots that can hug humans as seamlessly as humans hug other humans. However, hugs are complex affective interactions that need to adapt to the height, body shape, and preferences of the hugging partner, and they often include intra-hug gestures like squeezes. This dissertation aims to create a series of hugging robots that use visual and haptic perception to provide enjoyable interactive hugs. Each of the four presented HuggieBot versions is evaluated by measuring how users emotionally and behaviorally respond to hugging it; HuggieBot 4.0 is explicitly compared to a human hugging partner using physiological measures. Building on research both within and outside of human-robot interaction (HRI), this thesis proposes eleven tenets of natural and enjoyable robotic hugging. These tenets were iteratively crafted through a design process combining user feedback and experimenter observation, and they were evaluated through user studies. A good hugging robot should (1) be soft, (2) be warm, (3) be human-sized, (4) autonomously invite the user for a hug when it detects someone in its personal space, and then it should wait for the user to begin walking toward it before closing its arms to ensure a consensual and synchronous hugging experience. It should also (5) adjust its embrace to the user's size and position, (6) reliably release when the user wants to end the hug, and (7) perceive the user's height and adapt its arm positions accordingly to comfortably fit around the user at appropriate body locations. Finally, a hugging robot should (8) accurately detect and classify gestures applied to its torso in real time, regardless of the user's hand placement, (9) respond quickly to their intra-hug gestures, (10) adopt a gesture paradigm that blends user preferences with slight variety and spontaneity, and (11) occasionally provide unprompted, proactive affective social touch to the user through intra-hug gestures. We believe these eleven tenets are essential to delivering high-quality robot hugs. Their presence results in a hug that pleases the user, and their absence results in a hug that is likely to be inadequate. We present these tenets as guidelines for future hugging robot creators to follow when designing new hugging robots to ensure user acceptance. We tested the four versions of HuggieBot through six user studies. First, we analyzed data collected in a previous study with a modified Willow Garage Personal Robot 2 (PR2) to evaluate human responses to different robot physical characteristics and hugging behaviors. Participants experienced and evaluated twelve hugs with the robot, divided into three randomly ordered trials that focused on physical robot characteristics (single factor, three levels) and nine randomly ordered trials with low, medium, and high hug pressure and duration (two factors, three levels each). Second, we created an entirely new robotic platform, HuggieBot 2.0, according to our first six tenets. The new platform features a soft, warm, inflated body (HuggieChest) and uses visual and haptic sensing to deliver closed-loop hugging. We first verified the outward appeal of this platform compared to the previous PR2-based HuggieBot 1.0 via an online video-watching study involving 117 users. We then conducted an in-person experiment in which 32 users each exchanged eight hugs with HuggieBot 2.0, experiencing all combinations of visual hug initiation, haptic sizing, and haptic releasing. We then refine the original fourth tenet (visually perceive its user) and present the remaining five tenets for designing interactive hugging robots; we validate the full list of eleven tenets through more in-person studies with our custom robot. To enable perceptive and pleasing autonomous robot behavior, we investigated robot responses to four human intra-hug gestures: holding, rubbing, patting, and squeezing. The robot's inflated torso's microphone and pressure sensor collected data of 32 people repeatedly demonstrating these gestures, which were used to develop a perceptual algorithm that classifies user actions with 88% accuracy. From user preferences, we created a probabilistic behavior algorithm that chooses robot responses in real time. We implemented improvements to the robot platform to create a third version of our robot, HuggieBot 3.0. We then validated its gesture perception system and behavior algorithm in a fifth user study with 16 users. Finally, we refined the quality and comfort of the embrace by adjusting the joint torques and joint angles of the closed pose position, we further improved the robot's visual perception to detect changes in user approach, we upgraded the robot's response to users who do not press on its back, and we had the robot respond to all intra-hug gestures with squeezes to create our final version of the robotic platform, HuggieBot 4.0. In our sixth user study, we investigated the emotional and physiological effects of hugging a robot compared to the effects of hugging a friendly but unfamiliar person. We continuously monitored participant heart rate and collected saliva samples at seven time points across the 3.5-hour study to measure the temporal evolution of cortisol and oxytocin. We used an adapted Trier Social Stress Test (TSST) protocol to reliably and ethically induce stress in the participants. They then experienced one of five different hug intervention methods before all interacting with HuggieBot 4.0. The results of these six user studies validated our eleven hugging tenets and informed the iterative design of HuggieBot. We see that users enjoy robot softness, robot warmth, and being physically squeezed by the robot. Users dislike being released too soon from a hug and equally dislike being held by the robot for too long. Adding haptic reactivity definitively improves user perception of a hugging robot; the robot's responses and proactive intra-hug gestures were greatly enjoyed. In our last study, we learned that HuggieBot can positively affect users on a physiological level and is somewhat comparable to hugging a person. Participants have more favorable opinions about hugging robots after prolonged interaction with HuggieBot in all of our research studies.

hi

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Chemically active micromotors
Chemically active micromotors

Yu, T.

University of Stuttgart, Stuttgart, July 2021 (phdthesis)

Abstract
Motion is a mark of living systems. It is realised by energy conversion to perform vital tasks and is thus of great importance for all living systems. One approach to achieve motion is by including active motion of micro/nano objects. Unlike in the fluid at the macro scale, active swimming cannot be achieved by reciprocal movements at the micro scale. Breaking symmetry at the micro scale thus becomes a critical issue. The challenge is that this often requires outside intervention to build systems that already show symmetry breaking. And another challenge is that there are few examples where active microscale motion can cause a macroscopic effect, or facilitate a useful application. In the first part of the thesis, the first challenge is addressed and a new route of spontaneous symmetry breaking is developed. Microscale motion in artificial chemical systems has thus far been realised in chemical motion. These are microparticles that are fabricated to possess two different halves, known as Janus particles. One half is catalytically active and drives the self-phoretics. The Janus micromotors are generally fabricated using fabrication techniques such as PVD, CVD. These techniques require deposition onto a surface, which limit the number of structures that can be fabricated. In this work, we show that two species of isotropic (symmetric) micro particles, one is a photocatalytically active particle TiO2, the other is a passive SiO2 particle can spontaneously form a dimer structure. Under UV illumination, a chemical gradient is generated around the photo active particles. The passive particle is attracted toward the highest chemical concentration of the reaction product towards the active particle. A dimer forms that starts to self-propel. The speed of the dimer can be controlled by adjusting the UV intensity. The mechanism of the dimer formation is examined and shown to be due to a diffusiophoretic interaction between the active and the passive particle. The interaction force and the propulsion of the dimer swimmers are examined. The role of salts, particle size and concentration are studied. An additional repulsion interaction is observed between two active particles. An optimal volumetric particle density of ≤ 2% is identified for dimer formation and the dimers remain active for > 20s. This thesis thereby demonstrates a self-assembly route where the chemical activity causes dimer formation and thus spontaneous symmetry breaking which does not require any physical fabrication steps. Most work thus far has studied the behaviour of individual chemical micromotors (Janus particles) at the micro scale. To induce a macroscopic effect and facilitate an application using individual micro/nano active particles requires cooperative effects of many "micromotors". Therefore, we developed a novel fabrication method which allows a large number of Janus structures to be assembled in an ordered manner. We fabricated an array of photoactive Janus micro structures on a surface by glancing angle deposition (GLAD) onto a photolithography patterned substrate. Illuminating the surface of Janus array structures with UV light initiates the water splitting reaction, which produces an osmotic flow around the micro structures. The osmotic flow at each structure is coupled with the flows generated by the neighbouring particles. The microscopic osmotic flow thereby results in a macroscopic fluid flow. By adjusting the spacing between single micro structures, an optimised pumping velocity is achieved with a micro pillar diameter of 2 μm and a spacing of ∼ 2 μm. We compared the pumping performance of the micro pillar array with other topological chemical structures, such as micro Janus bar arrays and 2D micro Janus disk arrays, and find that the 3D structure is essential to generate a chemical gradient on the surface. We believe that this is the first chemical micropump formed by chemically active Janus structures. The active pumping surface can provide a flow speed of up to 4 μm/s. This active surface consisting of micropillar arrays can be easily integrated in most microchannels and serve as an on-board micropump. A theoretical model and numerical simulations are presented to describe the microchannel pumping. The theory reproduces the experimentally measured flow profiles very well. We have thus established a new type of chemical pump, which can wirelessly pump fluid in a microchannel, and the pumping volume rate and flow profile can be modified simply by changing the nature and orientation of the self-pumping walls.

pf

DOI [BibTex]

DOI [BibTex]


no image
Röntgenmikroskopische Untersuchungen der Magnetisierungsdynamik in nanoskaligen magnonischen Wellenleiterstrukturen

Träger, N.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), June 2021 (phdthesis)

mms

[BibTex]

[BibTex]


Promoting metacognitive learning through systematic reflection
Promoting metacognitive learning through systematic reflection

Frederic Becker, , Lieder, F.

The first edition of Life Improvement Science Conference, June 2021 (poster)

Abstract
Human decision-making is sometimes systematically biased toward suboptimal decisions. For example, people often make short-sighted choices because they don't give enough weight to the long-term consequences of their actions. Previous studies showed that it is possible to overcome such biases by teaching people a more rational decision strategy through instruction, demonstrations, or practice with feedback. The benefits of these approaches tend to be limited to situations that are very similar to those used during the training. One way to overcome this limitation is to create general tools and strategies that people can use to improve their decision-making in any situation. Here we propose one such approach, namely directing people to systematically reflect on how they make their decisions. In systematic reflection, past experience is re-evaluated with the intention to learn. In this study, we investigate how reflection affects how people learn to plan and whether reflective learning can help people to discover more far-sighted planning strategies. In our experiment participants solve a series of 30 planning problems where the immediate rewards are smaller and therefore less important than long-term rewards. Building on Wolfbauer et al. (2020), the experimental group is guided by four reflection prompts asking the participant to describe their planning strategy, the strategy's performance, and his or her emotional response, insights, and intention to change their strategy. The control group practices planning without reflection prompts. Our pilot data suggest that systematic reflection helps people to more rapidly discover adaptive planning strategies. Our findings suggest that reflection is useful not only for helping people learn what to do in a specific situation but also for helping people learn how to think about what to do. In future work, we will compare the effects of different types of reflection on the subsequent changes in people's decision strategies. Developing apps that prompt people to reflect on their decisions may be a promising approach to accelerating cognitive growth and promoting lifelong learning.

re

[BibTex]

[BibTex]


no image
Optimization Algorithms for Machine Learning

Raj, A.

University of Tübingen, Germany, June 2021 (phdthesis)

ei

[BibTex]

[BibTex]


no image
Causal Inference in Vision

Meding, K.

Eberhard Karls Universität Tübingen, Tübingen, June 2021 (phdthesis)

ei

[BibTex]

[BibTex]


no image
An electric machine with two-phase planar Lorentz coils and a ring-shaped Halbach array for high torque density and high-precision applications

Nguyen, V., Javot, B., Kuchenbecker, K. J.

(EP21170679.1), April 2021 (patent)

Abstract
An electric machine, in particular a motor or a generator, comprising a rotor and a stator, wherein the rotor comprises a planar, ring-shaped rotor base element and the stator comprises a planar ring-shaped stator base element, wherein the rotor base element and the stator base element are aligned along an axial axis (Z) of the electric machine, wherein a plurality of magnet elements are arranged around the circumference of the ring-shaped rotor base element forming a Halbach magnet-ring assembly, wherein the Halbach magnet-ring assembly generates a magnetic field (BR) with axial and azimuthal components, wherein a plurality of coils are arranged around the circumference (C) of the ring-shaped stator base element.

hi

Project Page [BibTex]


no image
Machine Learning Methods for Modeling Synthesizable Molecules

Bradshaw, J.

University of Cambridge, UK, Cambridge, April 2021, (Cambridge-Tübingen-Fellowship) (phdthesis)

ei

DOI [BibTex]

DOI [BibTex]


Advanced Diffusion Studies of Active Enzymes and Nanosystems
Advanced Diffusion Studies of Active Enzymes and Nanosystems

Günther, J.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), February 2021 (phdthesis)

Abstract
Enzymes are fascinating chemical nanomachines that catalyze many reactions, which are essential for life. Studying enzymes is therefore important in a biological and medical context, but the catalytic potential of enzymes also finds use in organic synthesis. This thesis is concerned with the fundamental question whether the catalytic reaction of an enzyme or molecular catalyst can cause it to show enhanced diffusion. Diffusion measurements were performed with advanced fluorescence correlation spectroscopy (FCS) and diffusion nuclear magnetic resonance (NMR) spectroscopy techniques. The measurement results lead to the unraveling of artefacts in enzyme FCS and molecular NMR measurements, and thus seriously question several recent publications, which claim that enzymes and molecular catalysts are active matter and experience enhanced diffusion. In addition to these fundamental questions, this thesis also examines the use of enzymes as biocatalysts. A novel nanoconstruct – the enzyme-phage-colloid (E-P-C) – is presented, which utilizes filamentous viruses as immobilization templates for enzymes. E-P-Cs can be used for biocatalysis with convenient magnetic recovery of enzymes and serve as enzymatic micropumps. The latter can autonomously pump blood at physiological urea concentrations.

pf

link (url) [BibTex]

link (url) [BibTex]