Header logo is


2020


no image
Vision-based Force Estimation for a da Vinci Instrument Using Deep Neural Networks

Lee, Y., Husin, H. M., Forte, M. P., Lee, S., Kuchenbecker, K. J.

Extended abstract presented as an Emerging Technology ePoster at the Annual Meeting of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), Cleveland, Ohio, USA, August 2020 (misc) Accepted

hi

[BibTex]

2020


[BibTex]


no image
Where Does It End? - Reasoning About Hidden Surfaces by Object Intersection Constraints

Strecke, M., Stückler, J.

In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020 (inproceedings) Accepted

ev

[BibTex]

[BibTex]


A Fabric-Based Sensing System for Recognizing Social Touch
A Fabric-Based Sensing System for Recognizing Social Touch

Burns, R. B., Lee, H., Seifi, H., Kuchenbecker, K. J.

Work-in-progress paper (3 pages) to be presented at the IEEE Haptics Symposium, Washington, DC, USA, March 2020 (misc) Accepted

Abstract
We present a fabric-based piezoresistive tactile sensor system designed to detect social touch gestures on a robot. The unique sensor design utilizes three layers of low-conductivity fabric sewn together on alternating edges to form an accordion pattern and secured between two outer high-conductivity layers. This five-layer design demonstrates a greater resistance range and better low-force sensitivity than previous designs that use one layer of low-conductivity fabric with or without a plastic mesh layer. An individual sensor from our system can presently identify six different communication gestures – squeezing, patting, scratching, poking, hand resting without movement, and no touch – with an average accuracy of 90%. A layer of foam can be added beneath the sensor to make a rigid robot more appealing for humans to touch without inhibiting the system’s ability to register social touch gestures.

hi

Project Page [BibTex]

Project Page [BibTex]


Do Touch Gestures Affect How Electrovibration Feels?
Do Touch Gestures Affect How Electrovibration Feels?

Vardar, Y., Kuchenbecker, K. J.

Hands-on demonstration (1 page) presented at the IEEE Haptics Symposium, Washington, DC, USA, March 2020 (misc) Accepted

hi

[BibTex]

[BibTex]


Learning to Predict Perceptual Distributions of Haptic Adjectives
Learning to Predict Perceptual Distributions of Haptic Adjectives

Richardson, B. A., Kuchenbecker, K. J.

Frontiers in Neurorobotics, 13(116):1-16, Febuary 2020 (article)

Abstract
When humans touch an object with their fingertips, they can immediately describe its tactile properties using haptic adjectives, such as hardness and roughness; however, human perception is subjective and noisy, with significant variation across individuals and interactions. Recent research has worked to provide robots with similar haptic intelligence but was focused on identifying binary haptic adjectives, ignoring both attribute intensity and perceptual variability. Combining ordinal haptic adjective labels gathered from human subjects for a set of 60 objects with features automatically extracted from raw multi-modal tactile data collected by a robot repeatedly touching the same objects, we designed a machine-learning method that incorporates partial knowledge of the distribution of object labels into training; then, from a single interaction, it predicts a probability distribution over the set of ordinal labels. In addition to analyzing the collected labels (10 basic haptic adjectives) and demonstrating the quality of our method's predictions, we hold out specific features to determine the influence of individual sensor modalities on the predictive performance for each adjective. Our results demonstrate the feasibility of modeling both the intensity and the variation of haptic perception, two crucial yet previously neglected components of human haptic perception.

hi

DOI [BibTex]

DOI [BibTex]


no image
Exercising with Baxter: Preliminary Support for Assistive Social-Physical Human-Robot Interaction

Fitter, N. T., Mohan, M., Kuchenbecker, K. J., Johnson, M. J.

Journal of NeuroEngineering and Rehabilitation, 17(19), Febuary 2020 (article)

Abstract
Background: The worldwide population of older adults will soon exceed the capacity of assisted living facilities. Accordingly, we aim to understand whether appropriately designed robots could help older adults stay active at home. Methods: Building on related literature as well as guidance from experts in game design, rehabilitation, and physical and occupational therapy, we developed eight human-robot exercise games for the Baxter Research Robot, six of which involve physical human-robot contact. After extensive iteration, these games were tested in an exploratory user study including 20 younger adult and 20 older adult users. Results: Only socially and physically interactive games fell in the highest ranges for pleasantness, enjoyment, engagement, cognitive challenge, and energy level. Our games successfully spanned three different physical, cognitive, and temporal challenge levels. User trust and confidence in Baxter increased significantly between pre- and post-study assessments. Older adults experienced higher exercise, energy, and engagement levels than younger adults, and women rated the robot more highly than men on several survey questions. Conclusions: The results indicate that social-physical exercise with a robot is more pleasant, enjoyable, engaging, cognitively challenging, and energetic than similar interactions that lack physical touch. In addition to this main finding, researchers working in similar areas can build on our design practices, our open-source resources, and the age-group and gender differences that we found.

hi

DOI [BibTex]

DOI [BibTex]


no image
TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures

Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D.

In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer International Publishing, 2020 (inbook)

ev

[BibTex]

[BibTex]


Physical Variables Underlying Tactile Stickiness during Fingerpad Detachment
Physical Variables Underlying Tactile Stickiness during Fingerpad Detachment

Nam, S., Vardar, Y., Gueorguiev, D., Kuchenbecker, K. J.

Frontiers in Neuroscience, 2020 (article) Accepted

Abstract
One may notice a relatively wide range of tactile sensations even when touching the same hard, flat surface in similar ways. Little is known about the reasons for this variability, so we decided to investigate how the perceptual intensity of light stickiness relates to the physical interaction between the skin and the surface. We conducted a psychophysical experiment in which nine participants actively pressed their finger on a flat glass plate with a normal force close to 1.5 N and detached it after a few seconds. A custom-designed apparatus recorded the contact force vector and the finger contact area during each interaction as well as pre- and post-trial finger moisture. After detaching their finger, participants judged the stickiness of the glass using a nine-point scale. We explored how sixteen physical variables derived from the recorded data correlate with each other and with the stickiness judgments of each participant. These analyses indicate that stickiness perception mainly depends on the pre-detachment pressing duration, the time taken for the finger to detach, and the impulse in the normal direction after the normal force changes sign; finger-surface adhesion seems to build with pressing time, causing a larger normal impulse during detachment and thus a more intense stickiness sensation. We additionally found a strong between-subjects correlation between maximum real contact area and peak pull-off force, as well as between finger moisture and impulse.

hi

[BibTex]


no image
Planning from Images with Deep Latent Gaussian Process Dynamics

Bosch, N., Achterhold, J., Leal-Taixe, L., Stückler, J.

2nd Annual Conference on Learning for Dynamics and Control (L4DC) , 2020 (conference) Accepted

ev

[BibTex]

[BibTex]


no image
Visual-Inertial Mapping with Non-Linear Factor Recovery

Usenko, V., Demmel, N., Schubert, D., Stückler, J., Cremers, D.

IEEE Robotics and Automation Letters (RA-L), 5, 2020, accepted for presentation at IEEE International Conference on Robotics and Automation (ICRA) 2020, to appear, arXiv:1904.06504 (article)

Abstract
Cameras and inertial measurement units are complementary sensors for ego-motion estimation and environment mapping. Their combination makes visual-inertial odometry (VIO) systems more accurate and robust. For globally consistent mapping, however, combining visual and inertial information is not straightforward. To estimate the motion and geometry with a set of images large baselines are required. Because of that, most systems operate on keyframes that have large time intervals between each other. Inertial data on the other hand quickly degrades with the duration of the intervals and after several seconds of integration, it typically contains only little useful information. In this paper, we propose to extract relevant information for visual-inertial mapping from visual-inertial odometry using non-linear factor recovery. We reconstruct a set of non-linear factors that make an optimal approximation of the information on the trajectory accumulated by VIO. To obtain a globally consistent map we combine these factors with loop-closing constraints using bundle adjustment. The VIO factors make the roll and pitch angles of the global map observable, and improve the robustness and the accuracy of the mapping. In experiments on a public benchmark, we demonstrate superior performance of our method over the state-of-the-art approaches.

ev

[BibTex]

[BibTex]


no image
DirectShape: Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation

Wang, R., Yang, N., Stückler, J., Cremers, D.

In Accepted for IEEE international Conference on Robotics and Automation (ICRA), 2020, arXiv:1904.10097 (inproceedings) Accepted

ev

[BibTex]

[BibTex]


Nanopropellers and Their Actuation in Complex Viscoelastic Media
Nanopropellers and Their Actuation in Complex Viscoelastic Media

Schamel, D., Mark, A. G., Gibbs, J. G., Miksch, C., Morozov, K. I., Leshansky, A. M., Fischer, P.

ACS Nano, 8(9):8794-8801, June 2014, Featured cover article. (article)

Abstract
Tissue and biological fluids are complex viscoelastic media with a nanoporous macromolecular structure. Here, we demonstrate that helical nanopropellers can be controllably steered through such a biological gel. The screw-propellers have a filament diameter of about 70 nm and are smaller than previously reported nanopropellers as well as any swimming microorganism. We show that the nanoscrews will move through high-viscosity solutions with comparable velocities to that of larger micropropellers, even though they are so small that Brownian forces suppress their actuation in pure water. When actuated in viscoelastic hyaluronan gels, the nanopropellers appear to have a significant advantage, as they are of the same size range as the gel’s mesh size. Whereas larger helices will show very low or negligible propulsion in hyaluronan solutions, the nanoscrews actually display significantly enhanced propulsion velocities that exceed the highest measured speeds in Newtonian fluids. The nanopropellers are not only promising for applications in the extracellular environment but small enough to be taken up by cells.

Featured cover article.

pf

Video - Helical Micro and Nanopropellers for Applications in Biological Fluidic Environments link (url) DOI [BibTex]


Convertor
Convertor

Fischer, P., Mark, A.

May 2014 (patent)

pf

[BibTex]

[BibTex]


3D nanofabrication on complex seed shapes using glancing angle deposition
3D nanofabrication on complex seed shapes using glancing angle deposition

Hyeon-Ho, J., Mark, A. G., Gibbs, J. G., Reindl, T., Waizmann, U., Weis, J., Fischer, P.

In 2014 IEEE 27th International Conference on Micro Electro Mechanical Systems (MEMS), pages: 437-440, January 2014 (inproceedings)

Abstract
Three-dimensional (3D) fabrication techniques promise new device architectures and enable the integration of more components, but fabricating 3D nanostructures for device applications remains challenging. Recently, we have performed glancing angle deposition (GLAD) upon a nanoscale hexagonal seed array to create a variety of 3D nanoscale objects including multicomponent rods, helices, and zigzags [1]. Here, in an effort to generalize our technique, we present a step-by-step approach to grow 3D nanostructures on more complex nanoseed shapes and configurations than before. This approach allows us to create 3D nanostructures on nanoseeds regardless of seed sizes and shapes.

pf

DOI [BibTex]

DOI [BibTex]


Circular polarization interferometry: circularly polarized modes of cholesteric liquid crystals
Circular polarization interferometry: circularly polarized modes of cholesteric liquid crystals

Sanchez-Castillo, A., Eslami, S., Giesselmann, F., Fischer, P.

OPTICS EXPRESS, 22(25):31227-31236, 2014 (article)

Abstract
We describe a novel polarization interferometer which permits the determination of the refractive indices for circularly-polarized light. It is based on a Jamin-Lebedeff interferometer, modified with waveplates, and permits us to experimentally determine the refractive indices n(L) and n(R) of the respectively left- and right-circularly polarized modes in a cholesteric liquid crystal. Whereas optical rotation measurements only determine the circular birefringence, i.e. the difference (n(L) - n(R)), the interferometer also permits the determination of their absolute values. We report refractive indices of a cholesteric liquid crystal in the region of selective (Bragg) reflection as a function of temperature. (C) 2014 Optical Society of America

pf

DOI [BibTex]

DOI [BibTex]


Self-Propelling Nanomotors in the Presence of Strong Brownian Forces
Self-Propelling Nanomotors in the Presence of Strong Brownian Forces

Lee, T., Alarcon-Correa, M., Miksch, C., Hahn, K., Gibbs, J. G., Fischer, P.

NANO LETTERS, 14(5):2407-2412, 2014 (article)

Abstract
Motility in living systems is due to an array of complex molecular nanomotors that are essential for the function and survival of cells. These protein nanomotors operate not only despite of but also because of stochastic forces. Artificial means of realizing motility rely on local concentration or temperature gradients that are established across a particle, resulting in slip velocities at the particle surface and thus motion of the particle relative to the fluid. However, it remains unclear if these artificial motors can function at the smallest of scales, where Brownian motion dominates and no actively propelled living organisms can be found. Recently, the first reports have appeared suggesting that the swimming mechanisms of artificial structures may also apply to enzymes that are catalytically active. Here we report a scheme to realize artificial Janus nanoparticles (JNPs) with an overall size that is comparable to that of some enzymes similar to 30 nm. Our JNPs can catalyze the decomposition of hydrogen peroxide to water and oxygen and thus actively move by self-electrophoresis. Geometric anisotropy of the Pt-Au Janus nanoparticles permits the simultaneous observation of their translational and rotational motion by dynamic light scattering. While their dynamics is strongly influenced by Brownian rotation, the artificial Janus nanomotors show bursts of linear ballistic motion resulting in enhanced diffusion.

pf

DOI [BibTex]


Shape control in wafer-based aperiodic 3D nanostructures
Shape control in wafer-based aperiodic 3D nanostructures

Hyeon-Ho, J., Mark, A. G., Gibbs, J. G., Reindl, T., Waizmann, U., Weis, J., Fischer, P.

NANOTECHNOLOGY, 25(23), 2014, Cover article. (article)

Abstract
Controlled local fabrication of three-dimensional (3D) nanostructures is important to explore and enhance the function of single nanodevices, but is experimentally challenging. We present a scheme based on e-beam lithography (EBL) written seeds, and glancing angle deposition (GLAD) grown structures to create nanoscale objects with defined shapes but in aperiodic arrangements. By using a continuous sacrificial corral surrounding the features of interest we grow isolated 3D nanostructures that have complex cross-sections and sidewall morphology that are surrounded by zones of clean substrate.

Cover article.

pf

DOI [BibTex]

DOI [BibTex]


Active Microrheology of the Vitreous of the Eye applied to Nanorobot Propulsion
Active Microrheology of the Vitreous of the Eye applied to Nanorobot Propulsion

Qiu, T., Schamel, D., Mark, A. G., Fischer, P.

In 2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), pages: 3801-3806, IEEE International Conference on Robotics and Automation ICRA, 2014, Best Automation Paper Award – Finalist. (inproceedings)

Abstract
Biomedical applications of micro or nanorobots require active movement through complex biological fluids. These are generally non-Newtonian (viscoelastic) fluids that are characterized by complicated networks of macromolecules that have size-dependent rheological properties. It has been suggested that an untethered microrobot could assist in retinal surgical procedures. To do this it must navigate the vitreous humor, a hydrated double network of collagen fibrils and high molecular-weight, polyanionic hyaluronan macromolecules. Here, we examine the characteristic size that potential robots must have to traverse vitreous relatively unhindered. We have constructed magnetic tweezers that provide a large gradient of up to 320 T/m to pull sub-micron paramagnetic beads through biological fluids. A novel two-step electrical discharge machining (EDM) approach is used to construct the tips of the magnetic tweezers with a resolution of 30 mu m and high aspect ratio of similar to 17:1 that restricts the magnetic field gradient to the plane of observation. We report measurements on porcine vitreous. In agreement with structural data and passive Brownian diffusion studies we find that the unhindered active propulsion through the eye calls for nanorobots with cross-sections of less than 500 nm.

Best Automation Paper Award – Finalist.

pf

[BibTex]

[BibTex]


Swimming by reciprocal motion at low Reynolds number
Swimming by reciprocal motion at low Reynolds number

Qiu, T., Lee, T., Mark, A. G., Morozov, K. I., Muenster, R., Mierka, O., Turek, S., Leshansky, A. M., Fischer, P.

NATURE COMMUNICATIONS, 5, 2014, Max Planck Press Release. (article)

Abstract
Biological microorganisms swim with flagella and cilia that execute nonreciprocal motions for low Reynolds number (Re) propulsion in viscous fluids. This symmetry requirement is a consequence of Purcell's scallop theorem, which complicates the actuation scheme needed by microswimmers. However, most biomedically important fluids are non-Newtonian where the scallop theorem no longer holds. It should therefore be possible to realize a microswimmer that moves with reciprocal periodic body-shape changes in non-Newtonian fluids. Here we report a symmetric `micro-scallop', a single-hinge microswimmer that can propel in shear thickening and shear thinning (non-Newtonian) fluids by reciprocal motion at low Re. Excellent agreement between our measurements and both numerical and analytical theoretical predictions indicates that the net propulsion is caused by modulation of the fluid viscosity upon varying the shear rate. This reciprocal swimming mechanism opens new possibilities in designing biomedical microdevices that can propel by a simple actuation scheme in non-Newtonian biological fluids.

Max Planck Press Release.

pf

Video - A Swimming Micro-Scallop Video - Winner of the Micro-robotic Design Challenge in Hamlyn Symposium on Medical Robotics DOI [BibTex]

Video - A Swimming Micro-Scallop Video - Winner of the Micro-robotic Design Challenge in Hamlyn Symposium on Medical Robotics DOI [BibTex]


Nanohelices by shadow growth
Nanohelices by shadow growth

Gibbs, J. G., Mark, A. G., Lee, T., Eslami, S., Schamel, D., Fischer, P.

NANOSCALE, 6(16):9457-9466, 2014 (article)

Abstract
The helix has remarkable qualities and is prevalent in many fields including mathematics, physics, chemistry, and biology. This shape, which is chiral by nature, is ubiquitous in biology with perhaps the most famous example being DNA. Other naturally occurring helices are common at the nanoscale in the form of protein secondary structures and in various macromolecules. Nanoscale helices exhibit a wide range of interesting mechanical, optical, and electrical properties which can be intentionally engineered into the structure by choosing the correct morphology and material. As technology advances, these fabrication parameters can be fine-tuned and matched to the application of interest. Herein, we focus on the fabrication and properties of nanohelices grown by a dynamic shadowing growth method combined with fast wafer-scale substrate patterning which has a number of distinct advantages. We review the fabrication methodology and provide several examples that illustrate the generality and utility of nanohelices shadow-grown on nanopatterns.

pf

Video - Fabrication of Designer Nanostructures DOI [BibTex]


Chiral Nanomagnets
Chiral Nanomagnets

Eslami, S., Gibbs, J. G., Rechkemmer, Y., van Slageren, J., Alarcon-Correa, M., Lee, T., Mark, A. G., Rikken, G. L. J. A., Fischer, P.

ACS PHOTONICS, 1(11):1231-1236, 2014 (article)

Abstract
We report on the enhanced optical properties of chiral magnetic nanohelices with critical dimensions comparable to the ferromagnetic domain size. They are shown to be ferromagnetic at room temperature, have defined chirality, and exhibit large optical activity in the visible as verified by electron microscopy, superconducting quantum interference device (SQUID) magnetometry, natural circular dichroism (NCD), and magnetic circular dichroism (MCD) measurements. The structures exhibit magneto-chiral dichroism (MChD), which directly demonstrates coupling between their structural chirality and magnetism. A chiral nickel (Ni) film consisting of an array of nanohelices similar to 100 nm in length exhibits an MChD anisotropy factor g(MChD) approximate to 10(-4) T-1 at room temperature in a saturation field of similar to 0.2 T, permitting polarization-independent control of the film's absorption properties through magnetic field modulation. This is also the first report of MChD in a material with structural chirality on the order of the wavelength of light, and therefore the Ni nanohelix array is a metamaterial with magnetochiral properties that can be tailored through a dynamic deposition process.

pf

DOI [BibTex]

DOI [BibTex]


Wireless powering of e-swimmers
Wireless powering of e-swimmers

Roche, J., Carrara, S., Sanchez, J., Lannelongue, J., Loget, G., Bouffier, L., Fischer, P., Kuhn, A.

SCIENTIFIC REPORTS, 4, 2014 (article)

Abstract
Miniaturized structures that can move in a controlled way in solution and integrate various functionalities are attracting considerable attention due to the potential applications in fields ranging from autonomous micromotors to roving sensors. Here we introduce a concept which allows, depending on their specific design, the controlled directional motion of objects in water, combined with electronic functionalities such as the emission of light, sensing, signal conversion, treatment and transmission. The approach is based on electric field-induced polarization, which triggers different chemical reactions at the surface of the object and thereby its propulsion. This results in a localized electric current that can power in a wireless way electronic devices in water, leading to a new class of electronic swimmers (e-swimmers).

pf

DOI [BibTex]

DOI [BibTex]


Swelling and shrinking behaviour of photoresponsive phosphonium-based ionogel microstructures
Swelling and shrinking behaviour of photoresponsive phosphonium-based ionogel microstructures

Czugala, M., O’Connell, C., Blin, C., Fischer, P., Fraser, K. J., Benito-Lopez, F., Diamond, D.

SENSORS AND ACTUATORS B-CHEMICAL, 194, pages: 105-113, 2014 (article)

Abstract
Photoresponsive N-isopropylacrylamide ionogel microstructures are presented in this study. These ionogels are synthesised using phosphonium based room temperature ionic liquids, together with the photochromic compound benzospiropyran. The microstructures can be actuated using light irradiation, facilitating non-contact and non-invasive operation. For the first time, the characterisation of the swelling and shrinking behaviour of several photopatterned ionogel microstructures is presented and the influence of surface-area-to-volume ratio on the swelling kinetics is evaluated. It was found that the swelling and shrinking behaviour of the ionogels is strongly dependent on the nature of the ionic liquid. In particular, the {[}P-6,P-6,P-6,P-14]{[}NTf2] ionogel exhibits the greatest degree of swelling, reaching up to 180\% of its initial size, and the fastest shrinkage rate (k(sh) = 29 +/- 4 x 10(-2) s(-1)). (C) 2014 Elsevier B. V. All rights reserved.

pf

DOI [BibTex]

DOI [BibTex]