Dr. Metin Sitti received the BSc and MSc degrees in electrical and electronics engineering from Bogazici University, Istanbul, Turkey, in 1992 and 1994, respectively, and the PhD degree in electrical engineering from the University of Tokyo, Tokyo, Japan, in 1999. He was a research scientist at UC Berkeley during 1999-2002. He has been a professor in the Department of Mechanical Engineering and Robotics Institute at Carnegie Mellon University, Pittsburgh, USA since 2002. He is currently a director at the Max Planck Institute for Intelligent Systems in Stuttgart. His research interests include small-scale physical intelligence, mobile microrobotics, bio-inspired materials and miniature robots, soft robotics, and micro-/nanomanipulation. He is an IEEE Fellow. He received the SPIE Nanoengineering Pioneer Award in 2011 and NSF CAREER Award in 2005. He received many best paper, video and poster awards in major robotics and adhesion conferences. He is the editor-in-chief of the Journal of Micro-Bio Robotics.
In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 6612-6618, 2017 (inproceedings)
Most demonstrated mobile microrobot tasks so far have been achieved via pick-and-placing and dynamic trapping with teleoperation or simple path following algorithms. In our previous work, an untethered magnetic microgripper has been developed which has advanced functions, such as gripping objects. Both teleoperated manipulation in 2D and 3D have been demonstrated. However, it is challenging to control the magnetic microgripper to carry out manipulation tasks, because the grasping of objects so far in the literature relies heavily on teleoperation, which takes several minutes with even a skilled human expert. Here, we propose a new spin-walking locomotion and an automated 2D grasping motion planner for the microgripper, which enables time-efficient automatic grasping of microobjects that has not been achieved yet for untethered microrobots. In its locomotion, the microgripper repeatedly rotates about two principal axes to regulate its pose and move precisely on a surface. The motion planner could plan different motion primitives for grasping and compensate the uncertainties in the motion by learning the uncertainties and planning accordingly. We experimentally demonstrated that, using the proposed method, the microgripper could align to the target pose with error less than 0.1 body length and grip the objects within 40 seconds. Our method could significantly improve the time efficiency of micro-scale manipulation and have potential applications in microassembly and biomedical engineering.
In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 3404-3410, June 2017 (inproceedings)
Magnetic untethered millirobots, which are actuated and controlled by remote magnetic fields, have been proposed for medical applications due to their ability to safely pass through tissues at long ranges. For example, magnetic resonance imaging (MRI) systems with a 3-7 T constant unidirectional magnetic field and 3D gradient coils have been used to actuate magnetic robots. Such magnetically constrained systems place limits on the degrees of freedom that can be actuated for untethered devices. This paper presents a design and actuation methodology for a magnetic millirobot that exhibits both position and orientation control in 2D under a magnetic field, dominated by a constant unidirectional magnetic field as found in MRI systems. Placing a spherical permanent magnet, which is free to rotate inside the millirobot and located away from the center of mass, allows the generation of net forces and torques with applied 3D magnetic field gradients. We model this system in a 3D planar case and experimentally demonstrate open-loop control of both position and orientation by the applied 2D field gradients. The actuation performance is characterized across the most important design variables, and we experimentally demonstrate that the proposed approach is feasible.
In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 1132-1139, May 2017 (inproceedings)
This paper presents a magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy (B-MASCE) in the upper gastrointestinal tract. A thin and hollow needle is attached to the capsule, which can penetrate deeply into tissues to obtain subsurface biopsy sample. The design utilizes a soft elastomer body as a compliant mechanism to guide the needle. An internal permanent magnet provides a means for both actuation and tracking. The capsule is designed to roll towards its target and then deploy the biopsy needle in a precise location selected as the target area. B-MASCE is controlled by multiple custom-designed electromagnets while its position and orientation are tracked by a magnetic sensor array. In in vitro trials, B-MASCE demonstrated rolling locomotion and biopsy of a swine tissue model positioned inside an anatomical human stomach model. It was confirmed after the experiment that a tissue sample was retained inside the needle.
ACS Nano, 11(9):8910-8923, September 2017, PMID: 28873304 (article)
High-performance, multifunctional bacteria-driven microswimmers are introduced using an optimized design and fabrication method for targeted drug delivery applications. These microswimmers are made of mostly single Escherichia coli bacterium attached to the surface of drug-loaded polyelectrolyte multilayer (PEM) microparticles with embedded magnetic nanoparticles. The PEM drug carriers are 1 μm in diameter and are intentionally fabricated with a more viscoelastic material than the particles previously studied in the literature. The resulting stochastic microswimmers are able to swim at mean speeds of up to 22.5 μm/s. They can be guided and targeted to specific cells, because they exhibit biased and directional motion under a chemoattractant gradient and a magnetic field, respectively. Moreover, we demonstrate the microswimmers delivering doxorubicin anticancer drug molecules, encapsulated in the polyelectrolyte multilayers, to 4T1 breast cancer cells under magnetic guidance in vitro. The results reveal the feasibility of using these active multifunctional bacteria-driven microswimmers to perform targeted drug delivery with significantly enhanced drug transfer, when compared with the passive PEM microparticles.
Biohybrid cell-driven microsystems offer unparalleled possibilities for realization of soft microrobots at the micron scale. Here, we introduce a bacteria-driven microswimmer that combines the active locomotion and sensing capabilities of bacteria with the desirable encapsulation and viscoelastic properties of a soft double-micelle microemulsion for active transport and delivery of cargo (e.g., imaging agents, genes, and drugs) to living cells. Quasi-monodisperse double emulsions were synthesized with an aqueous core that encapsulated the fluorescence imaging agents, as a proof-of-concept cargo in this study, and an outer oil shell that was functionalized with streptavidin for specific and stable attachment of biotin-conjugated Escherichia coli. Motile bacteria effectively propelled the soft microswimmers across a Transwell membrane, actively delivering imaging agents (i.e., dyes) encapsulated inside of the micelles to a monolayer of cultured MCF7 breast cancer and J744A.1 macrophage cells, which enabled real-time, live-cell imaging of cell organelles, namely mitochondria, endoplasmic reticulum, and Golgi body. This in vitro model demonstrates the proof-of-concept feasibility of the proposed soft microswimmers and offers promise for potential biomedical applications in active and/or targeted transport and delivery of imaging agents, drugs, stem cells, siRNA, and therapeutic genes to live tissue in in vitro disease models (e.g., organ-on-a-chip devices) and stagnant or low-flow-velocity fluidic regions of the human body.
ACS Nano, 0(0):null, September 2017, PMID: 28933815 (article)
Biofilm colonies are typically resistant to general antibiotic treatment and require targeted methods for their removal. One of these methods includes the use of nanoparticles as carriers for antibiotic delivery, where they randomly circulate in fluid until they make contact with the infected areas. However, the required proximity of the particles to the biofilm results in only moderate efficacy. We demonstrate here that the nonpathogenic magnetotactic bacteria Magnetosopirrillum gryphiswalense (MSR-1) can be integrated with drug-loaded mesoporous silica microtubes to build controllable microswimmers (biohybrids) capable of antibiotic delivery to target an infectious biofilm. Applying external magnetic guidance capability and swimming power of the MSR-1 cells, the biohybrids are directed to and forcefully pushed into matured Escherichia coli (E. coli) biofilms. Release of the antibiotic, ciprofloxacin, is triggered by the acidic microenvironment of the biofilm, ensuring an efficient drug delivery system. The results reveal the capabilities of a nonpathogenic bacteria species to target and dismantle harmful biofilms, indicating biohybrid systems have great potential for antibiofilm applications.
In the last decade, researchers and medical device companies have made major advances towards transforming passive capsule endoscopes into active medical robots. One of the major challenges is to endow capsule robots with accurate perception of the environment inside the human body, which will provide necessary information and enable improved medical procedures. We extend the success of deep learning approaches from various research fields to the problem of uncalibrated, asynchronous, and asymmetric sensor fusion for endoscopic capsule robots. The results performed on real pig stomach datasets show that our method achieves sub-millimeter precision for both translational and rotational movements and contains various advantages over traditional sensor fusion techniques.
A reliable, real time multi-sensor fusion functionality is crucial for localization of actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we propose a novel multi-sensor fusion approach based on a particle filter that incorporates an online estimation of sensor reliability and a non-linear kinematic model learned by a recurrent neural network. Our method sequentially estimates the true robot pose from noisy pose observations delivered by multiple sensors. We experimentally test the method using 5 degree-of-freedom (5-DoF) absolute pose measurement by a magnetic localization system and a 6-DoF relative pose measurement by visual odometry. In addition, the proposed method is capable of detecting and handling sensor failures by ignoring corrupted data, providing the robustness expected of a medical device. Detailed analyses and evaluations are presented using ex-vivo experiments on a porcine stomach model prove that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.
Since the development of capsule endoscopcy technology, substantial progress were made in converting passive capsule endoscopes to robotic active capsule endoscopes which can be controlled by the doctor. However, robotic capsule endoscopy still has some challenges. In particular, the use of such devices to generate a precise and globally consistent three-dimensional (3D) map of the entire inner organ remains an unsolved problem. Such global 3D maps of inner organs would help doctors to detect the location and size of diseased areas more accurately, precisely, and intuitively, thus permitting more accurate and intuitive diagnoses. The proposed 3D reconstruction system is built in a modular fashion including preprocessing, frame stitching, and shading-based 3D reconstruction modules. We propose an efficient scheme to automatically select the key frames out of the huge quantity of raw endoscopic images. Together with a bundle fusion approach that aligns all the selected key frames jointly in a globally consistent way, a significant improvement of the mosaic and 3D map accuracy was reached. To the best of our knowledge, this framework is the first complete pipeline for an endoscopic capsule robot based 3D map reconstruction containing all of the necessary steps for a reliable and accurate endoscopic 3D map. For the qualitative evaluations, a real pig stomach is employed. Moreover, for the first time in literature, a detailed and comprehensive quantitative analysis of each proposed pipeline modules is performed using a non-rigid esophagus gastro duodenoscopy simulator, four different endoscopic cameras, a magnetically activated soft capsule robot (MASCE), a sub-millimeter precise optical motion tracker and a fine-scale 3D optical scanner.
In 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pages: 250-251, June 2017 (inproceedings)
The dynamic analysis has been considered as one of the important design methods to design robots. In this research, we derive dynamic equation of hexapedal water-running robot to design compliant joints. The compliant joints that connect three bodies will be used to improve mobility and stability of water-running motion's pitch behavior. We considered all of parts as rigid body including links of six Klann mechanisms and three main frames. And then, we derived dynamic equation by using the Lagrangian method with external force of the water. We are expecting that the dynamic analysis is going to be used to design parts of the water running robot.
In 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pages: 1117-1122, July 2017 (inproceedings)
Magnetic control of drug carriers using systems with open-configurations is essential to enable scaling to the size of in vivo applications. In this study, we demonstrate motion control of paramagnetic microparticles in a low Reynolds number fluid, using a permanent magnet-based robotic system with an open-configuration. The microparticles are controlled in three-dimensional (3D) space using a cylindrical NdFeB magnet that is fixed to the end-effector of a robotic arm. We develop a kinematic map between the position of the microparticles and the configuration of the robotic arm, and use this map as a basis of a closed-loop control system based on the position of the microparticles. Our experimental results show the ability of the robot configuration to control the exerted field gradient on the dipole of the microparticles, and achieve positioning in 3D space with maximum error of 300 µm and 600 µm in the steady-state during setpoint and trajectory tracking, respectively.
In 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pages: 1103-1110, July 2017 (inproceedings)
The development of optimal XY θz flexure mechanisms, which can deliver high precision motion about the z-axis, and along the x- and y-axes is highly desirable for a wide range of micro/nano-positioning tasks pertaining to biomedical research, microscopy technologies and various industrial applications. Although maximizing the stiffness ratios is a very critical design requirement, the achievable translational and rotational stiffness ratios of existing XY θz flexure mechanisms are still restricted between 0.5 and 130. As a result, these XY θz flexure mechanisms are unable to fully optimize their workspace and capabilities to reject disturbances. Here, we present an optimal XY θz flexure mechanism, which is designed to have maximum stiffness ratios. Based on finite element analysis (FEA), it has translational stiffness ratio of 248, rotational stiffness ratio of 238 and a large workspace of 2.50 mm × 2.50 mm × 10°. Despite having such a large workspace, FEA also predicts that the proposed mechanism can still achieve a high bandwidth of 70 Hz. In comparison, the bandwidth of similar existing flexure mechanisms that can deflect more than 0.5 mm or 0.5° is typically less than 45 Hz. Hence, the high stiffness ratios of the proposed mechanism are achieved without compromising its dynamic performance. Preliminary experimental results pertaining to the mechanism's translational actuating stiffness and bandwidth were in agreement with the FEA predictions as the deviation was within 10%. In conclusion, the proposed flexure mechanism exhibits superior performance and can be used across a wide range of applications.
Ingestible wireless capsule endoscopy is an emerging minimally invasive diagnostic technology for inspection of the GI tract and diagnosis of a wide range of diseases and pathologies. Medical device companies and many research groups have recently made substantial progresses in converting passive capsule endoscopes to active capsule robots, enabling more accurate, precise, and intuitive detection of the location and size of the diseased areas. Since a reliable real time pose estimation functionality is crucial for actively controlled endoscopic capsule robots, in this study, we propose a monocular visual odometry (VO) method for endoscopic capsule robot operations. Our method lies on the application of the deep Recurrent Convolutional Neural Networks (RCNNs) for the visual odometry task, where Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are used for the feature extraction and inference of dynamics across the frames, respectively. Detailed analyses and evaluations made on a real pig stomach dataset proves that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.
Google Patents, 2017, US Patent 9,731,422 (patent)
The present invention are methods for fabrication of micro- and/or nano-scale adhesive fibers and their use for movement and manipulation of objects. Further disclosed is a method of manipulating a part by providing a manipulation device with a plurality of fibers, where each fiber has a tip with a flat surface that is parallel to a backing layer, contacting the flat surfaces on an object, moving the object to a new location, then disengaging the tips from the object.
Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 473(2203), The Royal Society, 2017 (article)
To add to the current state of knowledge about bacterial swimming dynamics, in this paper, we study the fractal swimming dynamics of populations of Serratia marcescens bacteria both in vitro and in silico, while accounting for realistic conditions like volume exclusion, chemical interactions, obstacles and distribution of chemoattractant in the environment. While previous research has shown that bacterial motion is non-ergodic, we demonstrate that, besides the non-ergodicity, the bacterial swimming dynamics is multi-fractal in nature. Finally, we demonstrate that the multi-fractal characteristic of bacterial dynamics is strongly affected by bacterial density and chemoattractant concentration.
Mobile Microrobotics, pages: 304, The MIT Press, Cambridge, MA, 2017 (book)
Progress in micro- and nano-scale science and technology has created a demand for new microsystems for high-impact applications in healthcare, biotechnology, manufacturing, and mobile sensor networks. The new robotics field of microrobotics has emerged to extend our interactions and explorations to sub-millimeter scales. This is the first textbook on micron-scale mobile robotics, introducing the fundamentals of design, analysis, fabrication, and control, and drawing on case studies of existing approaches.
The book covers the scaling laws that can be used to determine the dominant forces and effects at the micron scale; models forces acting on microrobots, including surface forces, friction, and viscous drag; and describes such possible microfabrication techniques as photo-lithography, bulk micromachining, and deep reactive ion etching. It presents on-board and remote sensing methods, noting that remote sensors are currently more feasible; studies possible on-board microactuators; discusses self-propulsion methods that use self-generated local gradients and fields or biological cells in liquid environments; and describes remote microrobot actuation methods for use in limited spaces such as inside the human body. It covers possible on-board powering methods, indispensable in future medical and other applications; locomotion methods for robots on surfaces, in liquids, in air, and on fluid-air interfaces; and the challenges of microrobot localization and control, in particular multi-robot control methods for magnetic microrobots. Finally, the book addresses current and future applications, including noninvasive medical diagnosis and treatment, environmental remediation, and scientific tools.
Journal of The Royal Society Interface, 14(131):20170134, The Royal Society, June 2017 (article)
Animals using adhesive pads to climb smooth surfaces face the problem of keeping their pads clean and functional. Here, a self-cleaning mechanism is proposed whereby soiled feet would slip on the surface due to a lack of adhesion but shed particles in return. Our study offers an in situ quantification of self-cleaning performance in fibrillar adhesives, using the dock beetle as a model organism. After beetles soiled their pads by stepping into patches of spherical beads, we found that their gait was significantly affected. Specifically, soiled pads slipped 10 times further than clean pads, with more particles deposited for longer slips. Like previous studies, we found that particle size affected cleaning performance. Large (45 μm) beads were removed most effectively, followed by medium (10 μm) and small (1 μm). Consistent with our results from climbing beetles, force measurements on freshly severed legs revealed larger detachment forces of medium particles from adhesive pads compared to a flat surface, possibly due to interlocking between fibres. By contrast, dock leaves showed an overall larger affinity to the beads and thus reduced the need for cleaning. Self-cleaning through slippage provides a mechanism robust to particle size and may inspire solutions for artificial adhesives.
Advanced Materials, May 2017, Back Cover (article)
A facile approach is proposed for superior conformation and adhesion of wearable sensors to dry and wet skin. Bioinspired skin-adhesive films are composed of elastomeric microfibers decorated with conformal and mushroom-shaped vinylsiloxane tips. Strong skin adhesion is achieved by crosslinking the viscous vinylsiloxane tips directly on the skin surface. Furthermore, composite microfibrillar adhesive films possess a high adhesion strength of 18 kPa due to the excellent shape adaptation of the vinylsiloxane tips to the multiscale roughness of the skin. As a utility of the skin-adhesive films in wearable-device applications, they are integrated with wearable strain sensors for respiratory and heart-rate monitoring. The signal-to-noise ratio of the strain sensor is significantly improved to 59.7 because of the considerable signal amplification of microfibrillar skin-adhesive films.
Despite the large body of experimental work recently on biohybrid microsystems, few studies have focused on theoretical modeling of such systems, which is essential to understand their underlying functioning mechanisms and hence design them optimally for a given application task. Therefore, this study focuses on developing a mathematical model to describe the 3D motion and chemotaxis of a type of widely studied biohybrid microswimmer, where spherical microbeads are driven by multiple attached bacteria. The model is developed based on the biophysical observations of the experimental system and is validated by comparing the model simulation with experimental 3D swimming trajectories and other motility characteristics, including mean squared displacement, speed, diffusivity, and turn angle. The chemotaxis modeling results of the microswimmers also agree well with the experiments, where a collective chemotactic behavior among multiple bacteria is observed. The simulation result implies that such collective chemotaxis behavior is due to a synchronized signaling pathway across the bacteria attached to the same microswimmer. Furthermore, the dependencies of the motility and chemotaxis of the microswimmers on certain system parameters, such as the chemoattractant concentration gradient, swimmer body size, and number of attached bacteria, toward an optimized design of such biohybrid system are studied. The optimized microswimmers would be used in targeted cargo, e.g., drug, imaging agent, gene, and RNA, transport and delivery inside the stagnant or low-velocity fluids of the human body as one of their potential biomedical applications.
Science Advances, 3(5):e1602522, American Association for the Advancement of Science, May 2017 (article)
Dynamic self-assembled material systems constantly consume energy to maintain their spatiotemporal structures and functions. Programmable self-assembly translates information from individual parts to the collective whole. Combining dynamic and programmable self-assembly in a single platform opens up the possibilities to investigate both types of self-assembly simultaneously and to explore their synergy. This task is challenging because of the difficulty in finding suitable interactions that are both dissipative and programmable. We present a dynamic and programmable self-assembling material system consisting of spinning at the air-water interface circular magnetic micro-rafts of radius 50 μm and with cosinusoidal edge-height profiles. The cosinusoidal edge-height profiles not only create a net dissipative capillary repulsion that is sustained by continuous torque input but also enable directional assembly of micro-rafts. We uncover the layered arrangement of micro-rafts in the patterns formed by dynamic self-assembly and offer mechanistic insights through a physical model and geometric analysis. Furthermore, we demonstrate programmable self-assembly and show that a 4-fold rotational symmetry encoded in individual micro-rafts translates into 90° bending angles and square-based tiling in the assembled structures of micro-rafts. We anticipate that our dynamic and programmable material system will serve as a model system for studying nonequilibrium dynamics and statistical mechanics in the future
Proceedings of the National Academy of Sciences, pages: 201620344, National Acad Sciences, May 2017 (article)
For adhering to three-dimensional (3D) surfaces or objects, current adhesion systems are limited by a fundamental trade-off between 3D surface conformability and high adhesion strength. This limitation arises from the need for a soft, mechanically compliant interface, which enables conformability to nonflat and irregularly shaped surfaces but significantly reduces the interfacial fracture strength. In this work, we overcome this trade-off with an adhesion-based soft-gripping system that exhibits enhanced fracture strength without sacrificing conformability to nonplanar 3D surfaces. Composed of a gecko-inspired elastomeric microfibrillar adhesive membrane supported by a pressure-controlled deformable gripper body, the proposed soft-gripping system controls the bonding strength by changing its internal pressure and exploiting the mechanics of interfacial equal load sharing. The soft adhesion system can use up to ∼26% of the maximum adhesion of the fibrillar membrane, which is 14× higher than the adhering membrane without load sharing. Our proposed load-sharing method suggests a paradigm for soft adhesion-based gripping and transfer-printing systems that achieves area scaling similar to that of a natural gecko footpad.
In Proc. R. Soc. B, 284(1849):20162867, Febuary 2017 (inproceedings)
Most studies on the adhesive mechanisms of climbing animals have addressed attachment against flat surfaces, yet many animals can climb highly curved surfaces, like twigs and small branches. Here we investigated whether tree frogs use a clamping grip by recording the ground reaction forces on a cylindrical object with either a smooth or anti-adhesive, rough surface. Furthermore, we measured the contact area of fore and hindlimbs against differently sized transparent cylinders and the forces of individual pads and subarticular tubercles in restrained animals. Our study revealed that frogs use friction and normal forces of roughly a similar magnitude for holding on to cylindrical objects. When challenged with climbing a non-adhesive surface, the compressive forces between opposite legs nearly doubled, indicating a stronger clamping grip. In contrast to climbing flat surfaces, frogs increased the contact area on all limbs by engaging not just adhesive pads but also subarticular tubercles on curved surfaces. Our force measurements showed that tubercles can withstand larger shear stresses than pads. SEM images of tubercles revealed a similar structure to that of toe pads including the presence of nanopillars, though channels surrounding epithelial cells were less pronounced. The tubercles' smaller size, proximal location on the toes and shallow cells make them probably less prone to buckling and thus ideal for gripping curved surfaces.
Background: Successful gene delivery requires overcoming both systemic and intracellular obstacles before the nucleic acid cargo can successfully reach its tissue and subcellular target location.
Materials & Methods: Non-viral mechanisms to enable targeting while avoiding off-target delivery have arisen via biological, chemical, and physical engineering strategies.
Discussion: Herein we will discuss the physical parameters in particle design that promote tissue- and cell-targeted delivery of genetic cargo. We will discuss systemic concerns, such as circulation, tissue localization, and clearance, as well as cell-scale obstacles, such as cellular uptake and nucleic acid packaging.
Conclusion: In particular, we will focus on engineering particle shape and size in order to enhance delivery and promote precise targeting. We will also address methods to program or change particle shape in situ using environmentally triggered cues.
arXiv preprint arXiv:1705.06196, May 2017 (article)
Since its development, ingestible wireless endoscopy is considered to be a painless diagnostic method to detect a number of diseases inside GI tract. Medical related engineering companies have made significant improvements in this technology in last decade; however, some major limitations still residue. Localization of the next generation steerable endoscopic capsule robot in six degreeof-freedom (DoF) and active motion control are some of these limitations. The significance of localization capability concerns with the doctors correct diagnosis of the disease area. This paper presents a very robust 6-DoF localization method based on supervised training of an architecture consisting of recurrent networks (RNN) embedded into a convolutional neural network (CNN) to make use of both just-in-moment information obtained by CNN and correlative information across frames obtained by RNN. To our knowledge, our idea of embedding RNNs into a CNN architecture is for the first time proposed in literature. The experimental results show that the proposed RNN-in-CNN architecture performs very well for endoscopic capsule robot localization in cases vignetting, reflection distortions, noise, sudden camera movements and lack of distinguishable features.
IEEE Robotics and Automation Letters, 2(2):927-934, IEEE, January 2017 (article)
The risk of side effects from thrombolytic agents can be minimized by using smaller doses, assisted by mechanical rubbing against blood clots using helical robots. Quantifying this observation, we study the influence of rubbing against clots on their removal rate in vitro. First, we present a hydrodynamic model of the helical robot based on the resistive-force theory to investigate the rubbing behavior of the clots using robot driven by two rotating dipole fields. Second, we experimentally evaluate the influence of the rubbing on the removal rate of the blood clots. Not only do we find that the removal rate of mechanical rubbing (-0.56 ± 0.27 mm3 /min) is approximately three times greater than the dissolution rate of chemical lysis using streptokinase (-0.17 ± 0.032 mm3/min), but we also show that this removal rate can be controlled via the rubbing speed of the robot.
Current Gene Therapy, 17, pages: 000-000, 2017 (article)
There is a growing interest in transdermal delivery systems because of their noninvasive, targeted, and on-demand delivery of gene and drugs. However, efficient penetration of therapeutic compounds into the skin is still challenging largely due to the impermeability of the outermost layer of the skin, known as stratum corneum. Recently, there have been major research activities to enhance the skin penetration depth of pharmacological agents. This article reviews recent advances in the development of various strategies for skin penetration enhancement. We show that approaches such as ultrasound waves, laser, and microneedle patches have successfully been employed to physically disrupt the stratum corneum structure for enhanced transdermal delivery. Rather than physical approaches, several non-physical route have also been utilized for efficient transdermal delivery across the skin barrier. Finally, we discuss some clinical applications of transdermal delivery systems for gene and drug delivery. This paper shows that transdermal delivery devices can potentially function for diverse healthcare and medical applications while further investigations are still necessary for more efficient skin penetration of gene and drugs.
In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is emerging as a novel, minimally invasive diagnostic technology for inspection of the GI tract and diagnosis of a wide range of diseases and pathologies. Since the development of this technology, medical device companies and many research groups have made substantial progress in converting passive capsule endoscopes to robotic active capsule endoscopes with most of the functionality of current active flexible endoscopes. However, robotic capsule endoscopy still has some challenges. In particular, the use of such devices to generate a precise three-dimensional (3D) mapping of the entire inner organ remains an unsolved problem. Such global 3D maps of inner organs would help doctors to detect the location and size of diseased areas more accurately and intuitively, thus permitting more reliable diagnoses. To our knowledge, this paper presents the first complete pipeline for a complete 3D visual map reconstruction of the stomach. The proposed pipeline is modular and includes a preprocessing module, an image registration module, and a final shape-from-shading-based 3D reconstruction module; the 3D map is primarily generated by a combination of image stitching and shape-from-shading techniques, and is updated in a frame-by-frame iterative fashion via capsule motion inside the stomach. A comprehensive quantitative analysis of the proposed 3D reconstruction method is performed using an esophagus gastro duodenoscopy simulator, three different endoscopic cameras, and a 3D optical scanner.
Lab on a Chip, 17(10):1705-1724, Royal Society of Chemistry, 2017 (article)
Untethered micron-scale mobile robots can navigate and non-invasively perform specific tasks inside unprecedented and hard-to-reach inner human body sites and inside enclosed organ-on-a-chip microfluidic devices with live cells. They are aimed to operate robustly and safely in complex physiological environments where they will have a transforming impact in bioengineering and healthcare. Research along this line has already demonstrated significant progress, increasing attention, and high promise over the past several years. The first-generation microrobots, which could deliver therapeutics and other cargo to targeted specific body sites, have just been started to be tested inside small animals toward clinical use. Here, we review frontline advances in design, fabrication, and testing of untethered mobile microrobots for bioengineering applications. We convey the most impactful and recent strategies in actuation, mobility, sensing, and other functional capabilities of mobile microrobots, and discuss their potential advantages and drawbacks to operate inside complex, enclosed and physiologically relevant environments. We lastly draw an outlook to provide directions in the veins of more sophisticated designs and applications, considering biodegradability, immunogenicity, mobility, sensing, and possible medical interventions in complex microenvironments.
IEEE/ASME Transactions on Mechatronics, IEEE, 2017 (article)
Water-running robots have been developed inspired by a basilisk lizard, which demonstrates highly agile, stable, and energy-efficient locomotion on water surfaces. Current water-running robots are not as stable and agile as their biological counterparts. This study shows how the stability of a water-running robot in the pitch direction can be improved by using an active tail to enable increased propulsion. The mobility of the robot is also increased. To generate force in the pitch and yaw directions, a two-degrees-of-freedom tail is implemented with two circular plates to provide drag. We developed two types of dynamic models for pitch and yaw behavior, and the results are recursively calculated by considering the correlation between the models. The relationship between pitch motion and propulsion was analyzed by simulations. The steering behavior of the robot is also validated while considering the pitch behavior. Experiments were conducted to verify the simulation results.
PloS one, 12(4):e0175428, Public Library of Science, April 2017 (article)
The considerable morbidity associated with hospitalized patients and clinics in developed countries due to biofilm formation on biomedical implants and surgical instruments is a heavy economic burden. An alternative to chemically treated surfaces for bactericidal activity started emerging from micro/nanoscale topographical cues in the last decade. Here, we demonstrate a putative antibacterial surface using copper nanowhiskers deposited by molecular beam epitaxy. Furthermore, the control of biological response is based on hydrophobic pinning of water droplets in the Wenzel regime, causing mechanical injury and cell death. Scanning electron microscopy images revealed the details of the surface morphology and non-contact mode laser scanning of the surface revealed the microtopography-associated quantitative parameters. Introducing the bacterial culture over nanowhiskers produces mechanical injury to cells, leading to a reduction in cell density over time due to local pinning of culture medium to whisker surfaces. Extended culture to 72 hours to observe biofilm formation revealed biofilm inhibition with scattered microcolonies and significantly reduced biovolume on nanowhiskers. Therefore, surfaces patterned with copper nanowhiskers can serve as potential antibiofilm surfaces. The topography-based antibacterial surfaces introduce a novel prospect in developing mechanoresponsive nanobiomaterials to reduce the risk of medical device biofilm-associated infections, contrary to chemical leaching of copper as a traditional bactericidal agent.
arXiv preprint arXiv:1705.05444, May 2017 (article)
In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is considered as a minimally invasive novel diagnostic technology to inspect the entire GI tract and to diagnose various diseases and pathologies. Since the development of this technology, medical device companies and many groups have made significant progress to turn such passive capsule endoscopes into robotic active capsule endoscopes to achieve almost all functions of current active flexible endoscopes. However, the use of robotic capsule endoscopy still has some challenges. One such challenge is the precise localization of such active devices in 3D world, which is essential for a precise three-dimensional (3D) mapping of the inner organ. A reliable 3D map of the explored inner organ could assist the doctors to make more intuitive and correct diagnosis. In this paper, we propose to our knowledge for the first time in literature a visual simultaneous localization and mapping (SLAM) method specifically developed for endoscopic capsule robots. The proposed RGB-Depth SLAM method is capable of capturing comprehensive dense globally consistent surfel-based maps of the inner organs explored by an endoscopic capsule robot in real time. This is achieved by using dense frame-to-model camera tracking and windowed surfelbased fusion coupled with frequent model refinement through non-rigid surface deformations.
Bacteria-driven biohybrid microswimmers (bacteriabots), which integrate motile bacterial cells and functional synthetic cargo parts (e.g., microparticles encapsulating drug), are recently studied for targeted drug delivery. However, adhesion of such bacteriabots to the tissues on the site of a disease (which can increase the drug delivery efficiency) is not studied yet. Here, this paper proposes an approach to attach bacteriabots to certain types of epithelial cells (expressing mannose on the membrane), based on the affinity between lectin molecules on the tip of bacterial type I pili and mannose molecules on the epithelial cells. It is shown that the bacteria can anchor their cargo particles to mannose-functionalized surfaces and mannose-expressing cells (ATCC HTB-9) using the lectin–mannose bond. The attachment mechanism is confirmed by comparing the adhesion of bacteriabots fabricated from bacterial strains with or without type I pili to mannose-covered surfaces and cells. The proposed bioadhesive motile system can be further improved by expressing more specific adhesion moieties on the membrane of the bacteria.
Bacteria biohybrids employ the motility and power of swimming bacteria to carry and maneuver microscale particles. They have the potential to perform microdrug and cargo delivery in vivo, but have been limited by poor design, reduced swimming capabilities, and impeded functionality. To address these challenge, motile Escherichia coli are captured inside electropolymerized microtubes, exhibiting the first report of a bacteria microswimmer that does not utilize a spherical particle chassis. Single bacterium becomes partially trapped within the tube and becomes a bioengine to push the microtube though biological media. Microtubes are modified with “smart” material properties for motion control, including a bacteria-attractant polydopamine inner layer, addition of magnetic components for external guidance, and a biochemical kill trigger to cease bacterium swimming on demand. Swimming dynamics of the bacteria biohybrid are quantified by comparing “length of protrusion” of bacteria from the microtubes with respect to changes in angular autocorrelation and swimmer mean squared displacement. The multifunctional microtubular swimmers present a new generation of biocompatible micromotors toward future microbiorobots and minimally invasive medical applications.
We present a robust deep learning based 6 degrees-of-freedom (DoF) localization system for endoscopic capsule robots. Our system mainly focuses on localization of endoscopic capsule robots inside the GI tract using only visual information captured by a mono camera integrated to the robot. The proposed system is a 23-layer deep convolutional neural network (CNN) that is capable to estimate
the pose of the robot in real time using a standard CPU. The dataset for the evaluation of the system was recorded inside a surgical human stomach model with realistic surface texture, softness, and surface liquid properties so that the pre-trained CNN architecture can be transferred confidently into a real endoscopic scenario. An average error of 7.1% and 3.4% for translation and rotation has been obtained, respectively. The results accomplished from the experiments demonstrate that a CNN pre-trained with raw 2D endoscopic images performs accurately inside the GI tract and is robust to various challenges posed by reflection distortions, lens imperfections, vignetting, noise, motion blur, low resolution, and lack of unique landmarks to track.
Advanced Materials, 28(25):5087-5087, May 2016 (article)
Gallium exhibits highly reversible and switchable adhesion when it undergoes a solid–liquid phase transition. The robustness of gallium is notable as it exhibits strong performance on a wide range of smooth and rough surfaces, under both dry and wet conditions. Gallium may therefore find numerous applications in transfer printing, robotics, electronic packaging, and biomedicine.
Applied Physics Letters, 109(3):033701, AIP Publishing, July 2016 (article)
We investigate the microswimming behaviour of robotic sperms in viscous fluids. These robotic sperms are fabricated from polystyrene dissolved in dimethyl formamide and iron-oxide nanoparticles. This composition allows the nanoparticles to be concentrated within the bead of the robotic sperm and provide magnetic dipole, whereas the flexibility of the ultra-thin tail enables flagellated locomotion using magnetic fields in millitesla range. We show that these robotic sperms have similar morphology and swimming behaviour to those of sperm cells. Moreover, we show experimentally that our robotic sperms swim controllably at an average speed of approximately one body length per second (around 125 μm s−1), and they are relatively faster than the microswimmers that depend on planar wave propulsion in low-Reynolds number fluids.
In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pages: 4945-4950, October 2016 (inproceedings)
Many highly dynamic novel mobile robots have been developed being inspired by animals. In this study, we are inspired by a basilisk lizard's ability to run and steer on water surface for a hexapedal robot. The robot has an active tail with a circular plate, which the robot rotates to steer on water. We dynamically modeled the platform and conducted simulations and experiments on steering locomotion with a bang-bang controller. The robot can steer on water by rotating the tail, and the controlled steering locomotion is stable. The dynamic modelling approximates the robot's steering locomotion and the trends of the simulations and experiments are similar, although there are errors between the desired and actual angles. The robot's maneuverability on water can be improved through further research.
In Advanced Intelligent Mechatronics (AIM), 2016 IEEE International Conference on, pages: 1386-1391, July 2016 (inproceedings)
In this paper, we present the analysis of the torque transmitted to a tilted permanent magnet that is to be embedded in a capsule robot to achieve targeted drug delivery. This analysis is carried out by using an analytical model and experimental results for a small cubic permanent magnet that is driven by an external magnetic system made of an array of arc-shaped permanent magnets (ASMs). Our experimental results, which are in agreement with the analytical results, show that the cubic permanent magnet can safely be actuated for inclinations lower than 75° without having to make positional adjustments in the external magnetic system. We have found that with further inclinations, the cubic permanent magnet to be embedded in a drug delivery mechanism may stall. When it stalls, the external magnetic system's position and orientation would have to be adjusted to actuate the cubic permanent magnet and the drug release mechanism. This analysis of the transmitted torque is helpful for the development of real-time control strategies for magnetically articulated devices.
Advanced healthcare materials, 5(18):2325-2331, May 2016 (article)
A surface patterning technique and a specific and strong biotin–streptavidin bonding of bacteria on patterned surfaces are proposed to fabricate Janus particles that are propelled by the attached bacteria. Bacteria-driven Janus microswimmers with diameters larger than 3 μm show enhanced mean propulsion speed. Such microswimmers could be used for future applications such as targeted drug delivery and environmental remediation.