Header logo is


2006


no image
Learning operational space control

Peters, J., Schaal, S.

In Robotics: Science and Systems II (RSS 2006), pages: 255-262, (Editors: Gaurav S. Sukhatme and Stefan Schaal and Wolfram Burgard and Dieter Fox), Cambridge, MA: MIT Press, RSS , 2006, clmc (inproceedings)

Abstract
While operational space control is of essential importance for robotics and well-understood from an analytical point of view, it can be prohibitively hard to achieve accurate control in face of modeling errors, which are inevitable in complex robots, e.g., humanoid robots. In such cases, learning control methods can offer an interesting alternative to analytical control algorithms. However, the resulting learning problem is ill-defined as it requires to learn an inverse mapping of a usually redundant system, which is well known to suffer from the property of non-covexity of the solution space, i.e., the learning system could generate motor commands that try to steer the robot into physically impossible configurations. A first important insight for this paper is that, nevertheless, a physically correct solution to the inverse problem does exits when learning of the inverse map is performed in a suitable piecewise linear way. The second crucial component for our work is based on a recent insight that many operational space controllers can be understood in terms of a constraint optimal control problem. The cost function associated with this optimal control problem allows us to formulate a learning algorithm that automatically synthesizes a globally consistent desired resolution of redundancy while learning the operational space controller. From the view of machine learning, the learning problem corresponds to a reinforcement learning problem that maximizes an immediate reward and that employs an expectation-maximization policy search algorithm. Evaluations on a three degrees of freedom robot arm illustrate the feasability of our suggested approach.

am ei

link (url) [BibTex]

2006


link (url) [BibTex]


no image
Reinforcement Learning for Parameterized Motor Primitives

Peters, J., Schaal, S.

In Proceedings of the 2006 International Joint Conference on Neural Networks, pages: 73-80, IJCNN, 2006, clmc (inproceedings)

Abstract
One of the major challenges in both action generation for robotics and in the understanding of human motor control is to learn the "building blocks of movement generation", called motor primitives. Motor primitives, as used in this paper, are parameterized control policies such as splines or nonlinear differential equations with desired attractor properties. While a lot of progress has been made in teaching parameterized motor primitives using supervised or imitation learning, the self-improvement by interaction of the system with the environment remains a challenging problem. In this paper, we evaluate different reinforcement learning approaches for improving the performance of parameterized motor primitives. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and outline both established and novel algorithms for the gradient-based improvement of parameterized policies. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm.

am ei

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
An ultrasonic standing-wave-actuated nano-positioning walking robot: piezoelectric-metal composite beam modeling

Son, K. J., Kartik, V., Wickert, J. A., Sitti, M.

Journal of vibration and control, 12(12):1293-1309, Sage Publications, 2006 (article)

pi

[BibTex]

[BibTex]


no image
IEEE TRANSACTIONS ON ROBOTICS

VOLZ, RICHARD A, TARN, TJ, MACIEJEWSKI, ANTHONY A, LEE, SUKHAN, BICCHI, ANTONIO, DE LUCA, ALESSANDRO, LUH, PETER B, TAYLOR, RUSSELL H, BEKEY, GEORGE A, ARAI, HIROHIKO, others

2006 (article)

pi

[BibTex]

[BibTex]


no image
Design methodology for biomimetic propulsion of miniature swimming robots

Behkam, B., Sitti, M.

Trans.-ASME Journal of Dynamic Systems Measurement and Control, 128(1):36, ASME, 2006 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Augmented reality user interface for an atomic force microscope-based nanorobotic system

Vogl, W., Ma, B. K., Sitti, M.

IEEE transactions on nanotechnology, 5(4):397-406, IEEE, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Friction enhancement via micro-patterned wet elastomer adhesives on small intestinal surfaces

Kwon, J., Cheung, E., Park, S., Sitti, M.

Biomedical Materials, 1(4):216, IOP Publishing, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Statistical Learning of LQG controllers

Theodorou, E.

Technical Report-2006-1, Computational Action and Vision Lab University of Minnesota, 2006, clmc (techreport)

am

PDF [BibTex]

PDF [BibTex]


no image
Miniature endoscopic capsule robot using biomimetic micro-patterned adhesives

Karagozler, M. E., Cheung, E., Kwon, J., Sitti, M.

In Biomedical Robotics and Biomechatronics, 2006. BioRob 2006. The First IEEE/RAS-EMBS International Conference on, pages: 105-111, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Compliant and low-cost humidity nanosensors using nanoporous polymer membranes

Yang, B., Aksak, B., Lin, Q., Sitti, M.

Sensors and Actuators B: Chemical, 114(1):254-262, Elsevier, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Task-based and stable telenanomanipulation in a nanoscale virtual environment

Kim, S., Sitti, M.

IEEE Transactions on automation science and engineering, 3(3):240-247, IEEE, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Drawing suspended polymer micro-/nanofibers using glass micropipettes

Nain, A. S., Wong, J. C., Amon, C., Sitti, M.

Applied Physics Letters, 89(18):183105, AIP, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Approximate nearest neighbor regression in very high dimensions

Vijayakumar, S., DSouza, A., Schaal, S.

In Nearest-Neighbor Methods in Learning and Vision, pages: 103-142, (Editors: Shakhnarovich, G.;Darrell, T.;Indyk, P.), Cambridge, MA: MIT Press, 2006, clmc (inbook)

am

link (url) [BibTex]

link (url) [BibTex]


no image
Toward micro wall-climbing robots using biomimetic fibrillar adhesives

Greuter, M., Shah, G., Caprari, G., Tâche, F., Siegwart, R., Sitti, M.

In Proceedings of the 3rd International Symposium on Autonomous Minirobots for Research and Edutainment (AMiRE 2005), pages: 39-46, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Geckobot: A gecko inspired climbing robot using elastomer adhesives

Unver, O., Uneri, A., Aydemir, A., Sitti, M.

In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pages: 2329-2335, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Towards hybrid swimming microrobots: bacteria assisted propulsion of polystyrene beads

Behkam, B., Sitti, M.

In Engineering in Medicine and Biology Society, 2006. EMBS’06. 28th Annual International Conference of the IEEE, pages: 2421-2424, 2006 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Biologically inspired polymer microfibers with spatulate tips as repeatable fibrillar adhesives

Kim, S., Sitti, M.

Applied Physics Letters, 89(26):261911-261911, AIP, 2006 (article)

pi

Project Page [BibTex]


no image
Soft microcontact printing with force control using microrobotic assembly based templates

Tafazzoli, A., Sitti, M.

In Advanced Motion Control, 2006. 9th IEEE International Workshop on, pages: 500-505, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Modeling of the supporting legs for designing biomimetic water strider robots

Song, Y. S., Suhr, S. H., Sitti, M.

In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pages: 2303-2310, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Two-dimensional vision-based autonomous microparticle manipulation using a nanoprobe

Pawashe, C., Sitti, M.

Journal of Micromechatronics, 3(3):285-306, Brill, 2006 (article)

pi

[BibTex]

[BibTex]


no image
A novel water running robot inspired by basilisk lizards

Floyd, S., Keegan, T., Palmisano, J., Sitti, M.

In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pages: 5430-5436, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
A biomimetic climbing robot based on the gecko

Menon, C., Sitti, M.

Journal of Bionic Engineering, 3(3):115-125, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Force-controlled microcontact printing using microassembled particle templates

Tafazzoli, A., Pawashe, C., Sitti, M.

In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pages: 263-268, 2006 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Proximal probes based nanorobotic drawing of polymer micro/nanofibers

Nain, A. S., Amon, C., Sitti, M.

IEEE transactions on nanotechnology, 5(5):499-510, IEEE, 2006 (article)

pi

[BibTex]

[BibTex]


no image
Waalbot: An agile small-scale wall climbing robot utilizing pressure sensitive adhesives

Murphy, M. P., Tso, W., Tanzini, M., Sitti, M.

In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pages: 3411-3416, 2006 (inproceedings)

pi

[BibTex]

[BibTex]

2004


no image
E. Coli Inspired Propulsion for Swimming Microrobots

Behkam, Bahareh, Sitti, Metin

pages: 1037–1041, 2004 (article)

Abstract
Medical applications are among the most fascinating areas of microrobotics. For long, scientists have dreamed of miniature smart devices that can travel inside the human body and carry out a host of complex operations such as minimally invasive surgery (MIS), highly localized drug delivery, and screening for diseases that are in their very early stages. Still a distant dream, significant progress in micro and nanotechnology brings us closer to materializing it. For such a miniature device to be injected into the body, it has to be 800 μm or smaller in diameter. Miniature, safe and energy efficient propulsion systems hold the key to maturing this technology but they pose significant challenges. Scaling the macroscale natation mechanisms to micro/nano length scales is unfeasible. It has been estimated that a vibrating-fin driven swimming robot shorter than 6 mm can not overcome the viscous drag forces in water. In this paper, the authors propose a new type of propulsion inspired by the motility mechanism of bacteria with peritrichous flagellation, such as Escherichia coli, Salmonella typhimurium and Serratia marcescens. The perfomance of the propulsive mechanism is estimated by modeling the dynamics of the motion. The motion of the moving organelle is simulated and key parameters such as velocity, distribution of force and power requirments for different configurations of the tail are determined theoretically. In order to validate the theoretical result, a scaled up model of the swimming robot is fabricated and characterized in silicone oil using the Buckingham PI theorem for scaling. The results are compared with the theoretically computed values. These robots are intended to swim in stagnation/low velocity biofluid and reach currently inaccessible areas of the human body for disease inspection and possibly treatment. Potential target regions to use these robots include eyeball cavity, cerebrospinal fluid and the urinary system.

pi

link (url) DOI [BibTex]

2004


link (url) DOI [BibTex]


no image
E. coli inspired propulsion for swimming microrobots

Behkam, B., Sitti, M.

In ASME 2004 International Mechanical Engineering Congress and Exposition, pages: 1037-1041, 2004 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Dynamic modes of nanoparticle motion during nanoprobe-based manipulation

Tafazzoli, A., Sitti, M.

In Nanotechnology, 2004. 4th IEEE Conference on, pages: 35-37, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Modeling and design of biomimetic adhesives inspired by gecko foot-hairs

Shah, G. J., Sitti, M.

In Robotics and Biomimetics, 2004. ROBIO 2004. IEEE International Conference on, pages: 873-878, 2004 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Discovering optimal imitation strategies

Billard, A., Epars, Y., Calinon, S., Cheng, G., Schaal, S.

Robotics and Autonomous Systems, 47(2-3):68-77, 2004, clmc (article)

Abstract
This paper develops a general policy for learning relevant features of an imitation task. We restrict our study to imitation of manipulative tasks or of gestures. The imitation process is modeled as a hierarchical optimization system, which minimizes the discrepancy between two multi-dimensional datasets. To classify across manipulation strategies, we apply a probabilistic analysis to data in Cartesian and joint spaces. We determine a general metric that optimizes the policy of task reproduction, following strategy determination. The model successfully discovers strategies in six different imitative tasks and controls task reproduction by a full body humanoid robot.

am

[BibTex]

[BibTex]


no image
Learning Composite Adaptive Control for a Class of Nonlinear Systems

Nakanishi, J., Farrell, J. A., Schaal, S.

In IEEE International Conference on Robotics and Automation, pages: 2647-2652, New Orleans, LA, USA, April 2004, 2004, clmc (inproceedings)

am

link (url) [BibTex]

link (url) [BibTex]


no image
Rhythmic movement is not discrete

Schaal, S., Sternad, D., Osu, R., Kawato, M.

Nature Neuroscience, 7(10):1137-1144, 2004, clmc (article)

Abstract
Rhythmic movements, like walking, chewing, or scratching, are phylogenetically old mo-tor behaviors found in many organisms, ranging from insects to primates. In contrast, discrete movements, like reaching, grasping, or kicking, are behaviors that have reached sophistication primarily in younger species, particularly in primates. Neurophysiological and computational research on arm motor control has focused almost exclusively on dis-crete movements, essentially assuming similar neural circuitry for rhythmic tasks. In con-trast, many behavioral studies focused on rhythmic models, subsuming discrete move-ment as a special case. Here, using a human functional neuroimaging experiment, we show that in addition to areas activated in rhythmic movement, discrete movement in-volves several higher cortical planning areas, despite both movement conditions were confined to the same single wrist joint. These results provide the first neuroscientific evi-dence that rhythmic arm movement cannot be part of a more general discrete movement system, and may require separate neurophysiological and theoretical treatment.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Augmented reality user interface for nanomanipulation using atomic force microscopes

Vogl, W., Sitti, M., Ehrenstrasser, M., Zäh, M.

In Proc. of Eurohaptics, pages: 413-416, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
WaalBots for Space applications

Menon, C., Murphy, M., Angrilli, F., Sitti, M.

In 55th IAC Conference, Vancouver, Canada, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Learning from demonstration and adaptation of biped locomotion

Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.

Robotics and Autonomous Systems, 47(2-3):79-91, 2004, clmc (article)

Abstract
In this paper, we introduce a framework for learning biped locomotion using dynamical movement primitives based on non-linear oscillators. Our ultimate goal is to establish a design principle of a controller in order to achieve natural human-like locomotion. We suggest dynamical movement primitives as a central pattern generator (CPG) of a biped robot, an approach we have previously proposed for learning and encoding complex human movements. Demonstrated trajectories are learned through movement primitives by locally weighted regression, and the frequency of the learned trajectories is adjusted automatically by a novel frequency adaptation algorithmbased on phase resetting and entrainment of coupled oscillators. Numerical simulations and experimental implementation on a physical robot demonstrate the effectiveness of the proposed locomotioncontroller.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Towards Tractable Parameter-Free Statistical Learning (Phd Thesis)

D’Souza, A

Department of Computer Science, University of Southern California, Los Angeles, 2004, clmc (phdthesis)

am

link (url) [BibTex]

link (url) [BibTex]


no image
A framework for learning biped locomotion with dynamic movement primitives

Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.

In IEEE-RAS/RSJ International Conference on Humanoid Robots (Humanoids 2004), IEEE, Los Angeles, CA: Nov.10-12, Santa Monica, CA, 2004, clmc (inproceedings)

Abstract
This article summarizes our framework for learning biped locomotion using dynamical movement primitives based on nonlinear oscillators. Our ultimate goal is to establish a design principle of a controller in order to achieve natural human-like locomotion. We suggest dynamical movement primitives as a central pattern generator (CPG) of a biped robot, an approach we have previously proposed for learning and encoding complex human movements. Demonstrated trajectories are learned through movement primitives by locally weighted regression, and the frequency of the learned trajectories is adjusted automatically by a frequency adaptation algorithm based on phase resetting and entrainment of coupled oscillators. Numerical simulations and experimental implementation on a physical robot demonstrate the effectiveness of the proposed locomotion controller. Furthermore, we demonstrate that phase resetting contributes to robustness against external perturbations and environmental changes by numerical simulations and experiments.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Learning Motor Primitives with Reinforcement Learning

Peters, J., Schaal, S.

In Proceedings of the 11th Joint Symposium on Neural Computation, http://resolver.caltech.edu/CaltechJSNC:2004.poster020, 2004, clmc (inproceedings)

Abstract
One of the major challenges in action generation for robotics and in the understanding of human motor control is to learn the "building blocks of move- ment generation," or more precisely, motor primitives. Recently, Ijspeert et al. [1, 2] suggested a novel framework how to use nonlinear dynamical systems as motor primitives. While a lot of progress has been made in teaching these mo- tor primitives using supervised or imitation learning, the self-improvement by interaction of the system with the environment remains a challenging problem. In this poster, we evaluate different reinforcement learning approaches can be used in order to improve the performance of motor primitives. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and line out how these lead to a novel algorithm which is based on natural policy gradients [3]. We compare this algorithm to previous reinforcement learning algorithms in the context of dynamic motor primitive learning, and show that it outperforms these by at least an order of magnitude. We demonstrate the efficiency of the resulting reinforcement learning method for creating complex behaviors for automous robotics. The studied behaviors will include both discrete, finite tasks such as baseball swings, as well as complex rhythmic patterns as they occur in biped locomotion

am

[BibTex]

[BibTex]


no image
Atomic force microscope probe based controlled pushing for nanotribological characterization

Sitti, M.

IEEE/ASME Transactions on mechatronics, 9(2):343-349, IEEE, 2004 (article)

pi

[BibTex]

[BibTex]


no image
Dynamic behavior and simulation of nanoparticle sliding during nanoprobe-based positioning

Tafazzoli, A., Sitti, M.

In Proc. ASME International Mechanical Engineering Conference, 19, pages: 32, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Three-dimensional nanoscale manipulation and manufacturing using proximal probes: controlled pulling of polymer micro/nanofibers

Nain, A. S., Amon, C., Sitti, M.

In Mechatronics, 2004. ICM’04. Proceedings of the IEEE International Conference on, pages: 224-230, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Feedback error learning and nonlinear adaptive control

Nakanishi, J., Schaal, S.

Neural Networks, 17(10):1453-1465, 2004, clmc (article)

Abstract
In this paper, we present our theoretical investigations of the technique of feedback error learning (FEL) from the viewpoint of adaptive control. We first discuss the relationship between FEL and nonlinear adaptive control with adaptive feedback linearization, and show that FEL can be interpreted as a form of nonlinear adaptive control. Second, we present a Lyapunov analysis suggesting that the condition of strictly positive realness (SPR) associated with the tracking error dynamics is a sufficient condition for asymptotic stability of the closed-loop dynamics. Specifically, for a class of second order SISO systems, we show that this condition reduces to KD^2 > KP; where KP and KD are positive position and velocity feedback gains, respectively. Moreover, we provide a ÔpassivityÕ-based stability analysis which suggests that SPR of the tracking error dynamics is a necessary and sufficient condition for asymptotic hyperstability. Thus, the condition KD^2>KP mentioned above is not only a sufficient but also necessary condition to guarantee asymptotic hyperstability of FEL, i.e. the tracking error is bounded and asymptotically converges to zero. As a further point, we explore the adaptive control and FEL framework for feedforward control formulations, and derive an additional sufficient condition for asymptotic stability in the sense of Lyapunov. Finally, we present numerical simulations to illustrate the stability properties of FEL obtained from our mathematical analysis.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Computational approaches to motor learning by imitation

Schaal, S., Ijspeert, A., Billard, A.

In The Neuroscience of Social Interaction, (1431):199-218, (Editors: Frith, C. D.;Wolpert, D.), Oxford University Press, Oxford, 2004, clmc (inbook)

Abstract
Movement imitation requires a complex set of mechanisms that map an observed movement of a teacher onto one's own movement apparatus. Relevant problems include movement recognition, pose estimation, pose tracking, body correspondence, coordinate transformation from external to egocentric space, matching of observed against previously learned movement, resolution of redundant degrees-of-freedom that are unconstrained by the observation, suitable movement representations for imitation, modularization of motor control, etc. All of these topics by themselves are active research problems in computational and neurobiological sciences, such that their combination into a complete imitation system remains a daunting undertaking - indeed, one could argue that we need to understand the complete perception-action loop. As a strategy to untangle the complexity of imitation, this paper will examine imitation purely from a computational point of view, i.e. we will review statistical and mathematical approaches that have been suggested for tackling parts of the imitation problem, and discuss their merits, disadvantages and underlying principles. Given the focus on action recognition of other contributions in this special issue, this paper will primarily emphasize the motor side of imitation, assuming that a perceptual system has already identified important features of a demonstrated movement and created their corresponding spatial information. Based on the formalization of motor control in terms of control policies and their associated performance criteria, useful taxonomies of imitation learning can be generated that clarify different approaches and future research directions.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Micro-and nano-scale robotics

Sitti, M.

In American Control Conference, 2004. Proceedings of the 2004, 1, pages: 1-8, 2004 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Gecko inspired surface climbing robots

Menon, C., Murphy, M., Sitti, M.

In Robotics and Biomimetics, 2004. ROBIO 2004. IEEE International Conference on, pages: 431-436, 2004 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]

1992


no image
Ins CAD integrierte Kostenkalkulation (CAD-Integrated Cost Calculation)

Ehrlenspiel, K., Schaal, S.

Konstruktion 44, 12, pages: 407-414, 1992, clmc (article)

am

[BibTex]

1992


[BibTex]


no image
Integrierte Wissensverarbeitung mit CAD am Beispiel der konstruktionsbegleitenden Kalkulation (Ways to smarter CAD Systems)

Schaal, S.

Hanser 1992. (Konstruktionstechnik München Band 8). Zugl. München: TU Diss., München, 1992, clmc (book)

am

[BibTex]

[BibTex]


no image
Informationssysteme mit CAD (Information systems within CAD)

Schaal, S.

In CAD/CAM Grundlagen, pages: 199-204, (Editors: Milberg, J.), Springer, Buchreihe CIM-TT. Berlin, 1992, clmc (inbook)

am

[BibTex]

[BibTex]


no image
What should be learned?

Schaal, S., Atkeson, C. G., Botros, S.

In Proceedings of Seventh Yale Workshop on Adaptive and Learning Systems, pages: 199-204, New Haven, CT, May 20-22, 1992, clmc (inproceedings)

am

[BibTex]

[BibTex]