Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-27T19:01:17.196Z Has data issue: false hasContentIssue false
  • English
  • Français

Motion Adaptation Based on Learning the Manifold of Task and Dynamic Movement Primitive Parameters

Published online by Cambridge University Press:  18 December 2020

Yosef Cohen ,
Or Bar-Shira  and
Sigal Berman Open the ORCID record for Sigal Berman [Opens in a new window]
Yosef Cohen
Affiliation:
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, Israel
Or Bar-Shira
Affiliation:
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, Israel
Sigal Berman*
Affiliation:
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, Israel Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Beer-Sheva, Israel
*
*Corresponding author. E-mail: sigalbe@bgu.ac.il
Rights & Permissions [Opens in a new window]

Summary

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Dynamic movement primitives (DMP) are motion building blocks suitable for real-world tasks. We suggest a methodology for learning the manifold of task and DMP parameters, which facilitates runtime adaptation to changes in task requirements while ensuring predictable and robust performance. For efficient learning, the parameter space is analyzed using principal component analysis and locally linear embedding. Two manifold learning methods: kernel estimation and deep neural networks, are investigated for a ball throwing task in simulation and in a physical environment. Low runtime estimation errors are obtained for both learning methods, with an advantage to kernel estimation when data sets are small.

Keywords

Dynamic movement primitivesKernel estimationDeep Neural networksMotion planningLearning
Type
Article
Information
Robotica , Volume 39 , Issue 7 , July 2021 , pp. 1299 - 1315
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2020. Published by Cambridge University Press

References

Giszter, S. F., Loeb, E., Mussa-Ivaldi, F. A. and Bizzi, E., “Repeatable spatial maps of a few force and joint torque patterns elicited by microstimulation applied throughout the lumbar spinal cord of the spinal frog. Hum,” Mov. Sci. 19, 597–626 (2000).CrossRefGoogle Scholar
Billard, A., Calinon, S., Dillmann, R. and Schaal, S., “Robot Programming by Demonstration,” In: Handbook of Robotics (Siciliano, B., Khatib, O., eds.) (Springer, Berlin, 2008) chapter 59, pp. 1371–1394.CrossRefGoogle Scholar
Seker, M. Y., Imre, M., Piater, J. and Ugur, E., “Conditional Neural Movement Primitives,” Robotics: Science and Systems (RSS) Conference, Freiburg, Germany (2019). https://doi.org/10.15607/RSS.2019.XV.071.CrossRefGoogle Scholar
Argall, B. D., Chernova, S., Veloso, M. and Browning, B., “A survey of robot learning from demonstration,” Rob. Auton. Syst. 57, 469–483 (2009).CrossRefGoogle Scholar
Kormushev, P., Calinon, S. and Caldwell, D. G., “Reinforcement learning in robotics: Applications and real-world challenge,” Robotics. 2(3), 122–148 (2013).CrossRefGoogle Scholar
Kober, J., Bagnell, J. A. and Peters, J., “Reinforcement learning in robotics: A survey,” Int. J. Rob. Res. 32(11), 1238–1274 (2013).CrossRefGoogle Scholar
Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E. and Schaal, S., “From dynamic movement primitives to associative skill memories,” Rob. Auton. Syst. 61(4), 351–361 (2013).CrossRefGoogle Scholar
da Silva, B. C., Baldassarre, G., Konisidaris, G. and Barto, A., “Learning Parameterized Motor Skills on a Humanoid Robot,” IEEE International Conference on Robotics and Automation, Hong Kong, China (2014) pp. 5239–5244.Google Scholar
Muelling, K., Kober, J., Kroemer, O. and Peters, J., “Learning to select and generalize striking movements in robot table tennis,” Int. J. Rob. Res. 32(3), 263–279 (2013).CrossRefGoogle Scholar
Nemec, B. and Ude, A., “Action sequencing using dynamic movement primitives”, Robotica. 30, 837–846 (2011).CrossRefGoogle Scholar
McGovern, A. and Barto, A. G., “Automatic Discovery of Subgoals in Reinforcement Learning Using Diverse Density,” Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, USA (2001).Google Scholar
Kober, J., Oztop, E. and Peters, J., “Reinforcement Learning to Adjust Robot Movements to New Situations,” International Joint Conference on Artificial Intelligence, Barcelona, Spain (2011) pp. 2650–2655.Google Scholar
Stulp, F., Raiola, G., Hoarau, A.1, Ivaldi, S. and Sigaud, O., “Learning Compact Parameterized Skills with a Single Regression”, IEEE-RAS International Conference on Humanoid Robots (Humanoids), Atlanta, USA (2013) pp. 417–422.Google Scholar
Ijspeert, A. J., Nakanishi, J. and Schaal, S., “Movement Imitation with Nonlinear Dynamical Systems in Humanoid Robots,” IEEE International Conference on Robotics and Automation, vol. 2, Washington, DC, USA (2002) pp. 1398–1403.Google Scholar
Ijspeert, A. J., Nakanishi, J. and Schaal, S., “Learning Attractor Landscapes for Learning Motor Primitives,” In: Advances in Neural Information Processing Systems (Becker, S., Thrun, S. and Obermayer, K., eds.), vol. 15 (Vancouver, Canada, 2003) pp. 1523–1530.Google Scholar
Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P. and Schaal, S., “Dynamical movement primitives: Learning attractor models for motor behaviors,” Neural Comput. 25, 328–373 (2013).CrossRefGoogle ScholarPubMed
Hoffmann, H., Pastor, P., Park, D. H. and Schaal, S., “Biologically-Inspired Dynamical Systems for Movement Generation: Automatic Real-Time Goal Adaptation and Obstacle Avoidance,” IEEE International Conference on Robotics and Automation, vol. 2, Kobe, Japan (2009), pp. 1–7.Google Scholar
Pastor, P., Kalakrishnan, M., Chitta, S., Theodorou, E. and Schaal, S., “Skill Learning and Task Outcome Prediction for Manipulation,” IEEE International Conference on Robotics and Automation, Shanghai, China (2011) pp. 9–13.Google Scholar
Tamošiunaite, M., Asfour, T. and Worgotter, F., “Learning to reach by reinforcement learning using receptive field based function approximation with continuous actions,” Biol. Cybern. 100, 249–260 (2009).CrossRefGoogle ScholarPubMed
Ugur, E. and Girgin, H., “Compliant parametric dynamic movement primitives,” Robotica. 1–18 (2019). First view https://doi.org/10.1017/S026357471900078X.CrossRefGoogle Scholar
Girgin, H. and Ugur, E., “Associative Skill Memory Models,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain (2018) pp. 6043–6048.Google Scholar
Tamošiunaite, M., Nemec, B., Ude, A. and Worgotter, F., “Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives,” Rob. Auton. Syst. 59, 910–922 (2011).CrossRefGoogle Scholar
Stulp, F. and Schaal, S., “Hierarchical Reinforcement Learning with Movement Primitives,” IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia (2011) pp. 231–238.Google Scholar
Kalakrishnan, M., Pastor, P., Righetti, L. and Schaal, S., “Learning Objective Functions for Manipulation,” IEEE International Conference on Robotics and Automation, Karlsruhe, Germany (2013) pp. 1323–1328.Google Scholar
Deisenroth, M., Englert, P., Peters, J. and Fox, D., “Multi-Task policy search for robotics,” arXiv preprint, arXiv:1307.0813 (2013).CrossRefGoogle Scholar
Eizicovits, D. and Berman, S., “Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning,” Rob. Auton. Syst. 62, 1208–1219 (2014).CrossRefGoogle Scholar
de Granville, C., Wang, D., Southerland, J., Platt, R., Andrew, J. and Fagg, H., “Grasping Affordances: Learning to Connect Vision to Hand Action,” In: The Path to Autonomous Robots (Sukhatme, G. ed.) (Springer-Verlag Germany, 2009) pp. 1–22.Google Scholar
Rueckert, E., Mundo, J., Paraschos, A., Peters, J. and Neumann, G., “Extracting Low-Dimensional Control Variables for Movement Primitives,” IEEE International Conference on Robotics and Automation, Seattle, USA (2015) pp. 1511–1518.Google Scholar
Paraschos, A., Daniel, C., Peters, J. and Neumann, G., “Probabilistic Movement Primitives,” Advances in Neural Information Processing Systems, vol. 26, Lake Tahoe, USA (2013) pp. 2616–2624.Google Scholar
He, T., Kong, R., Holmes, A. J., Sabuncu, M. R., Eickhoff, S. B., Bzdok, D., Feng, J. and Yeo, B. T. T., “Is Deep Learning Better than Kernel Estimation for Functional Connectivity Prediction of Fluid Intelligence?,” International Workshop on Pattern Recognition in Neuroimaging, Singapore (2018) pp. 1–4.Google Scholar
Mutono, N. C., Anthony Waititu, G. and Kiberia, W. A., “Feed forward neural network versus kernel estimation a case of body mass index and body dimensions,” Am. J. Theor. Appl. Stat. 5(4), 180–185 (2016).CrossRefGoogle Scholar
Basri, R. and Jacobs, D. W., “Efficient Representation of Low-Dimensional Manifolds Using Deep Networks,” International Conference on Learning Representations, Toulon, France (2017).Google Scholar
Jolliffe, I., Principal Component Analysis (Springer-Verlag, New York, 1986).CrossRefGoogle Scholar
Wold, S., Esbensen, K. and Geladi, P., “Principal component analysis,” Chemom. Intell. Lab. Syst. 2(1–3), 37–52 (1987).CrossRefGoogle Scholar
Roweis, S. and Saul, L., “Nonlinear dimensionality reduction by locally linear embedding,” Science. 290(5500), 2323–2326 (2000).CrossRefGoogle ScholarPubMed
Wang, J., Geometric Structure of High-Dimensional Data and Dimensionality Reduction (Springer-Verlag, Berlin, 2012) Chapter 10.Google Scholar
Liu, K., Weissenfeld, A. and Ostermann, J., “Parameterization of mouth images by LLE and PCA for image-based facial animation,” IEEE International Conference on Acoustics, Speech, and Signal Processing, Toulouse, France (2006) pp. 461–464.Google Scholar
Cohen, Y. and Berman, S., “Tight Dynamic Movement Primitives for Complex Trajectory Generation,” IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK (2013) pp. 2402–2407.Google Scholar
Schultz, B., Geometrical Method of Mathematical Physics (Cambridge: Cambridge University Press, 1999).Google Scholar
Racine, J. and Li, Q., “Nonparametric estimation of regression functions with both categorical and continuous data,” J. Economet. 119(1), 99–130 (2004).CrossRefGoogle Scholar
Shinamazaki, H. and Shinomoto, S., “Kernel bandwidth optimization in spike rate estimation,” J. Comput. Neurosci. 29(1–2), 171–182 (2010).CrossRefGoogle Scholar
Sze, V., Chen, Y. H., Yang, T. J. and Emer, J. S., “Efficient processing of deep neural networks: A tutorial and survey,” Proceedings of the IEEE, 105(12), 2295–2329 (2017).Google Scholar
Ruder, S., “An overview of gradient descent optimization algorithms”, arXiv preprint arXiv:1609.04747 (2016).Google Scholar
Glorot, X., Bordes, A. and Bengio, Y., “Deep Sparse Rectifier Neural Networks,” International Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, USA (2011) 315–323.Google Scholar
Bergstra, J. and Bengio, Y., “Random search for hyper-parameter optimization,” J. Mach. Learn. Res. 13, 281–305 (2012).Google Scholar
Corke, P., Robotics, Vision and Control: Fundamental Algorithms in MATLAB (Springer-Verlag, Germany, 2011).CrossRefGoogle Scholar
Prechelt, L., “Early Stopping-But When? In: Neural Networks: Tricks of the Trade (Montavon, G., Orr, G. B. and Müller, K., eds.) (Springer-Verlag, Germany 1998) pp. 55–69.CrossRefGoogle Scholar
Kober, J. and Peters, J., “Policy Search for Motor Primitives in Robotics,” Advances in Neural Information Processing Systems, vol. 22, Vancouver, Canada (2009) pp. 171–203.Google Scholar
Kormushev, P., Calinon, S. and Caldwell, D. G., “Approaches for Learning Human-Like Motor Skills which Require Variable Stiffness During Execution”, IEEE International Conference on Humanoid Robots, Santa Monica, USA (2010).Google Scholar
Kormushev, P., Calinon, S. and Caldwell, D. G., “Robot Motor Skill Coordination with EM-based Reinforcement Learning,” IEEE/RSJ International Conference Intelligent Robots and Systems, on Intelligent Robots and Systems, Taipei, Taiwan (2010) pp. 3232–3237.Google Scholar
Peters, J. and Schaal, S., “Policy Gradient Methods for Robotics”, IEEE/RSJ International Conference Intelligent Robots and Systems, Beijing, China (2006) pp. 2219–2225.Google Scholar
Schaal, S., Mohajerian, P. and Ijspeert, A., “Dynamics Systems vs. Optimal Control — A Unifying View,” In: Progress in Brain Research (Cisek, P., Drew, T. and Kalaska, J. F. eds.), vol. 165 (2007) pp. 425–445.Google Scholar
Schaal, S., Kotosaka, S. and Sternad, D., “Nonlinear Dynamical Systems as Movement Primitives,” IEEE International Conference Humanoid Robotics, Santa Monica, USA (2000).Google Scholar
Pongas, D., Billard, A. and Schaal, S., “Rapid Synchronization and Accurate Phase-Locking of Rhythmic Motor Primitives,” IEEE/RSJ International Conference Intelligent Robots and Systems, Edmonton, Canada (2005) pp. 2911–2916.Google Scholar