Hostname: page-component-77f85d65b8-pkds5 Total loading time: 0 Render date: 2026-04-21T13:49:03.111Z Has data issue: false hasContentIssue false
  • English
  • Français

Toward an Intuitive and Iterative 6D Virtual Guide Programming Framework for Assisted Human–Robot Comanipulation

Published online by Cambridge University Press:  03 February 2020

Susana Sánchez Restrepo ,
Gennaro Raiola Open the ORCID record for Gennaro Raiola [Opens in a new window] ,
Joris Guerry ,
Evelyn D’Elia ,
Xavier Lamy  and
Daniel Sidobre
Susana Sánchez Restrepo
Affiliation:
Interactive Robotics Laboratory (LRI), CEA-List, Gif-sur-Yvette, France E-mails: xavier.lamy@cea.fr, susisanchezr@gmail.com LAAS-CNRS, University of Toulouse, CNRS, UPS,Toulouse, France E-mail: daniel.sidobre@laas.fr
Gennaro Raiola*
Affiliation:
Interactive Robotics Laboratory (LRI), CEA-List, Gif-sur-Yvette, France E-mails: xavier.lamy@cea.fr, susisanchezr@gmail.com Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy E-mails: gennaro.raiola@gmail.com, edelia111@gmail.com
Joris Guerry
Affiliation:
EDF R&D, Chatou, France E-mail: joguerry@gmail.com
Evelyn D’Elia
Affiliation:
Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy E-mails: gennaro.raiola@gmail.com, edelia111@gmail.com
Xavier Lamy
Affiliation:
Interactive Robotics Laboratory (LRI), CEA-List, Gif-sur-Yvette, France E-mails: xavier.lamy@cea.fr, susisanchezr@gmail.com
Daniel Sidobre
Affiliation:
LAAS-CNRS, University of Toulouse, CNRS, UPS,Toulouse, France E-mail: daniel.sidobre@laas.fr
*
*Corresponding author. E-mail: gennaro.raiola@gmail.com
Get access Rights & Permissions [Opens in a new window]

Summary

In human–robot comanipulation, virtual guides are an important tool used to assist the human worker as they constrain the movement of the robot to improve the task accuracy and to avoid undesirable effects, such as collisions with the environment. Consequently, the physical effort and cognitive overload are reduced during accomplishment of comanipulative tasks. However, the construction of virtual guides often requires expert knowledge and modeling of the task, which restricts the usefulness of virtual guides to scenarios with fixed constraints. Moreover, few approaches have addressed the implementation of virtual guides enforcing orientation constraints and, when done, these approaches have treated translation and orientation separately, and consequently there is no synchronization of the translational and rotational motions. To overcome these challenges and enhance the programming flexibility of virtual guides, we present a new framework that allows the user to create 6D virtual guides through XSplines which we define as a combination of Akima splines for the translation component and spherical cubic interpolation of quaternions for the orientation component. For complex tasks, the user is able to initially define a 3D virtual guide and then use this assistance in translational motion to concentrate only on defining the orientations along the path. It is also possible for the user to modify a particular point or portion of a guide while being assisted by it. We demonstrate in an industrial scenario that these innovations provide an intuitive solution to extend the use of virtual guides to 6 degrees of freedom and increase the human worker’s comfort during the programming phase of these guides in an assisted human–robot comanipulation context.

Keywords

Human-robot interactionKinesthetic teachingVirtual guidesSplines

Information

Type
Articles
Information
Robotica , Volume 38 , Special Issue 10: Human–Robot Interaction (HRI) , October 2020 , pp. 1778 - 1806
Copyright
© The Author(s) 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Lin, H. C., Mills, K., Kazanzides, P., Hager, G. D., Marayong, P., Okamura, A. M. and Karam, R., “Portability and Applicability of Virtual Fixtures Across Medical and Manufacturing Tasks,” Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006, May 2006, pp. 225–230.Google Scholar
Bettini, A., Marayong, P., Member, S., Lang, S., Okamura, A. M. and Hager, G. D., “Vision Assisted Control for Manipulation using Virtual Fixtures,” International Conference on Intelligent Robots and Systems (IROS) (2004) pp. 1171–1176.Google Scholar
Raiola, G., Restrepo, S. S., Chevalier, P., Rodriguez-Ayerbe, P., Lamy, X., Tliba, S. and Stulp, F., “Co-manipulation with a library of virtual guiding fixtures,” Auto. Robot. 42(5), 1037–1051 (2018).CrossRefGoogle Scholar
Rosenberg, L., “Virtual Fixtures: Perceptual Tools for Telerobotic Manipulation,” IEEE Virtual Reality Annual International Symposium (1993).CrossRefGoogle Scholar
Davies, B., Jakopec, M., Harris, S. J., Baena, F. R. Y., Barrett, A., Evangelidis, A., Gomes, P., Henckel, J. and Cobb, J., “Active-constraint robotics for surgery,” Proc. IEEE 94(9), 1696–1704 (2006).CrossRefGoogle Scholar
Colgate, J., Peshkin, M. and Klostermeyer, S., “Intelligent Assist Devices in Industrial Applications: A Review,” 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. (IROS 2003). Proceedings, October 2003.Google Scholar
Bowyer, S. A., Davies, B. L. and Baena, F. R. y, “Active constraints/virtual fixtures: A survey,” IEEE Trans. Robot. 30(1), 138–157 (2014).Google Scholar
David, O., Russotto, F.-X., Da Silva Simoes, M. and Measson, Y., “Collision avoidance, virtual guides and advanced supervisory control teleoperation techniques for high-tech construction: Framework design,” Auto. Construct. 44, 63–72 (2014).Google Scholar
Xia, T., Léonard, S., Kandaswamy, I., Blank, A., Whitcomb, L. L. and Kazanzides, P., “Model-Based Telerobotic Control with Virtual Fixtures for Satellite Servicing Tasks,” 2013 IEEE International Conference on Robotics and Automation, May 2013, pp. 1479–1484.Google Scholar
Abbott, J., Marayong, P. and Okamura, A., “Haptic Virtual Fixtures for Robot-Assisted Manipulation,” In: Springer Tracts in Advanced Robotics, vol. 28 (Springer, Berlin, Heidelberg, 2007), pp. 49–67.CrossRefGoogle Scholar
Marayong, P., Li, M., Okamura, A. M. and Hager, G. D., “Spatial Motion Constraints: Theory and Demonstrations for Robot Guidance using Virtual Fixtures,” ICRA (IEEE, 2003), pp. 1954–1959.Google Scholar
Abbott, J. J. and Okamura, A. M., “Virtual Fixture Architectures for Telemanipulation,” 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), vol. 2, IEEE (2003).Google Scholar
Joly, L. and Andriot, C., “Imposing Motion Constraints to a Force Reflecting Telerobot Through Real-Time Simulation of a Virtual Mechanism,” 1995 IEEE International Conference on Robotics and Automation, 1995. Proceedings, May 1995.Google Scholar
Pezzementi, Z., Hager, G. D. and Okamura, A. M., “Dynamic Guidance with Pseudoadmittance Virtual Fixtures,” IEEE International Conference on Robotics and Automation (2007) pp. 1761–1767.Google Scholar
Aarno, D., Ekvall, S. and Kragic, D., “Adaptive Virtual Fixtures for Machine-Assisted Teleoperation Tasks,” Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005, (2005).Google Scholar
Kuang, A., Payandeh, S., Zheng, B., Henigman, F. and MacKenzie, C., “Assembling Virtual Fixtures for Guidance in Training Environments,” Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS’04 (2004), pp. 367–374.Google Scholar
Pruks, V., Farkhatdinov, I. and Ryu, J.-H., “Preliminary Study on Real-Time Interactive Virtual Fixture Generation Method for Shared Teleoperation in Unstructured Environments,” International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (Springer, 2018) pp. 648–659.Google Scholar
Calinon, S., D’halluin, F., Sauser, E., Caldwell, D. and Billard, A., “Learning and reproduction of gestures by imitation,” IEEE Robot. Auto. Mag. 17(2), 44–54 (2010).CrossRefGoogle Scholar
Lee, D. and Ott, C., “Incremental kinesthetic teaching of motion primitives using the motion refinement tube,” Auto. Robot. 31(2–3), 115–131 (2011).Google Scholar
Zeestraten, M. J., Havoutis, I., Silvério, J., Calinon, S. and Caldwell, D. G., “An approach for imitation learning on Riemannian manifolds,” IEEE Robot. Auto. Lett. 2(3), 1240–1247 (2017).CrossRefGoogle Scholar
Kronander, K. and Billard, A., “Learning compliant manipulation through kinesthetic and tactile human-robot interaction,” IEEE Trans. Haptics 7(3), 367–380 (2014).CrossRefGoogle ScholarPubMed
Bettini, A., Lang, S., Okamura, A. and Hager, G., “Vision Assisted Control for Manipulation using Virtual Fixtures,” 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001. Proceedings, vol. 2 (IEEE, 2001) pp. 1171–1176.Google Scholar
Nolin, J. T., Stemniski, P. M. and Okamura, A. M., “Activation Cues and Force Scaling Methods for Virtual Fixtures,” Proceedings of the 11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2003) pp. 404–409.Google Scholar
Prada, R. and Payandeh, S., “A Study on Design and Analysis of Virtual Fixtures for Cutting in Training Environments,” Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint (IEEE, 2005) pp. 375–380.Google Scholar
Li, M., Ishii, M. and Taylor, R. H., “Spatial motion constraints using virtual fixtures generated by anatomy,” IEEE Trans. Robotic. 23(1), 4–19 (2007).CrossRefGoogle Scholar
Bowyer, S. A. and Baena, F. R. y, “Dynamic Frictional Constraints for Robot Assisted Surgery,” World Haptics Conference (WHC), 2013 (2013) pp. 319–324.Google Scholar
Castillo-Cruces, R. A. and Wahrburg, J., “Virtual fixtures with autonomous error compensation for human–robot cooperative tasks,” Robotica 28(2), 267–277 (2010).CrossRefGoogle Scholar
Bowyer, S. A. and Baena, F. R. y, “Dynamic Frictional Constraints in Translation and Rotation,” in 2014 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2014) pp. 2685–2692.CrossRefGoogle Scholar
Bowyer, S. A. and Baena, F. R. y, “Dissipative control for physical human–robot interaction,” IEEE Trans. Robot. 31(6), 1281–1293 (2015).CrossRefGoogle Scholar
Zhang, D., Wang, L., Gu, J., Li, Z. and Chen, K., “Realization of Spatial Compliant Virtual Fixture using Eigenscrews,” 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2012) pp. 1506–1509.Google Scholar
Zhang, D., Zhu, Q., Xiong, J. and Wang, L., “Dynamic virtual fixture on the Euclidean group for admittance-type manipulator in deforming environments,” Biomed. Eng. Online 13(1), 51 (2014).CrossRefGoogle Scholar
Rozo, L., Calinon, S. and Caldwell, D., “Learning Force and Position Constraints in Human-Robot Cooperative Transportation,” 2014 RO-MAN: The 23rd IEEE International Symposium on Robot and Human Interactive Communication (2014).CrossRefGoogle Scholar
Boy, E. S., Burdet, E., Teo, C. L. and Colgate, J., “Investigation of motion guidance with scooter cobot and collaborative learning,” IEEE Trans. Robot. 23(2), 245–255 (2007).CrossRefGoogle Scholar
Mollard, Y., Munzer, T., Baisero, A., Toussaint, M. and Lopes, M., “Robot Programming from Demonstration, Feedback and Transfer,” 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015).CrossRefGoogle Scholar
Kastritsi, T., Dimeas, F. and Doulgeri, Z., “Progressive automation with dmp synchronization and variable stiffness control,” IEEE Robot. Auto. Lett. 3(4), 3789–3796 (2018).CrossRefGoogle Scholar
Peternel, L., Petrič, T. and Babič, J., “Human-in-the-Loop Approach for Teaching Robot Assembly Tasks using Impedance Control Interface,” 2015 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2015) pp. 1497–1502.CrossRefGoogle Scholar
Selvaggio, M., Fontanelli, G. A., Ficuciello, F., Villani, L. and Siciliano, B., “Passive virtual fixtures adaptation in minimally invasive robotic surgery,” IEEE Robot. Auto. Lett. 3(4), 3129–3136 (2018).CrossRefGoogle Scholar
Martin Tykal, A. M. and Kyrki, V., “Incrementally Assisted Kinesthetic Teaching for Programming by Demonstration,” 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (2016).CrossRefGoogle Scholar
Kosuge, K., Itoh, T., Fukuda, T. and Otsuka, M., “Tele-Manipulation System Based on Task-Oriented Virtual Tool,” 1995 IEEE International Conference on Robotics and Automation, 1995. Proceedings (1995).Google Scholar
Sanchez Restrepo, S., Raiola, G., Chevalier, P., Lamy, X. and Sidobre, D., “Iterative Virtual Guides Programming for Human-Robot Comanipulation,” IEEE International Conference on Advanced Intelligent Mechatronics (AIM) (2017).CrossRefGoogle Scholar
Dam, E. B., Koch, M. and Lillholm, M., Quaternions, Interpolation and Animation, vol. 2. Datalogisk Institut, Københavns Universitet (1998).Google Scholar
Wisama Khalil, E. D., Modélisation, identification et commande des robots (Hermès science, Paris, janvier 1999).Google Scholar
Hogan, N., “On the stability of manipulators performing contact tasks,” IEEE J. Robot. Auto. 4(6), 677–686 (1988).CrossRefGoogle Scholar
Akima, H., “A new method of interpolation and smooth curve fitting based on local procedures,” J. ACM 17(4), 589–602 (1970).CrossRefGoogle Scholar
Joly, L., Commande hybride position/force pour la teleoperation: une approche basée sur des analogies mécaniques Ph.D. Thesis (Paris 6, 1997).Google Scholar
Shoemake, K., “Animating rotation with quaternion curves,” ACM SIGGRAPH Comput. Graph. 19(3), 245–254, ACM (1985).CrossRefGoogle Scholar
Eberly, D., Quaternion Algebra and Calculus, vol. 26 (Magic Software Inc, Washington, 2002).Google Scholar
Farin, G., Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide (Elsevier, Philadelphia, USA, 2014).Google Scholar
Fritsch, F. N. and Carlson, R. E., “Monotone piecewise cubic interpolation,” SIAM J. Numer. Anal. 17(2), 238–246 (1980).CrossRefGoogle Scholar
Hanson, A. J., “Visualizing Quaternions,” ACM SIGGRAPH 2005 Courses (ACM, 2005) p. 1.Google Scholar
Benjamini, Y. and Hochberg, Y., “Controlling the false discovery rate: A practical and powerful approach to multiple testing,” J. R. Stat. Soc. Ser. B (Methodolog.) 57(1), 289–300 (1995).Google Scholar
Farel, R., Kchir, S., Lamy, X. and Grossard, M., “Challenges in Sustainable Manufacturing with Industrial and Collaborative Robots: A Case Study,” ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (American Society of Mechanical Engineers, 2018) pp. V004T05A040–V004T05A040.Google Scholar