Hostname: page-component-7479d7b7d-jwnkl Total loading time: 0 Render date: 2024-07-09T06:45:11.508Z Has data issue: false hasContentIssue false
  • English
  • Français

Understanding the effects of hand design on embodiment in virtual reality

Published online by Cambridge University Press:  02 March 2023

Jingjing Zhang ,
Mengjie Huang Open the ORCID record for Mengjie Huang [Opens in a new window] ,
Rui Yang ,
Yiqi Wang ,
Xiaohang Tang ,
Ji Han  and
Hai-Ning Liang
Jingjing Zhang
Affiliation:
Design School, Xi'an Jiaotong-Liverpool University, Suzhou, China School of Engineering, University of Liverpool, Liverpool, UK
Mengjie Huang*
Affiliation:
Design School, Xi'an Jiaotong-Liverpool University, Suzhou, China
Rui Yang
Affiliation:
School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou, China
Yiqi Wang
Affiliation:
UCL Interaction Center, University College London, London, UK
Xiaohang Tang
Affiliation:
School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou, China
Ji Han
Affiliation:
Business School, University of Exeter, Exeter, UK
Hai-Ning Liang
Affiliation:
School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou, China
*
Author for correspondence: Mengjie Huang, E-mail: mengjie.huang@xjtlu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Understanding user perceptions of interacting with the virtual world is one of the research focuses in recent years, given the rapid proliferation of virtual reality (VR) and driven to establish the metaverse. Users can generate a familiar connection between their bodies and the virtual world by being embodied in virtual hands, and hand representations can induce users’ embodiment in VR. The sense of embodiment represents the cognitive awareness of one's manifestation and includes three subcomponents: the sense of body ownership, agency and self-location. There is insufficient evidence in the literature about the effects of hand designs on the embodiment, especially based on studying its three subcomponents. This study investigates how virtual hand designs with five realism levels influence the three subcomponents of embodiment in VR. This research employs a self-report questionnaire commonly used in the literature to assess embodiment and evaluates agency and self-location by introducing implicit methods (intentional binding and proprioceptive measurement) derived from psychology. Besides, the objective data of eye tracking is used to explore the connection between embodiment and hand designs, and classifying participants’ eye tracking data to help analyze the link between embodiment and user attention. Overall, this research makes a major contribution through a systematic exploration of users’ embodied experience in VR and offers important evidence of the effects of virtual hand designs on body ownership, agency, and self-location, respectively. In addition, this study provides a valuable reference for further investigation of embodiment through implicit and objective methods, and practical design recommendations for virtual hand design in VR applications.

Keywords

Embodimenteye trackingvirtual hand designvirtual reality
Type
Research Article
Information
AI EDAM , Volume 37 , 2023 , e10
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alchalabi, B, Faubert, J and Labbe, DR (2019) EEG can be used to measure embodiment when controlling a walking self-avatar. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces.Google Scholar
Alzayat, A, Hancock, M and Nacenta, MA (2019) Quantitative measurement of tool embodiment for virtual reality input alternatives. In CHI Conference on Human Factors in Computing Systems Proceedings.CrossRefGoogle Scholar
Argelaguet, F, Hoyet, L, Trico, M and Lécuyer, A (2016) The role of interaction in virtual embodiment: effects of the virtual hand representation. In 2016 IEEE Virtual Reality.CrossRefGoogle Scholar
Batliner, M, Hess, S, Ehrlich-Adam, C, Lohmeyer, Q and Meboldt, M (2020) Automated areas of interest analysis for usability studies of tangible screen-based user interfaces using mobile eye tracking. Artificial Intelligence for Engineering Design Analysis and Manufacturing 34, 505–514.CrossRefGoogle Scholar
Bergström, J, Knibbe, J, Pohl, H and Hornbæk, K (2022) Sense of agency and user experience: is there a link? ACM Transactions on Computer-Human Interaction 29, 28.CrossRefGoogle Scholar
Borgianni, Y and Maccioni, L (2020) Review of the use of neurophysiological and biometric measures in experimental design research. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 34, 248–285.Google Scholar
Botvinick, M and Cohen, J (1998) Rubber hands ‘feel’ touch that eyes see. Nature 391, 756.Google ScholarPubMed
Cui, DX and Mousas, C (2021) An on-site and remote study during the COVID-19 pandemic on virtual hand appearance and tactile feedback. Behaviour and Information Technology, 40, 1278–1291.CrossRefGoogle Scholar
Dewey, JA and Knoblich, G (2014) Do implicit and explicit measures of the sense of agency measure the same thing? PLoS ONE 9, e110118.CrossRefGoogle ScholarPubMed
Dohan, M and Mu, M (2019) Understanding user attention in VR using gaze controlled games. In ACM International Conference on Interactive Experiences for TV and Online Video.Google Scholar
Ehrsson, HH (2012) The concept of body ownership and its relation to multisensory integration. The New Handbook of Multisensory Process.Google Scholar
Freeman, G and Maloney, D (2021) Body, avatar, and me: the presentation and perception of self in social virtual reality. In ACM on Human-Computer Interaction,1–27.CrossRefGoogle Scholar
Fribourg, R, Argelaguet, F, Lécuyer, A and Hoyet, L (2020) Avatar and sense of embodiment: studying the relative preference between appearance, control and point of view. IEEE Transactions on Visualization and Computer Graphics 26, 2062–2072.Google ScholarPubMed
Gonzalez-Franco, M and Peck, TC (2018) Avatar embodiment. Towards a standardised questionnaire. Frontiers in Robotics and AI 5, 74.CrossRefGoogle Scholar
Gorisse, G, Christmann, O, Amato, EA and Richir, S (2017) First- and third-person perspectives in immersive virtual environments: presence and performance analysis of embodied users. Frontiers in Robotics and AI 4, 33.Google Scholar
Heinrich, C, Cook, M, Langlotz, T and Regenbrecht, H (2020) My hands? Importance of personalised virtual hands in a neurorehabilitation scenario. Virtual Reality, 25, 1–18.Google Scholar
Hepperle, D, Purps, CF, Deuchler, J and Woelfel, M (2021) Aspects of visual avatar appearance: self-representation, display type, and uncanny valley. Visual Computer, 38, 1227–1244.CrossRefGoogle ScholarPubMed
Huang, H-C, Lee, Y-T, Chen, W-Y and Liang, C (2017) The sense of 1PP-location contributes to shaping the perceived self-location together with the sense of body-location. Frontiers in Psychology 8, 370.Google ScholarPubMed
Huang, K, Khalil, M, Luciani, E, Melesse, D and Ning, TK (2018) A data-driven approach for gaze tracking. In 14th IEEE International Conference on Signal Processing.Google Scholar
Ingram, LA, Butler, AA, Gandevia, SC and Walsh, LD (2019) Proprioceptive measurements of perceived hand position using pointing and verbal localisation tasks. PLoS ONE 14, e0210911.CrossRefGoogle ScholarPubMed
Kilteni, K, Groten, R and Slater, M (2012) The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments 21, 373–387.CrossRefGoogle Scholar
Kokkinara, E and Slater, M (2014) Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation an a virtual body ownership illusion. Perception 43, 43–58.Google ScholarPubMed
Krogmeier, C and Mousas, C (2020) Eye fixations and electrodermal activity during low-budget virtual reality embodiment. Computer Animation and Virtual Worlds 31, e1941.Google Scholar
Krogmeier, C and Mousas, C (2021) Evidence for a relationship between self-avatar fixations and perceived avatar similarity within low-cost virtual reality embodiment. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops.Google Scholar
Li, F, Lee, C-H, Feng, S, Trappey, A and Gilani, F (2021) Prospective on eye-tracking-based studies in immersive virtual reality. In 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design.Google Scholar
Lu, S, Rawlinson, TG and Harter, D (2019) Embodiment in virtual environments: the role of working memory in experiencing presence as revealed via eye tracking. Journal of Cognitive Education Psychology 18, 223–243.Google Scholar
Mousas, C, Koilias, A, Rekabdar, B, Kao, D and Anastaslou, D (2021) Toward understanding the effects of virtual character appearance on avoidance movement behavior. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR).Google Scholar
Nataraj, R and Sanford, S (2021) Control modification of grasp force covaries agency and performance on rigid and compliant surfaces. Frontiers in Bioengineering Biotechnology 8, 1544.Google ScholarPubMed
Nataraj, R, Hollinger, D, Liu, M and Shah, A (2020 a) Disproportionate positive feedback facilitates sense of agency and performance for a reaching movement task with a virtual hand. PLoS ONE 15, e0233175.Google ScholarPubMed
Nataraj, R, Sanford, S, Shah, A and Liu, M (2020 b) Agency and performance of reach-to-grasp with modified control of a virtual hand: implications for rehabilitation. Frontiers in Human Neuroscience 14, 126.CrossRefGoogle ScholarPubMed
Ogawa, N, Narumi, T and Hirose, M (2019) Virtual hand realism affects object size perception in body-based scaling. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces.Google Scholar
Peck, TC and Gonzalez-Franco, M (2021) Avatar embodiment. A standardised questionnaire. Frontiers in Virtual Reality 1, 44.Google Scholar
Perez-Marcos, D, Slater, M and Sanchez-Vives, MV (2009) Inducing a virtual hand ownership illusion through a brain–computer interface. NeuroReport 20, 589–594.CrossRefGoogle ScholarPubMed
Roth, D and Latoschik, M (2020) Construction of the virtual embodiment questionnaire (VEQ). In IEEE Transactions on Visualization and Computer Graphics, 3546–3556.Google Scholar
Schwind, V, Knierim, P, Tasci, C, Franczak, P, Haas, N and Henze, N (2017) “These are not my hands!” Effect of gender on the perception of avatar hands in virtual reality. In 2017 CHI Conference on Human Factors in Computing Systems.CrossRefGoogle Scholar
Suzuki, K, Lush, P, Seth, AK and Roseboom, W (2019) Intentional binding without intentional action. Psychological Science 30, 842–853.Google ScholarPubMed
Tieri, G, Gioia, A, Scandola, M, Pavone, EF and Aglioti, SM (2017) Visual appearance of a virtual upper limb modulates the temperature of the real hand: a thermal imaging study in immersive virtual reality. European Journal of Neuroscience 45, 1141–1151.CrossRefGoogle Scholar
Toet, A, Kuling, IA, Krom, BN and Van Erp, JB (2020) Toward enhanced teleoperation through embodiment. Frontiers in Robotics and AI 7, 14.Google ScholarPubMed
Van den Bussche, E, Alves, M, Murray, YP and Hughes, G (2020) The effect of cognitive effort on the sense of agency. PLoS ONE 15, 1–20.Google ScholarPubMed
Wan, Z, Yang, R, Huang, M, Zeng, N and Liu, X (2021) A review on transfer learning in EEG signal analysis. Neurocomputing 421, 1–14.Google Scholar
Wang, L, Huang, M, Wang, Y, Liao, K, Yang, R, Zhang, J and Sun, W (2021) Movement modulation in virtual rehabilitation: its influence on agency and motor performance. In IEEE 9th International Conference on Serious Games and Applications for Health.Google Scholar
Wang, L, Huang, M, Yang, R, Liang, HN, Han, J and Sun, Y (2022) Survey of movement reproduction in immersive virtual rehabilitation. IEEE Transactions on Visualization and Computer Graphics, 1–20.Google ScholarPubMed
Wolf, E, Merdan, N, Dollinger, N, Mal, D, Wienrich, C, Botsch, M and Latoschik, ME (2021) The embodiment of photorealistic avatars influences female body weight perception in virtual reality. In 28th IEEE Conference on Virtual Reality and 3D User Interfaces.Google Scholar
Yu, K, Gorbachev, G, Eck, U, Pankratz, F, Navab, N and Roth, D (2021) Avatars for teleconsultation: effects of avatar embodiment techniques on user perception in 3D asymmetric telepresence. IEEE Transactions on Visualization and Computer Graphics 27, 4129–4139.Google ScholarPubMed
Yu, N, Yang, R and Huang, M (2022) Deep common spatial pattern based motor imagery classification with improved objective function. International Journal of Network Dynamics and Intelligence 1, 73–84.Google Scholar
Zhang, J, Huang, M, Zhao, L, Yang, R, Liang, H-N, Han, J and Sun, W (2020) Influence of hand representation design on presence and embodiment in virtual environment. In 2020 13th International Symposium on Computational Intelligence and Design.CrossRefGoogle Scholar
Zhang, J, Huang, M, Liao, K-L and Yang, R (2022 a) Exploring the effect of virtual hand realism on embodiment by subjective ratings and eye tracking. In IEEE International Conference on Cognitive Aspects of Virtual Reality.Google Scholar
Zhang, J, Huang, M, Tang, X, Wang, Y and Yang, R (2022 b) How virtual body continuity with different hand representations influence on user perceptions and task performance. In 2022 15th International Conference on Human System Interaction.Google Scholar
Zhao, G, Li, Y and Xu, Q (2022) From emotion AI to cognitive AI. International Journal of Network Dynamics and Intelligence 1, 65–72.Google Scholar