No CrossRef data available.
Article contents
Virtual and real: Symbolic and natural experiences with social robots
Published online by Cambridge University Press: 05 April 2023
Abstract
Interactions with social robots are symbolic experiences guided by the pretense that robots depict real people. But they can also be natural experiences that are direct, automatic, and independent of any thoughtful mapping between what is real and depicted. Both experiences are important, both may apply within the same interaction, and they may vary within a person over time.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press
References
Bailenson, J. (2018). Experience on demand: What virtual reality is, how it works, and what it can do. WW Norton.Google Scholar
Bartsch, A., Kalch, A., & Oliver, M. B. (2014). Moved to think: The role of emotional media experiences in stimulating reflective thoughts. Journal of Media Psychology: Theories, Methods, and Applications, 26(3), 125.CrossRefGoogle Scholar
Bolls, P. D., Lang, A., & Potter, R. F. (2001). The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Communication Research, 28(5), 627–651.CrossRefGoogle Scholar
Chen, C., Hensel, L. B., Duan, Y., Ince, R. A., Garrod, O. G., Beskow, J., … Schyns, P. G. (2019). Equipping Social Robots with Culturally-Sensitive Facial Expressions of Emotion using Data-Driven Methods. 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) (pp. 1–8). IEEE.Google Scholar
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12(4), 335–359.Google Scholar
Cummings, J. J., & Reeves, B. (2022). Stimulus sampling and research integrity. In Jussim, L. J., Krosnick, J. A., & Stevens, S. T. (Eds.), Research integrity: Best practices for the social and behavioral sciences (pp. 203–223). Oxford University Press.CrossRefGoogle Scholar
Döring, N., Mohseni, M. R., & Walter, R. (2020). Design, use, and effects of sex dolls and sex robots: Scoping review. Journal of Medical Internet Research, 22(7), e18551.CrossRefGoogle ScholarPubMed
Hoffman, G., & Ju, W. (2014). Designing robots with movement in mind. Journal of Human–Robot Interaction, 3(1), 91–122.Google Scholar
Hu, Y., & Hoffman, G. (2019). Using Skin Texture Change to Design Emotion Expression in Social Robots. 2019 14th ACM/IEEE International Conference on Human–Robot Interaction (HRI) (pp. 2–10). IEEE.CrossRefGoogle Scholar
Jeong, S. H., Cho, H., & Hwang, Y. (2012). Media literacy interventions: A meta-analytic review. Journal of Communication, 62(3), 454–472.CrossRefGoogle Scholar
Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54.CrossRefGoogle ScholarPubMed
Kiilavuori, H., Sariola, V., Peltola, M. J., & Hietanen, J. K. (2021). Making eye contact with a robot: Psychophysiological responses to eye contact with a human and with a humanoid robot. Biological Psychology, 158, 107989.CrossRefGoogle ScholarPubMed
Kramer, A. D., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790.Google ScholarPubMed
Lang, A. (2000). The limited capacity model of mediated message processing. Journal of Communication 50(1), 46–70.CrossRefGoogle Scholar
Lang, A., Dhillon, K., & Dong, Q. (1995). The effects of emotional arousal and valence on television viewers’ cognitive capacity and memory. Journal of Broadcasting & Electronic Media, 39(3), 313–332.CrossRefGoogle Scholar
Li, J. J., Ju, W., & Reeves, B. (2017). Touching a mechanical body: Tactile contact with body parts of a humanoid robot is physiologically arousing. Journal of Human–Robot Interaction, 6(3), 118–130.CrossRefGoogle Scholar
Lim, S., & Reeves, B. (2010). Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player. International Journal of Human–Computer Studies, 68(1–2), 57–68.CrossRefGoogle Scholar
Marmpena, M., Lim, A., & Dahl, T. S. (2018). How does the robot feel? Perception of valence and arousal in emotional body language. Paladyn, Journal of Behavioral Robotics, 9(1), 168–182.CrossRefGoogle Scholar
Meshi, D., Tamir, D. I., & Heekeren, H. R. (2015). The emerging neuroscience of social media. Trends in Cognitive Sciences, 19(12), 771–782.CrossRefGoogle ScholarPubMed
Okita, S. Y., Bailenson, J., & Schwartz, D. L. (2007). The mere belief of social interaction improves learning. In Proceedings of the annual meeting of the cognitive science society (Vol. 29, p. 29). Lawrence Erlbaum Associates.Google Scholar
Reeves, B., Hancock, J., & Liu, S. (2020). How do people perceive social robots and what makes them effective. Technology, Mind and Behavior, 1(1), 1–37.Google Scholar
Reeves, B., Lang, A., Kim, E. Y., & Tatar, D. (1999). The effects of screen size and message content on attention and arousal. Media Psychology, 1(1), 49–67.CrossRefGoogle Scholar
Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people (Vol. 10, p. 236605). .Google Scholar
Reeves, B., Yeykelis, L., & Cummings, J. J. (2016). The use of media in media psychology. Media Psychology, 19(1), 49–71.CrossRefGoogle Scholar
Shepard, R. N. (1990). Mind sights: Original visual illusions, ambiguities, and other anomalies, with a commentary on the play of mind in perception and art. WH Freeman/Times Books/Henry Holt.Google Scholar
Willemse, C. J., & Van Erp, J. B. (2019). Social touch in human–robot interaction: Robot-initiated touches can induce positive responses without extensive prior bonding. International Journal of Social Robotics, 11(2), 285–304.CrossRefGoogle Scholar
Worth, S., & Gross, L. (1974). Symbolic strategies. Journal of Communication, 24(4), 27–39.CrossRefGoogle Scholar
Worth, S., & Gross, L. (2017). Symbolic strategies. In Communication theory (pp. 121–136). Routledge.CrossRefGoogle Scholar
Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, 1–78.Google Scholar
Target article
Related commentaries (29)
Children's interactions with virtual assistants: Moving beyond depictions of social agents
Dancing robots: Social interactions are performed, not depicted
Fictional emotions and emotional reactions to social robots as depictions of social agents
How cultural framing can bias our beliefs about robots and artificial intelligence
You have
Access
The scene is a crowded movie theater. On the big screen you see realistic dinosaurs rendered with advanced computer graphics. When they appear, your first responses are unexpected, involuntary, and quick. Your heart pounds, palms sweat, eyes open wide, body tilts backward, and your brain devises a plan to get up and run. The antidote to that discomfort is the familiar mantra – “Calm down, it's only a movie!”
Repeating “it's only a movie” works because it confirms that the raw pixels projected on the screen show things that are not actually there. The recognition that a picture is merely a depiction gives viewers room to appreciate and interpret the scene, and to consider the intentions of the filmmaker. Clark and Fischer (C&F) provide an excellent map, something missing in media psychology, of the dimensions of depiction and interpretation.
The sweating, however, is different. This is a natural response, the main requirement for which is the mere recognition that there are dinosaurs on the screen (Worth & Gross, Reference Worth and Gross1974, Reference Worth and Gross2017). This and other primitive responses are unfiltered by any thoughtful mapping between real and virtual (Lang, Reference Lang2000), and they signal that the moment may require action rather than interpretation. Evolved over millennia, the responses are not needed to survive a modern movie, but they are nevertheless difficult to circumvent just because the symbols that roused them only mimic reality (Meshi, Tamir, & Heekeren, Reference Meshi, Tamir and Heekeren2015; Reeves & Nass, Reference Reeves and Nass1996; Shepard, Reference Shepard1990). Natural responses seem difficultly related to the concept of depiction, and especially to the elements of interpretation, imagination, and appreciation.
Realism
Much of the history of media technology is about inventions that promote natural responses. Bigger screens with higher resolution, virtual and augmented reality, computer graphics, three-dimensional sound, and better interactivity – all promote a sense of “being there.” And the inventions work. It matters, for example, whether you watch the dinosaurs on a smartphone, in 3D IMAX, or with VR goggles (e.g., Bailenson, Reference Bailenson2018; Reeves, Lang, Kim, & Tatar, Reference Reeves, Lang, Kim and Tatar1999). And the primitive responses influence thoughtful ones; they are memorable (Bolls, Lang, & Potter, Reference Bolls, Lang and Potter2001; Lang, Dhillon, & Dong, Reference Lang, Dhillon and Dong1995), positively evaluated (Bartsch, Kalch, & Oliver, Reference Bartsch, Kalch and Oliver2014), and the excitation often transfers to other contexts (Kramer, Guillory, & Hancock, Reference Kramer, Guillory and Hancock2014).
There are similar advances in robotic realism, including human-like skin textures (Hu & Hoffman, Reference Hu and Hoffman2019), more purposive uses of touching (Willemse & Van Erp, Reference Willemse and Van Erp2019), better body language (Marmpena, Lim, & Dahl, Reference Marmpena, Lim and Dahl2018), and more detailed facial expressions (Chen et al., Reference Chen, Hensel, Duan, Ince, Garrod, Beskow and Schyns2019). These features may give some social robots a commercial edge precisely because they make humans and machines less distinguishable. Even in an imagined Star Trek Holodeck future, it may still be possible to say “it's only a robot!” but increased realism nevertheless favors natural reactions.
Time domains
Depiction seems most relevant to longer time domains, and like for other media technologies when they were new, that domain is the current emphasis in robotics research. When people interpret, construe, imagine, or appreciate, this primarily involves “slow thinking” (Kahneman, Reference Kahneman2011), and C&F cite numerous good examples of how people reason about social robots in this time scale.
Media realism, however, causes quick responses that occur in seconds or less. “Fast thinking” research about social robots is relatively new but increasing. For example, when people touch a robot they show heightened arousal, similar to touching humans (Li, Ju, & Reeves, Reference Li, Ju and Reeves2017). And within seconds, people make judgments about the warmth and competence of social robots, just as they do for people in real life (Reeves, Hancock, & Liu, Reference Reeves, Hancock and Liu2020). Designers are focusing on other primitive features like eye contact (Kiilavuori, Sariola, Peltola, & Hietanen, Reference Kiilavuori, Sariola, Peltola and Hietanen2021), and how robots negotiate physical space (Hoffman & Ju, Reference Hoffman and Ju2014).
Discretionary framing
Media experiences are not only determined by media stimuli. People can choose a frame, at least temporarily. In one relevant research paradigm, people switch between interacting with media characters that they believe are either controlled by a computer or by another real person. The mere belief that people are interacting with a real person (and not with a character that only depicts someone real) results in greater arousal and better learning (Lim & Reeves, Reference Lim and Reeves2010; Okita, Bailenson, & Schwartz, Reference Okita, Bailenson and Schwartz2007).
It is also noteworthy that interventions designed to reduce the negative consequences of media often teach people, and particularly children, how primitive responses are triggered and how media professionals use them to control attention. This can reduce negative effects by teaching people to choose a symbolic experience rather than a natural one (Jeong, Cho, & Hwang, Reference Jeong, Cho and Hwang2012).
Sampling robots
There is likely far more variance between media characters (robots included) than variance between human responses to any one of them (Reeves, Yeykelis, & Cummings, Reference Reeves, Yeykelis and Cummings2016). Several years ago, we cataloged 342 social robots that were used in over 1,000 studies in the last decade (available at https://goo.gl/Gqpzkx). Variance between the robots is impressive and the catalog doesn't even include some of the most interesting current products (e.g., there are no experimental studies that use sex robots as stimulus material; Döring, Mohseni, & Walter, Reference Döring, Mohseni and Walter2020)
The point of the catalog is to show that any selection of a single or few robots can be easily biased. Consequently, stimulus sampling (Clark, Reference Clark1973; Cummings & Reeves, Reference Cummings, Reeves, Jussim, Krosnick and Stevens2022; Judd, Westfall, & Kenny, Reference Judd, Westfall and Kenny2012; Yarkoni, Reference Yarkoni2022) is critical for social robots. A discussion based on robots that are dinosaurs, sex companions, soccer competitors, or dance partners, will be different than one based on characters from Michelangelo and Shakespeare and the most common (and least exciting?) robots in health care and children's learning.
Conclusion
Symbolic and natural experiences are both important for understanding how media characters are experienced. One is not more important than the other, and one cannot be explained by the other. The switching that occurs between these frames offers a different answer to the author's social artifact puzzle about how people can think media characters are real, and at the same time realize that they are mere mechanical artifacts. They are both. And what matters most is how either experience works, which is applied when, and how they might interact over time.
Financial support
The author reports no funding related to this commentary
Competing interest
None.