Hostname: page-component-7bb8b95d7b-nptnm Total loading time: 0 Render date: 2024-09-08T11:14:10.547Z Has data issue: false hasContentIssue false
  • English
  • Français

The Action-Sentence Compatibility Effect in ASL: the role of semantics vs. perception*

Published online by Cambridge University Press:  21 November 2014

KRISTEN SECORA  and
KAREN EMMOREY
KRISTEN SECORA*
Affiliation:
San Diego State University, and University of California San Diego
KAREN EMMOREY
Affiliation:
San Diego State University
*
†Address for correspondence: Kristen Secora, Lab for Language and Cognitive Neuroscience, 6495 Alvarado Road, Suite 200, San Diego, CA 92120. tel: (619) 594-8049; fax: (619) 594-8056; e-mail: ksecora@ucsd.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Embodied theories of cognition propose that humans use sensorimotor systems in processing language. The Action-Sentence Compatibility Effect (ACE) refers to the finding that motor responses are facilitated after comprehending sentences that imply movement in the same direction. In sign languages there is a potential conflict between sensorimotor systems and linguistic semantics: movement away from the signer is perceived as motion toward the comprehender. We examined whether perceptual processing of sign movement or verb semantics modulate the ACE. Deaf ASL signers performed a semantic judgment task while viewing signed sentences expressing toward or away motion. We found a significant congruency effect relative to the verb’s semantics rather than to the perceived motion. This result indicates that (a) the motor system is involved in the comprehension of a visual–manual language, and (b) motor simulations for sign language are modulated by verb semantics rather than by the perceived visual motion of the hands.

Keywords

Action-Sentence Compatibility Effectembodied cognitionmental simulationdeafsign language
Type
Research Article
Information
Language and Cognition , Volume 7 , Issue 2 , June 2015 , pp. 305 - 318
Copyright
Copyright © UK Cognitive Linguistics Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

This work was supported by a grant from the National Institute on Deafness and Other Communicative Disorders (NIDCD) (RO1 DC010997) awarded to Karen Emmorey and San Diego State University. Kristen Secora was supported by a training grant from NIDCD (T32 DC0007361). We thank the members of the Laboratory for Language and Cognitive Neuroscience at SDSU for their help with the study, Michael Secora for help with the illustrations, and all of the deaf participants who made this research possible.

References

references

Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(04), 577–660.Google Scholar
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.Google Scholar
Bergen, B., & Wheeler, K. (2005). Sentence understanding engages motor processes. Proceedings of the Twenty-Seventh Annual Conference of the Cognitive Science Society, 238–243.Google Scholar
Borghi, A. M., Glenberg, A. M., & Kaschak, M. P. (2004). Putting words in perspective. Memory & Cognition, 32(6), 863–873.Google Scholar
Borreggine, K. L., & Kaschak, M. P. (2006). The Action-Sentence Compatibility Effect: it’s all in the timing. Cognitive Science, 30, 1097–1112.Google Scholar
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565.CrossRefGoogle ScholarPubMed
Kaschak, M. P., & Borreggine, K. L. (2008). Temporal dynamics of the action-sentence compatibility effect. Quarterly Journal of Experimental Psychology, 61(6), 883–895.Google Scholar
Kaschak, M. P., Madden, C. J., Therriault, D. J., Yaxley, R. H., Aveyard, M., Blanchard, A. A., & Zwaan, R. A. (2005). Perception of motion affects language processing. Cognition, 94, B79–B89.Google Scholar
Kaschak, M. P., Zwaan, R. A., Aveyard, M., & Yaxley, R. H. (2006). Perception of auditory motion affects language processing. Cognitive Science, 30, 733–744.CrossRefGoogle ScholarPubMed
Klatzky, R. L., Pellegrino, J. W., McCloskey, B. P., & Doherty, S. (1989). Can you squeeze a tomato? The role of motor representations in semantic sensibility judgments. Journal of Memory and Language, 28(1), 56–77.Google Scholar
McCullough, S., Saygin, A. P., Korpics, F., & Emmorey, K. (2012). Motion-sensitive cortex and motion semantics in American Sign Language. NeuroImage, 63, 111–118.Google Scholar
Meteyard, L., Zokaei, N., Bahrami, B., & Vigliocco, G. (2008). Visual motion interferes with lexical decision on motion words. Current Biology, 18(17), R732–R733.Google Scholar
Perniss, P., Vinson, D., Fox, N., & Vigliocco, G. (2013). Comprehending with the body: action compatibility in sign language?Proceedings of the Thirty-Fifth Annual Conference of the Cognitive Science Society, 1133–1138.Google Scholar
Pyers, J., Perniss, P., & Emmorey, K. (2008). Viewpoint in the visual-spatial modality. Paper presented at the 30th annual convention of the German Society of Linguistics workshop on Gestures: A comparison of signed and spoken languages.Google Scholar
Scorolli, C., & Borghi, A. M. (2007). Sentence comprehension and action: effector specific modulation of the motor system. Brain Research, 1130, 119–124.Google Scholar
Skipper, J. I., Goldin-Meadow, S., Nusbaum, H. C., & Small, S. L. (2007). Speech-associated gestures, Broca’s area, and the human mirror system. Brain and Language, 101(3), 260–277.Google Scholar
Skipper, J. I., Goldin-Meadow, S., Nusbaum, H. C., & Small, S. L. (2009). Gestures orchestrate brain networks for language understanding. Current Biology, 19(8), 661–667.Google Scholar
Tseng, M., & Bergen, B. (2005). Lexical processing drives motor simulation. Proceedings of the Twenty-Seventh Annual Conference of the Cognitive Science Society, 2206–2211.Google Scholar
Yaxley, R. H., & Zwaan, R. A. (2007). Simulating visibility during language comprehension. Cognition, 105, 229–236.Google Scholar
Zwaan, R. A., Madden, C. J., Yaxley, R. H., & Aveyard, M. E. (2004). Moving words: dynamic representations in language comprehension. Cognitive Science, 28, 611–619.Google Scholar
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: motor resonance in language comprehension. Journal of Experimental Psychology, 135(1), 1–11.Google Scholar