Multimodal prosody:
Speech and gesture in interaction

Prosody research has traditionally focused on the acoustic values of pitch, intensity, and duration used to signal grouping, rhythmic, and accentual patterns in speech. Recent advances show that the study of spoken language needs to take into account not only the auditory modality but also the visual modality, as human communication is multimodal. Speakers use body movements such as hand gestures, body posture, facial expressions, or head movements when communicating with others using speech. There is cross-linguistic evidence that body movements complement and even supplement the formal and functional properties of spoken prosody. This relationship is well documented in typical adult speakers, in signed languages (where prosodic properties of utterances are conveyed through body and facial movements), and in typical and atypical language development, where visual and gestural means are often used to cope with an immature speech system.

The emerging evidence that spoken prosody is tightly connected to body movements has favored the development of new and diverse terminology to refer to this phenomenon (visual prosody, audiovisual prosody, multimodal prosody, prosodic gestures, rhythmic gestures). It has also encouraged debate about questions such as what prosody is, which aspects of prosody and gesture are part of the linguistic system and which parts are extra-grammatical, how this linkage is manifested in signed languages, and the extent to which language production and comprehension models need to take into account this linkage. These considerations highlight the need for a simultaneous assessment of prosodic and gestural components of language, e.g. multidimensional labeling methods that incorporate prosodic and visual features. Similarly, several debates involve the degree of temporal interconnectedness between the prosodic and gesture features of language, their semantic and pragmatic integration within the grammar architecture, and their joint role in facilitating language processing and comprehension. There is also discussion of whether these two features serve as precursors and facilitators of language development in typically and atypically developing populations.  

Call for contributions:

We welcome contributions on recent innovative research on multimodal aspects of prosody that can help advance our understanding of these currently-debated issues. Please send an expression of interest to contribute to the special issue accompanied by an initial abstract by 30 April 2022.

Deadlines:

• 30 April 2022: Deadline for authors to send an expression of interest to contribute to the special issue. 
• 30 June 2022: Deadline for authors to send in their abstracts.
• 15 October 2022: Deadline for authors to submit their papers. 
• Spring 2023: Expected publication of the Special Issue.

Information on publication fees:

Please note that there are no fees for non-Open Access publication of manuscripts. As per a special agreement in connection with this special issue, all non-OA manuscripts will be free-to-read for 3 months after the date of publication.

For Open Access publication, the standard fee of £2,045 applies. Authors that are based at institutions with a transitional agreement with CUP will be eligible to have their original research and/or review articles published as Open Access, with their fee being covered by their institution automatically.

The checker tool to confirm eligibility can be found here.

Contacts:

To send in your expressions of interest, and for any further information, please contact the Guest Editors of the special issue:

• Núria Esteve-Gibert - nesteveg@uoc.edu
• Pilar Prieto - pilar.prieto@upf.edu
• Stefanie Shattuck-Hufnagel - sshuf@mit.ed