This is the first collective work to emerge from the Multimodal Interaction Project, within the Language and Cognition Group at the Max Planck Institute, Nijmegen. In this project, we are concerned with describing the formal features of human social interaction, and characterizing their underlying principles. Language is of course at the heart of it, but our guiding position is that face-to-face interaction provides the infrastructure for language in all its facets: production, comprehension, acquisition and structuration. In practice, language is woven into the full visible and interactional setting. Through a range of disciplinary approaches the project asks what makes it possible for human beings to be able to navigate their exceedingly complex social worlds with such aplomb. The topic of ‘person reference’ provides an ideal case study: the simple act of referring to someone takes us straight to the core of multimodal interaction, to the mechanics of conversation, and to a set of fundamental issues in linguistics, sociology and social anthropology.

The chapters of this book take a broadly semiotic approach to the problem of social action. The key skill is people's ability to recognize and understand others' actions through their public behavior. One source of guidance for any social participant is the stock of cultural norms: a culture, as Sacks put it, is ‘an apparatus for generating recognizable action’. This suggests variation between human groups, as supported to some degree by this book's findings.