Hostname: page-component-788cddb947-55tpx Total loading time: 0 Render date: 2024-10-12T07:56:51.177Z Has data issue: false hasContentIssue false

On the semiotic and material constraints of ideographies

Published online by Cambridge University Press:  02 October 2023

Izzy Wisher
Affiliation:
Department of Linguistics, Cognitive Science and Semiotics, Aarhus University, Aarhus, Denmark kristian@cc.au.dk https://pure.au.dk/portal/en/persons/kristian-tylen(9950d8bc-e1cd-400a-a547-359cd0b07157).html izzywisher@cc.au.dk https://pure.au.dk/portal/en/persons/isobel-wisher(d0e2fb0e-cb67-4ff2-9786-3c9dff72d291).html
Kristian Tylén
Affiliation:
Department of Linguistics, Cognitive Science and Semiotics, Aarhus University, Aarhus, Denmark kristian@cc.au.dk https://pure.au.dk/portal/en/persons/kristian-tylen(9950d8bc-e1cd-400a-a547-359cd0b07157).html izzywisher@cc.au.dk https://pure.au.dk/portal/en/persons/isobel-wisher(d0e2fb0e-cb67-4ff2-9786-3c9dff72d291).html

Abstract

Despite obvious advantages, no generalised ideographic codes have evolved through cultural evolution to rely on iconicity. Morin suggests that this is because of missing means of standardisation, which glottographic codes get from natural languages. Although we agree, we also point to the important role of the available media, which might support some forms of reference more effectively than others.

Type
Open Peer Commentary
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Recently, there has been renewed interest in the role of iconicity in human communication (Fay, Ellison, & Garrod, Reference Fay, Ellison and Garrod2014; Perlman, Dale, & Lupyan, Reference Perlman, Dale and Lupyan2015). Although arbitrariness has historically been considered a central design feature of language (Hockett, Reference Hockett1960), sound symbolism appears more prevalent than first acknowledged (Dingemanse, Blasi, Lupyan, Christiansen, & Monaghan, Reference Dingemanse, Blasi, Lupyan, Christiansen and Monaghan2015; Monaghan, Shillcock, Christiansen, & Kirby, Reference Monaghan, Shillcock, Christiansen and Kirby2014). Similarly, new studies on the possible evolutionary roots of language (whether spoken, signed, or written) suggest a stage of iconically grounded reference in, for instance, pantomime, vocalisations, or figurative depiction (Garrod, Fay, Lee, Oberlander, & MacLeod, Reference Garrod, Fay, Lee, Oberlander and MacLeod2007; Nölle, Staib, Fusaroli, & Tylén, Reference Nölle, Staib, Fusaroli and Tylén2028; Perlman & Lupyan, Reference Perlman and Lupyan2018; Zlatev, Żywiczyński, & Wacewicz, Reference Zlatev, Żywiczyński and Wacewicz2020).

Indeed, preceding any forms of conventionalised writing, our Palaeolithic ancestors used iconic depictions in parietal and portable art to denote animals, humans, and perhaps even narrative scenes (Aubert et al., Reference Aubert, Lebe, Oktaviana, Tang, Burhan, Hamrullah and Brumm2019). From an ontogenetic perspective, iconicity facilitates early-visual communication as children typically learn to draw before they write, and pictorial narratives appear intuitively accessible even to preschool children (Jolley, Reference Jolley2009). Given these observations, it is puzzling that no generalised ideographic systems have evolved to be fully dependent on iconicity.

Morin suggests that the reason these apparently fundamental forms of human signification never evolve to become generalised ideographies is because of the lack of procedures for standardisation. We agree that any generalised form of communication will depend on the continuous contextualised negotiation of meaning (Dideriksen, Christiansen, Tylén, Dingemanse, & Fusaroli, Reference Dideriksen, Christiansen, Tylén, Dingemanse and Fusaroli2022). However, we also suggest that there might be semiotic factors intrinsic to the materiality of certain media that challenge the evolution of ideographies – in particular, to the extent they depend on iconicity. Heraldic signs, coins, or commodity brands work well in their specialised systems for marking identities (families, cities, values, or products), but they have no means of representing the course of multiple events; they cannot tell a story.

In written language, meaning is built-up as new words are added in a continuous linear succession. Any new detail or event manifests as a spatial prolongation of the text. In an iconic depiction, the addition of new detail will instead correspond to a transformation of the depiction itself (Lotman, Reference Lotman1975). Thus, a purely iconicity-based code faces the choice of three possibilities: (1) Presenting a single-static depiction of a salient moment from unfolding events to support the reader's inferences about immediately preceding and succeeding events; (2) using the linearisation principle from glottography to present a series of depictions representing a succession of events (like a comics strip); or (3) if the medium allows, animating the image to create a continuous transformation of the icon (like a cartoon).

The first option is what we appear to find already within examples of early or contemporary rock art. However, these depictions are often limited to a single event (e.g., a hunting scene; Aubert et al., Reference Aubert, Lebe, Oktaviana, Tang, Burhan, Hamrullah and Brumm2019) or require a preexisting knowledge of the narrative being communicated. In both Australian Aboriginal and San rock art, depicted scenes represent complex narratives related to how the world was created (e.g., the Dreamtime; McDonald, Reference McDonald2013; Tacon, Reference Tacon1989) or spiritual encounters and trance states. However, without an understanding of associated mythologies or certain cultural behaviours, the art cannot be decoded. This is highlighted by the misinterpretation of San rock art by Western ethnographers, where poor translations of oral traditions have confused concepts embedded in the art (Challis, Hollmann, & McGranaghan, Reference Challis, Hollmann and McGranaghan2013; McGranaghan & Challis, Reference McGranaghan and Challis2016). Thus, the potential for static depictions to serve as ideographies is limited, despite icons sometimes having standardised elements (e.g., X-ray depictions in Australian art represent living animals, solid-filled depictions represent dead animals; Tacon, Reference Tacon1989). Rather, these iconic depictions can be considered mnemonic devices that do not communicate independently.

The second option, the comics principle, has numerous historical instantiations (e.g., the famous Bayeux Tapestry; Brilliant, Reference Brilliant1991). Although it overcomes some of the outlined limitations with respect to representing complex sequences of events and has established itself as a rich genre of graphic storytelling (Cohn, Reference Cohn2013; Stjernfelt & Østergaard, Reference Stjernfelt, Østergaard, Pedri and Petit2013), it has not evolved into conventionalised ideographic codes. Among the reasons, as discussed by Morin, is probably that it resists the compression and standardisation needed to become an economic medium of communication. For instance, comics do not come with a functional system of anaphora by which a character or object mentioned in an earlier scene can be referred to with a shorthand “she” or “it.” Rather, reappearing characters are redrawn in new configurations in every panel. Bliss symbols could be considered a solution, combining schematic icons with some arbitrary elements, grammatical categories, and the linearisation principle from verbal language (Nawar, Reference Nawar2012). However, as a kind of “creole,” it depends on language-like conventions for composing meaning, bringing back issues of standardisation and the need for an oral gloss.

The third option – the animated cartoon – may be the solution most true to the inherent semiotic nature of iconicity (not piggybacking on discretisation and linearisation principles from language; Lotman, Reference Lotman1975). A hypothetical cartoon language would have several advantages for effective communication. On the receptive side, it would be highly intuitive and accessible, thus overcoming language barriers and possibly requiring minimal formal training (Berney & Bétrancourt, Reference Berney and Bétrancourt2016). Although such an ideographic language has not (yet) evolved, it is probably not because of cognitive factors pertaining, for instance, to the architecture of our visual system or working memory. It is also not because of social factors alone, because such a communication system would potentially require fewer conventions to standardise. The main reason is probably instead the lack of a suitable medium that would support such a code, which makes it an unfeasible solution for general communication. If the effortless production, transmission, and reception of cartoons could be effectively supported by available material media, this could hypothetically constitute Morin's missing ideographic code. The historical success of glottographic codes is thus not only a matter of the way it relies on standardisation processes from natural language, but also how it is supported by the available technological solutions and material media that historically has included clay tablets, pen, paper, and print, and is currently evolving with digital media.

Financial support

The authors are supported by an EU ERC consolidator grant (eSYMb, grant number: 101044626), awarded to Kristian Tylén.

Competing interest

None.

References

Aubert, M., Lebe, R., Oktaviana, A. A., Tang, M., Burhan, B., Hamrullah, , … Brumm, A. (2019). Earliest hunting scene in prehistoric art. Nature, 576, 442445.CrossRefGoogle ScholarPubMed
Berney, S., & Bétrancourt, M. (2016). Does animation enhance learning? A meta-analysis. Computers & Education, 101, 150167.CrossRefGoogle Scholar
Brilliant, R. (1991). The Bayeux Tapestry: A stripped narrative for their eyes and ears. Word & Image, 7(2), 98126.CrossRefGoogle Scholar
Challis, S., Hollmann, J., & McGranaghan, M. (2013). “Rain snakes” from the Senqu River: New light on Qing's commentary on San rock art from Sehonghong, Lesotho. Azania: Archaeologial Research in Africa, 48(3), 331354.CrossRefGoogle Scholar
Cohn, N. (2013). The visual language of comics: Introduction to the structure and cognition of sequential images. A&C Black.Google Scholar
Dideriksen, C., Christiansen, M. H., Tylén, K., Dingemanse, M., & Fusaroli, R. (2022). Quantifying the interplay of conversational devices in building mutual understanding. Journal of Experimental Psychology: General, 152(3), 864889.CrossRefGoogle ScholarPubMed
Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity, and systematicity in language. Trends in Cognitive Sciences, 19(10), 603615.CrossRefGoogle ScholarPubMed
Fay, N., Ellison, M., & Garrod, S. (2014). Iconicity: From sign to system in human communication and language. Pragmatics & Cognition, 22(2), 244263.CrossRefGoogle Scholar
Garrod, S., Fay, N., Lee, J., Oberlander, J., & MacLeod, T. (2007). Foundations of representation: Where might graphical symbol systems come from? Cognitive Science, 31(6), 961987.CrossRefGoogle ScholarPubMed
Hockett, C. F. (1960). The origin of speech. Scientific American, 203(3), 8897.CrossRefGoogle ScholarPubMed
Jolley, R. P. (2009). Children and pictures: Drawing and understanding. John Wiley.Google Scholar
Lotman, J. M. (1975). The discrete text and the iconic text: Remarks on the structure of narrative (F. Pfotenhauer, Trans.). New Literary History, 6(2), 333338.CrossRefGoogle Scholar
McDonald, J. (2013). Contemporary meanings and the recursive nature of rock art: Dilemmas for a purely archaeological understanding of rock art. Time and Mind: The Journal of Archaeology, Consciousness and Culture, 6(1), 6572.CrossRefGoogle Scholar
McGranaghan, M., & Challis, S. (2016). Reconfiguring hunting magic: Southern Bushman (San) perspectives on taming and their implications for understanding rock art. Cambridge Archaeological Journal, 26(4), 579599.CrossRefGoogle Scholar
Monaghan, P., Shillcock, R. C., Christiansen, M. H., & Kirby, S. (2014). How arbitrary is language? Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1651), 20130299.CrossRefGoogle ScholarPubMed
Nawar, H. (2012). Multicultural transposition: From alphabets to pictographs, towards semantographic communication. Technoetic Arts, 10(1), 5968.CrossRefGoogle Scholar
Nölle, J., Staib, M., Fusaroli, R., & Tylén, K. (2018). The emergence of systematicity: How environmental and communicative factors shape a novel communication system. Cognition, 181, 93104.CrossRefGoogle ScholarPubMed
Perlman, M., Dale, R., & Lupyan, G. (2015). Iconicity can ground the creation of vocal symbols. Royal Society Open Science, 2(8), 150152.CrossRefGoogle ScholarPubMed
Perlman, M., & Lupyan, G. (2018). People can create iconic vocalizations to communicate various meanings to naïve listeners. Scientific Reports, 8(1), 114.CrossRefGoogle ScholarPubMed
Stjernfelt, F., & Østergaard, S. (2013). FONK! HONK! WHAM! OOF!: Representation of events in Carl Barks – and in the aesthetics of comics in general. In Pedri, N. & Petit, L. (Eds.), Picturing the language of images (pp. 483508). Cambridge Scholars Press.Google Scholar
Tacon, P. S. C. (1989). From the “dreamtime” to the present: The changing role of Aboriginal rock paintings in Western Arnhem Land, Australia. The Canadian Journal of Native Studies, IX(2), 317339.Google Scholar
Zlatev, J., Żywiczyński, P., & Wacewicz, S. (2020). Pantomime as the original human-specific communicative system. Journal of Language Evolution, 5(2), 156174.CrossRefGoogle Scholar