Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-26T20:57:54.448Z Has data issue: false hasContentIssue false
  • English
  • Français

The Limitations of Representing Sound and Notation on Screen

Published online by Cambridge University Press:  13 November 2014

Lindsay Vickery
Lindsay Vickery*
Affiliation:
School of Music Western Australian Academy of Performing Arts, 2 Bradford Street, Mount Lawley, Western Australia
*
Email: l.vickery@ecu.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

Animated screen-based notation and visual representation of sound provide an important solution to visualising a range of musical phenomena and techniques including continuous parametrical changes, synchronisation with prerecorded audio or live processing, and nonlinear formal organisation. The limitations of human visual capabilities, however, place some constraints upon the efficacy of screen-based representation, particularly in regard to notation reading. Findings from sightreading studies exploring the manner in which notation is encoded, processed and executed are examined with the aim of identifying the perceptual and practical boundaries of presenting animated notation on screen. The development of efficient visual representation is proposed as an important requirement for alleviating the issues created by the time constraints of reading on screen. Studies in semantics and cross-modal activation are discussed as a foundation for the expansion of approaches to the visualisation of sound.

Type
Articles
Information
Organised Sound , Volume 19 , Special Issue 3: Mediation: Notation and Communication in Electroacoustic Music Performance , December 2014 , pp. 215 - 227
Copyright
© Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adkins, M. 2008. The Application of Memetic Analysis to Electroacoustic Music. www.mathewadkins.co.uk/article4.Google Scholar
Blackburn, M. 2009. Composing from Spectromorphological Vocabulary: Proposed Application, Pedagogy and Metadata. www.ems-network.org/ems09/papers/blackburn.pdf.Google Scholar
Blackburn, M. 2011. The Visual Sound-Shapes of Spectromorphology: An Illustrative Guide to Composition. Organised Sound 16(1): 5–13.CrossRefGoogle Scholar
Cardew, C. 1981. Wiggly Lines and Wobbly Music. Breaking the Sound Barrier: A Critical Anthology of the New Music. New York: Dutton. 235–253.Google Scholar
Dannenberg, R. B. 1996. Extending Music Notation Through Programming. Computer Music in Context 13(2): 63–76.Google Scholar
Deroy, O. and Spence, C. 2013. Weakening the Case for ‘Weak Synaesthesia’: Why Crossmodal Correspondences are not Synaesthetic. Psychonomic Bulletin & Review 20: 643–664.CrossRefGoogle Scholar
Dewar, J. 1998. The Information Age and the Printing Press: Looking Backward to See Ahead. Santa Monica, CA: RAND.CrossRefGoogle Scholar
Eitan, Z. and Timmers, R. 2010. Beethoven’s Last Piano Sonata and Those Who Follow Crocodiles: Cross-Domain Mappings of Auditory Pitch in a Musical Context. Cognition 114: 405–422.CrossRefGoogle Scholar
Freeman, J. 2008. Extreme Sight-Reading, Mediated Expression, and Audience Participation: Real-Time Music Notation in Live Performance. Computer Music Journal 32(3): 25–41.CrossRefGoogle Scholar
George, S. E. 2004. Visual Perception of Music Notation: On-line and Off-line Recognition. Hersey, PA: IRM Press.Google Scholar
Giannakis, K. 2006. A Comparative Evaluation of Auditory-Visual Mappings for Sound Visualization. Organised Sound 11(3): 297–307.Google Scholar
Gilman, E. and Underwood, G. 2003. Restricting the Field of View to Investigate the Perceptual Spans of Pianists. Visual Cognition 10(2): 201–232.Google Scholar
Goolsby, T. 1994a. Eye-Movement in Music Reading: Effects of Reading Ability, Notational Complexity, and Encounters. Music Perception 12: 77–96.Google Scholar
Goolsby, T. 1994b. Profiles of Processing: Eye-Movements during Sight-Reading. Music Perception 12: 97–123.Google Scholar
Greco, M., Stucchi, N., Zavagno, D. and Marino, B. 2008. On the Portability of Computer-Generated Presentations: The Effect of Text-Background Color Combinations on Text Legibility. Human Factors. The Journal of the Human Factors and Ergonomics Society 50: 821.CrossRefGoogle Scholar
Green-Armytage, P. 2010. A Colour Alphabet and the Limits of Colour Coding. Colour: Design & Creativity 510: 1–23.Google Scholar
Grill, T. and Flexer, A. 2012. Visualization of Perceptual Qualities in Textural Sounds. International Computer Music Conference 2012, Ljubljana, Slovenia.Google Scholar
Griscom, W. S. and Palmer, S. E. 2012. The Color of Musical Sounds in Non-Synesthetes. 12th Annual Meeting of the Vision Science Society, Naples, FL.CrossRefGoogle Scholar
Griscom, W. S. and Palmer, S. E. 2013. Cross-Modal Sound-to-Sight Associations with Musical Timbre in Non-Synesthetes. 13th Annual Meeting of the Vision Science Society, 10–15 May, Naples, FL.Google Scholar
Gunter, T. C., Schmidt, B.-H. et al. 2003. Let’s Face the Music: A Behavioral and Electrophysiological Exploration of Score Reading. Psychophysiology 40: 742–751.CrossRefGoogle Scholar
Hachman, M. 2010. Analyst Challenges Apple’s iPhone 4 ‘Retina Display’ Claims. PCMAG , www.pcmag.com/article2/0,2817,2364871,00.asp.Google Scholar
Hajdu, G. and Didkovsky, N. 2009. On the Evolution of Music Notation in Network Music Environments. Contemporary Music Review 28(4): 395–407.Google Scholar
Honing, H. 2002. Structure and Interpretation of Rhythm and Timing. Dutch Journal of Music Theory (Tijdschrift voor Muziektheorie) 7(3): 227–232.Google Scholar
Horn, R. E. 1998. Visual Language: Global Communication for the 21st Century. Bainbridge Island, WA: MacroVU.Google Scholar
Jones, B. 2012. Apple Retina Display (25 August). http://prometheus.med.utah.edu/~bwjones/2010/06/apple-retina-display.Google Scholar
Karkoschka, E. 1972. Notation in New Music: A Critical Guide to Interpretation and Realisation. New York: Praeger.Google Scholar
Kim-Boyle, D. 2010. Real-Time Score Generation for Extensible Open Forms. Contemporary Music Review 29(1): 3–15.Google Scholar
Kinsler, V. and Carpenter, R. H. S. 1995. Saccadic Eye Movements while Reading Music. Vision Research 35(10): 1447–1458.Google Scholar
Köhler, W. 1929. Gestalt Psychology. New York: Liveright.Google Scholar
Kojs, J. 2011. Notating Action-Based Music. Leonardo Music Journal 21: 65–72.Google Scholar
Lee, J. I. 2003. The Role of Working Memory and Short-Term Memory in Sight Reading. The 5th Triennial ESCOM Conference, Hanover University of Music And Drama, Germany.Google Scholar
Locher, E. J. and Nodine, C. G. 1974. The Role of Scanpaths in the Recognition of Random Shapes. Perception and Psychophysics 15: 308–314.CrossRefGoogle Scholar
Marks, L. E. 1996. On Perceptual Metaphors. Metaphor & Symbolic Activity 11: 39–66.CrossRefGoogle Scholar
Marks, L. E. and Odgar, E. C. 2005. Developmental Constraints on Theories of Synesthesia. Synesthesia: Perspectives from Cognitive Neuroscience. New York: Oxford University Press, 214–36.Google Scholar
Martino, G. and Marks, L. E. 2001. Synesthesia: Strong and Weak. Current Directions in Psychological Science 10: 61–65.Google Scholar
McClelland, C. and Alcorn, M. 2008. Exploring New Composer/Performer Interactions Using Real-Time Notation. International Computer Music Conference ʼ08. Belfast, Northern Ireland.Google Scholar
Medley, S. and Haddad, H. 2011. The Realism Continuum: Representation and Perception. The International Journal of the Image 1(2): 145–156.CrossRefGoogle Scholar
Mulder, A. 1994. Human Movement Tracking Technology. Hand Centered Studies of Human Movement Project, Technical Report 94-1. Burnaby, BC, Simon Fraser University.Google Scholar
Pasoulas, A. 2011. Temporal Associations, Semantic Content and Source Bonding. Organised Sound 16(1): 63–68.Google Scholar
Patel, R., Schooley, K. and Wilner, J. 2007. Visual Features that Convey Meaning in Graphic Symbols: A Comparison of PCS and Artists’ Depictions. Assistive Technology Outcomes and Benefits 41: 62–80.Google Scholar
Patton, K. 2007. Morphological Notation for Interactive Electroacoustic Music. Organised Sound 12: 123–128.CrossRefGoogle Scholar
Picking, R 1997. Reading Music from Screens vs Paper. Behaviour and Information technology 162: 72–78.Google Scholar
Pluth, E. 2012. Signifiers and Acts: Freedom in Lacan’s Theory of the Subject. New York: SUNY Press.Google Scholar
Prado-Leon, L. R., Schloss, K. B. and Palmer, S. E. 2011. Color, Music, and Emotion in Mexican and US Populations. New Directions in Colour Studies. Amsterdam: John Benjamins.Google Scholar
Ramachandran, V. S. and Hubbard, E. M. 2001. Synaesthesia: A Window into Perception, Thought and Language. Journal of Consciousness Studies 8(12): 3–34.Google Scholar
Rayner, K., Well, A.D., Pollatsek, A. and Bertera, J. H. 1982. The Availability of Useful Information to the Right of Fixation in Reading. Perception and Psychophysics 31: 537–550.Google Scholar
Sternberg, Robert J. 1996. Cognitive Psychology. Fort Worth, TX: Harcourt Brace College Publisher.Google Scholar
Sudarshan, B. and Wyse, L. 2012. Computer Mediated Visual Communication in Live Musical Performance: What’s the Score? In Arts and Technology. Berlin and Heidelberg: Springer. 54–62.Google Scholar
Tanzi, D. 2011. Extra-Musical Meanings and Spectromorphology. Organised Sound 16(1): 36–41.Google Scholar
Thoreson, L. 2007. Spectromorphological Analysis of Sound Objects: An Adaptation of Pierre Schaeffer’s Typomorphology. Organised Sound 12(2): 129–141.CrossRefGoogle Scholar
Thoreson, L. 2010. Form-Building Patterns and Metaphorical Meaning. Organised Sound 15(2): 82–95.CrossRefGoogle Scholar
Truitt, F. E., Clifton, C., Pollatsek, A. and abd Rayner, K. 1997. The Perceptual Span and Eye-Hand Span in Sight Reading Music. Visual Cognition 4: 143–161.Google Scholar
Vickery, L. 2010. Mobile Scores and Click-Tracks: Teaching Old Dogs. Proceedings of the Australasian Computer Music Conference 2010. Canberra, 63–70.Google Scholar
Vickery, L. 2011. The Possibilities of Novel Formal Structures through Computer Controlled Live Performance. Australasian Computer Music Association. Auckland, New Zealand, 112–17.Google Scholar
Vickery, L. 2012. The Evolution of Notational Innovations from the Mobile Score to the Screen Score. Organised Sound 17(4): 128–136.Google Scholar
Vinot, J.-L. and Athènes, S. 2012. Legible, Are You Sure? An Experimentation-Based Typographical Design in Safety-Critical Context. CHI 2012. Austin, TX.Google Scholar
Walker, P. 2012. Cross-Sensory Correspondences and Cross Talk Between Dimensions of Connotative Meaning: Visual Angularity is Hard, High-Pitched, and Bright. Attention Perception and Psychophysics 74: 1792–1809.Google Scholar
Waters, A. J., Underwood, G. and Findlay, J. M. 1997. Studying Expertise in Music Reading: Use of a Pattern-Matching Paradigm. Perception and Psychophysics 594: 477–488.Google Scholar
Waters, A. J., Townsend, E. and Underwood, G. 1998. Expertise in Musical Sight-Reading: A Study of Pianists. British Journal of Psychology 89: 123–149.Google Scholar
Wierzbicka, A. 1996. Semantics: Primes and Universals. New York: Oxford University Press.Google Scholar
Winkler, Gerhard E. 2004. The Real Time-Score. The Missing-Link in Computer-Music Performance. Sound and Music Computing ʼ04. IRCAM.Google Scholar