Books on perception usually concentrate on a single modality, such as vision or hearing, or a subdivision of a modality, for example, color vision or speech perception. Even when an introductory book on perception deals with several modalities, it is generally subdivided into sections with little overlap. The few books treating the senses together as a single topic generally emphasize philosophy or epistemology (but see Gibson, 1966; Marks, 1978).

Yet the senses are not independent. Events in nature are often multidimensional in character and stimulate more than one sensory system. An organism which optimizes its ability to interact appropriately with the environment is one that integrates relevant information across sensory systems. In a book dealing largely with single neurons that respond to more than one modality, Stein and Meredith (1993) stated that “we know of no animal with a nervous system in which the different sensory representations are organized so that they maintain exclusivity from one another.”

Multimodal perception

Interaction of vision with senses other than hearing

Depth perception in vision is based on a number of cues, including disparity of the images at the two retinae, and motion parallax. But, in addition to these cues transmitted by the optic nerve, there are proprioceptive ocular cues from the muscles producing accommodation (changes in the curvature of the lens necessary to produce a sharp image) and convergence (adjustment of the ocular axes so that the fixated object is imaged on corresponding points of each fovea).