VOLUME 1, ISSUE 3 PP 381–390
REPRINTS AVAILABLE DIRECTLY FROM THE PUBLISHERS.
PHOTOCOPYING PERMITTED BY LICENSE ONLY
© BERG 2006 PRINTED IN THE UK
Cross-talk between the Senses
The Handbook of Multisensory Processes, edited by Gemma Calvert, Charles Spence and Barry E. Stein, Cambridge, MA: The MIT Press, 2004, 915 pages. HB 0-262-03321-6. $125.00.Visual Music: Synaesthesia in Art and Music Since 1900, with contributions by Kerry Brougher, Olivia Mattis, Jeremy Strick, Ari Wiseman and Judith Zilzcer, London: Thames & Hudson, 2005, 272 pages, with 376 illustrations, 344 in color. HB 0-500-51217-5. £32.00.
It is commonly assumed that each sense has its proper sphere (e.g. sight is concerned with color, hearing with sound and taste with ﬂavor).This modular conception of the sensorium is reflected in the analytic orientation of most current research in the psychology of perception with its “sense-by-sense” – or, one sensory modality at a time – approach to the study of perceptual processes. In recent years, however, a more
David Howes is the author of Sensual Relations: Engaging the Senses in Culture and Social Theory (2003). He isProfessor of Anthropology at Concordia University, Montreal, and Director of the Concordia Sensoria Research Team (http://alcor. concordia.ca/~senses), as well as a contributing editor to The Senses and Society. firstname.lastname@example.org
Senses & Society
interactive, relational approach to the understanding of how the senses function has begun to take shape as aresult of the growing body of evidence that points to the “multisensory organization” or “integration” of the brain. As Calvert, Spence and Stein write in their introduction to The Handbook of Multisensory Processes, even those experiences that at ﬁrst may appear to be modalityspeciﬁc are most likely to have been inﬂuenced by activity in other sensory modalities, despite our lack of awareness ofsuch interactions . . . [To] fully appreciate the processes underlying much of sensory perception, we must understand not only how information from each sensory modality is transduced and decoded along the pathways primarily devoted to that sense, but also how this information is modulated by what is going on in the other sensory pathways. (2004: xi-xii) Examples of such modulation include thewell-documented fact that, in noisy surroundings, speakers can be understood more easily if they can be seen as well as heard. This ﬁnding is readily explicable in terms of the redundancy hypothesis of classic information theory. However, the new multisensory psychology of perception probes deeper to explore the relationships among the component parts of a multisensory signal. For example, in the caseof animal and human communication, redundant multisensory signals can be subclassiﬁed into those that produce responses in the receiver equivalent to the response to each unisensory component (equivalence) and those for which the response is superadditive – that is, which exceeds the response to the unisensory components (enhancement). Multisensory signals may also be made up of stimuli whichconvey different (i.e. nonredundant) information, as in the case of the McGurk effect, where seen lip-movements can alter which phoneme is heard for a particular sound (e.g., a sound of /ba/ tends to be perceived as /da/ when it is coupled with a visual lip movement associated with /ga/). In this instance, the response to the multisensory signal is new, qualitatively different from the response toeither of the unisensory components, and thus demonstrates emergence. The relationship between the components of a multisensory signal may otherwise be one of dominance as in the ventriloquism effect (where the seen lip-movements of the dummy alter or “capture” the apparent location of the speech sounds) or concatenation (my term) as in the case of the reproductive behavior of male oriental fruit...