5.6.1. In Action (2)
For enhancement to happen, it has to be labeled as belonging to the same event, and this is primarily done by the information arriving simultaneously. Individual neurons [Hack #9] are already set up to respond to timing information and frequently respond strongest to inputs from different sources arriving simultaneously. If information arrives at different times, it can suppress the activity of cells responsible for responding to inputs across senses (senses are called modalities, in the jargon).
So, what makes information from two modalities appear simultaneous? Obviously arriving at the exact same time is not possible; there must be a resolution of the senses in time below which two events appear to be simultaneous.
Although light moves a million times faster than sound, sound is processed faster once it gets to the ear [Hack #44] than light is processed once it gets to the eye. The relative speed of processing of each sense, coupled with the speed at which light and sound travel, leads to a “horizon of simultaneity”4 at about 10 meterswhere visual and auditory signals from the same source reach the cortex at the same time.
Most events don’t occur just on this 10-meter line, of course, so there must be some extra mechanisms at work in the brain to allow sound and light events to appear simultaneous. Previously, researchers had assumed that the calculation of simultaneity was approximate enough that time difference due to arrival time could be ignored (until you get to events very far awaylike lightning that arrives before thunder, for example). But now it appears that our brains make a preconscious adjustment for how far away something is when calculating whether the sound and the light are arriving at the same time.5 Another mechanism that operates is simply to override the timing information that comes from vision with the timing information from auditory information [Hack #53] .
5.6.3. End Notes
To start following up the research on crossmodal interactions, you could start by reading Crossmodal Space and Crossmodal Attention by Charles Spence and Jon Driver. This is an edited book with contributions from many of the people at the forefront of the field. You can read more about the Oxford University crossmodal research group on its home page: http://www.psych.ox.ac.uk/xmodal/default.htm.
Gillmeister, H., & Eimer, M. (submitted). Multisensory integration in perception: tactile enhancement of perceived loudness.
Macaluso, E., Frith, C. D., & Driver, J. (2000). Modulation of human visual cortex by crossmodal spatial attention. Science, 289, 1206-1208.
Pöppel, E. (1988). Mindworks: Time and Conscious Experience. New York: Harcourt Brace Jovanovich.
Sugita, Y., and Suzuki, Y. (2003). Audiovisual perception: Implicit estimation of sound-arrival time. Nature, 421, 911.
Taken from : Mind Hacks
