Audiovisual Integration

People experience the world through multisensory integration, or the integration in the brain of information from various senses. Perception of speech and language is likewise a multisensory process that involves the integration of visual information from a talker’s face and body with [Page 203]acoustic aspects of the speech signal. When speaking to another person, an individual sees the sequence of lip, tongue, and jaw movements that are synchronous with the speech sounds they produce. Thus, people experience speech perception as a combination of hearing (auditory) and seeing (visual) the talker’s message, by the integration of both modalities. This is referred to as audiovisual integration. This entry discusses research on audiovisual integration, the McGurk effect, audiovisual integration and development, and audiovisual integration and communication disorders.