Neural processing of asynchronous audiovisual speech perception

Ryan A. Stevenson, Nicholas A. Altieri, Sunah Kim, David B. Pisoni, Thomas W. James

Research output: Contribution to journalArticle

84 Scopus citations

Abstract

The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.

Original languageEnglish (US)
Pages (from-to)3308-3318
Number of pages11
JournalNeuroImage
Volume49
Issue number4
DOIs
StatePublished - Feb 15 2010

Keywords

  • fMRI
  • Integration
  • Multisensory
  • STS
  • Superior colliculus
  • Synchrony

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Fingerprint Dive into the research topics of 'Neural processing of asynchronous audiovisual speech perception'. Together they form a unique fingerprint.

  • Cite this