Neural processing of asynchronous audiovisual speech perception

Ryan A. Stevenson, Nicholas A. Altieri, Sunah Kim, David Pisoni, Thomas W. James

Research output: Contribution to journalArticle

81 Citations (Scopus)

Abstract

The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.

Original languageEnglish
Pages (from-to)3308-3318
Number of pages11
JournalNeuroImage
Volume49
Issue number4
DOIs
StatePublished - Feb 15 2010

Fingerprint

Speech Perception
Temporal Lobe
Visual Cortex
Occipital Lobe
Superior Colliculi
Individuality
Magnetic Resonance Imaging

Keywords

  • fMRI
  • Integration
  • Multisensory
  • STS
  • Superior colliculus
  • Synchrony

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this

Neural processing of asynchronous audiovisual speech perception. / Stevenson, Ryan A.; Altieri, Nicholas A.; Kim, Sunah; Pisoni, David; James, Thomas W.

In: NeuroImage, Vol. 49, No. 4, 15.02.2010, p. 3308-3318.

Research output: Contribution to journalArticle

Stevenson, RA, Altieri, NA, Kim, S, Pisoni, D & James, TW 2010, 'Neural processing of asynchronous audiovisual speech perception', NeuroImage, vol. 49, no. 4, pp. 3308-3318. https://doi.org/10.1016/j.neuroimage.2009.12.001
Stevenson, Ryan A. ; Altieri, Nicholas A. ; Kim, Sunah ; Pisoni, David ; James, Thomas W. / Neural processing of asynchronous audiovisual speech perception. In: NeuroImage. 2010 ; Vol. 49, No. 4. pp. 3308-3318.
@article{c44280fb7f064faab9b6710eda6e2415,
title = "Neural processing of asynchronous audiovisual speech perception",
abstract = "The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.",
keywords = "fMRI, Integration, Multisensory, STS, Superior colliculus, Synchrony",
author = "Stevenson, {Ryan A.} and Altieri, {Nicholas A.} and Sunah Kim and David Pisoni and James, {Thomas W.}",
year = "2010",
month = "2",
day = "15",
doi = "10.1016/j.neuroimage.2009.12.001",
language = "English",
volume = "49",
pages = "3308--3318",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",
number = "4",

}

TY - JOUR

T1 - Neural processing of asynchronous audiovisual speech perception

AU - Stevenson, Ryan A.

AU - Altieri, Nicholas A.

AU - Kim, Sunah

AU - Pisoni, David

AU - James, Thomas W.

PY - 2010/2/15

Y1 - 2010/2/15

N2 - The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.

AB - The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.

KW - fMRI

KW - Integration

KW - Multisensory

KW - STS

KW - Superior colliculus

KW - Synchrony

UR - http://www.scopus.com/inward/record.url?scp=73749083278&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=73749083278&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2009.12.001

DO - 10.1016/j.neuroimage.2009.12.001

M3 - Article

C2 - 20004723

AN - SCOPUS:73749083278

VL - 49

SP - 3308

EP - 3318

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

IS - 4

ER -