Language identification from visual-only speech signals

Rebecca E. Ronquest, Susannah V. Levi, David Pisoni

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Our goal in the present study was to examine how observers identify English and Spanish from visual-only displays of speech. First, we replicated the recent findings of Soto-Faraco et al. (2007) with Spanish and English bilingual and monolingual observers using different languages and a different experimental paradigm (identification). We found that prior linguistic experience affected response bias but not sensitivity (Experiment 1). In two additional experiments, we investigated the visual cues that observers use to complete the languageidentification task. The results of Experiment 2 indicate that some lexical information is available in the visual signal but that it is limited. Acoustic analyses confirmed that our Spanish and English stimuli differed acoustically with respect to linguistic rhythmic categories. In Experiment 3, we tested whether this rhythmic difference could be used by observers to identify the language when the visual stimuli is temporally reversed, thereby eliminating lexical information but retaining rhythmic differences. The participants performed above chance even in the backward condition, suggesting that the rhythmic differences between the two languages may aid language identification in visual-only speech signals. The results of Experiments 3A and 3B also confirm previous findings that increased stimulus length facilitates language identification. Taken together, the results of these three experiments replicate earlier findings and also show that prior linguistic experience, lexical information, rhythmic structure, and utterance length influence visual-only language identification.

Original languageEnglish
Pages (from-to)1601-1613
Number of pages13
JournalAttention, Perception, and Psychophysics
Volume72
Issue number6
DOIs
StatePublished - Aug 2010

Fingerprint

Language
experiment
Linguistics
language
stimulus
linguistics
available information
Acoustics
acoustics
Cues
Identification (Psychology)
Experiment
experience
paradigm
Observer
trend
Length
Stimulus

ASJC Scopus subject areas

  • Sensory Systems
  • Experimental and Cognitive Psychology

Cite this

Language identification from visual-only speech signals. / Ronquest, Rebecca E.; Levi, Susannah V.; Pisoni, David.

In: Attention, Perception, and Psychophysics, Vol. 72, No. 6, 08.2010, p. 1601-1613.

Research output: Contribution to journalArticle

Ronquest, Rebecca E. ; Levi, Susannah V. ; Pisoni, David. / Language identification from visual-only speech signals. In: Attention, Perception, and Psychophysics. 2010 ; Vol. 72, No. 6. pp. 1601-1613.
@article{2a5962d5059e4fa39699418adaf00f60,
title = "Language identification from visual-only speech signals",
abstract = "Our goal in the present study was to examine how observers identify English and Spanish from visual-only displays of speech. First, we replicated the recent findings of Soto-Faraco et al. (2007) with Spanish and English bilingual and monolingual observers using different languages and a different experimental paradigm (identification). We found that prior linguistic experience affected response bias but not sensitivity (Experiment 1). In two additional experiments, we investigated the visual cues that observers use to complete the languageidentification task. The results of Experiment 2 indicate that some lexical information is available in the visual signal but that it is limited. Acoustic analyses confirmed that our Spanish and English stimuli differed acoustically with respect to linguistic rhythmic categories. In Experiment 3, we tested whether this rhythmic difference could be used by observers to identify the language when the visual stimuli is temporally reversed, thereby eliminating lexical information but retaining rhythmic differences. The participants performed above chance even in the backward condition, suggesting that the rhythmic differences between the two languages may aid language identification in visual-only speech signals. The results of Experiments 3A and 3B also confirm previous findings that increased stimulus length facilitates language identification. Taken together, the results of these three experiments replicate earlier findings and also show that prior linguistic experience, lexical information, rhythmic structure, and utterance length influence visual-only language identification.",
author = "Ronquest, {Rebecca E.} and Levi, {Susannah V.} and David Pisoni",
year = "2010",
month = "8",
doi = "10.3758/APP.72.6.1601",
language = "English",
volume = "72",
pages = "1601--1613",
journal = "Attention, Perception, and Psychophysics",
issn = "1943-3921",
publisher = "Springer New York",
number = "6",

}

TY - JOUR

T1 - Language identification from visual-only speech signals

AU - Ronquest, Rebecca E.

AU - Levi, Susannah V.

AU - Pisoni, David

PY - 2010/8

Y1 - 2010/8

N2 - Our goal in the present study was to examine how observers identify English and Spanish from visual-only displays of speech. First, we replicated the recent findings of Soto-Faraco et al. (2007) with Spanish and English bilingual and monolingual observers using different languages and a different experimental paradigm (identification). We found that prior linguistic experience affected response bias but not sensitivity (Experiment 1). In two additional experiments, we investigated the visual cues that observers use to complete the languageidentification task. The results of Experiment 2 indicate that some lexical information is available in the visual signal but that it is limited. Acoustic analyses confirmed that our Spanish and English stimuli differed acoustically with respect to linguistic rhythmic categories. In Experiment 3, we tested whether this rhythmic difference could be used by observers to identify the language when the visual stimuli is temporally reversed, thereby eliminating lexical information but retaining rhythmic differences. The participants performed above chance even in the backward condition, suggesting that the rhythmic differences between the two languages may aid language identification in visual-only speech signals. The results of Experiments 3A and 3B also confirm previous findings that increased stimulus length facilitates language identification. Taken together, the results of these three experiments replicate earlier findings and also show that prior linguistic experience, lexical information, rhythmic structure, and utterance length influence visual-only language identification.

AB - Our goal in the present study was to examine how observers identify English and Spanish from visual-only displays of speech. First, we replicated the recent findings of Soto-Faraco et al. (2007) with Spanish and English bilingual and monolingual observers using different languages and a different experimental paradigm (identification). We found that prior linguistic experience affected response bias but not sensitivity (Experiment 1). In two additional experiments, we investigated the visual cues that observers use to complete the languageidentification task. The results of Experiment 2 indicate that some lexical information is available in the visual signal but that it is limited. Acoustic analyses confirmed that our Spanish and English stimuli differed acoustically with respect to linguistic rhythmic categories. In Experiment 3, we tested whether this rhythmic difference could be used by observers to identify the language when the visual stimuli is temporally reversed, thereby eliminating lexical information but retaining rhythmic differences. The participants performed above chance even in the backward condition, suggesting that the rhythmic differences between the two languages may aid language identification in visual-only speech signals. The results of Experiments 3A and 3B also confirm previous findings that increased stimulus length facilitates language identification. Taken together, the results of these three experiments replicate earlier findings and also show that prior linguistic experience, lexical information, rhythmic structure, and utterance length influence visual-only language identification.

UR - http://www.scopus.com/inward/record.url?scp=77957652068&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77957652068&partnerID=8YFLogxK

U2 - 10.3758/APP.72.6.1601

DO - 10.3758/APP.72.6.1601

M3 - Article

VL - 72

SP - 1601

EP - 1613

JO - Attention, Perception, and Psychophysics

JF - Attention, Perception, and Psychophysics

SN - 1943-3921

IS - 6

ER -