Automatic measurement of speech recognition performance: a comparison of six speaker-dependent recognition devices

Howard C. Nusbaum, David Pisoni

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Although performance data are often freely cited by vendors of speech recognition devices, the conditions under which these data were collected are seldom specified in sufficient detail to permit comparisons among different systems. To directly compare the performance of six commercially available speech recognition devices, we developed a computer-controlled testing system and a set of standard tests. We carried out these tests to assess the performance of speech recognition devices sold by Texas Instruments, Votan, Dragon, IBM, Interstate, and NEC. The results demonstrate several reliable performance differences among these systems. In general, however, performance differences among these devices are quite small and are reduced by appropriate training. Our results also indicate that the effects of training on performance of a speech recognition device are much greater for difficult vocabularies than they are for discriminable vocabularies. Finally, an examination of the results for recognition of the speech produced by one talker in the testing database suggests that user-specific difficulties in recognition performance may, in some cases, result from interactions among the application vocabulary, the user's speech, and the algorithm of a recognition device.

Original languageEnglish
Pages (from-to)87-108
Number of pages22
JournalComputer Speech and Language
Volume2
Issue number2
DOIs
StatePublished - 1987

Fingerprint

Speech Recognition
Speech recognition
Equipment and Supplies
Vocabulary
Dependent
performance
vocabulary
Testing
Databases
Sufficient
examination
interaction
Interaction
Demonstrate

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Experimental and Cognitive Psychology
  • Linguistics and Language

Cite this

Automatic measurement of speech recognition performance : a comparison of six speaker-dependent recognition devices. / Nusbaum, Howard C.; Pisoni, David.

In: Computer Speech and Language, Vol. 2, No. 2, 1987, p. 87-108.

Research output: Contribution to journalArticle

@article{cf1bfa1e78dd482fbd1f04688b93761b,
title = "Automatic measurement of speech recognition performance: a comparison of six speaker-dependent recognition devices",
abstract = "Although performance data are often freely cited by vendors of speech recognition devices, the conditions under which these data were collected are seldom specified in sufficient detail to permit comparisons among different systems. To directly compare the performance of six commercially available speech recognition devices, we developed a computer-controlled testing system and a set of standard tests. We carried out these tests to assess the performance of speech recognition devices sold by Texas Instruments, Votan, Dragon, IBM, Interstate, and NEC. The results demonstrate several reliable performance differences among these systems. In general, however, performance differences among these devices are quite small and are reduced by appropriate training. Our results also indicate that the effects of training on performance of a speech recognition device are much greater for difficult vocabularies than they are for discriminable vocabularies. Finally, an examination of the results for recognition of the speech produced by one talker in the testing database suggests that user-specific difficulties in recognition performance may, in some cases, result from interactions among the application vocabulary, the user's speech, and the algorithm of a recognition device.",
author = "Nusbaum, {Howard C.} and David Pisoni",
year = "1987",
doi = "10.1016/0885-2308(87)90002-7",
language = "English",
volume = "2",
pages = "87--108",
journal = "Computer Speech and Language",
issn = "0885-2308",
publisher = "Academic Press Inc.",
number = "2",

}

TY - JOUR

T1 - Automatic measurement of speech recognition performance

T2 - a comparison of six speaker-dependent recognition devices

AU - Nusbaum, Howard C.

AU - Pisoni, David

PY - 1987

Y1 - 1987

N2 - Although performance data are often freely cited by vendors of speech recognition devices, the conditions under which these data were collected are seldom specified in sufficient detail to permit comparisons among different systems. To directly compare the performance of six commercially available speech recognition devices, we developed a computer-controlled testing system and a set of standard tests. We carried out these tests to assess the performance of speech recognition devices sold by Texas Instruments, Votan, Dragon, IBM, Interstate, and NEC. The results demonstrate several reliable performance differences among these systems. In general, however, performance differences among these devices are quite small and are reduced by appropriate training. Our results also indicate that the effects of training on performance of a speech recognition device are much greater for difficult vocabularies than they are for discriminable vocabularies. Finally, an examination of the results for recognition of the speech produced by one talker in the testing database suggests that user-specific difficulties in recognition performance may, in some cases, result from interactions among the application vocabulary, the user's speech, and the algorithm of a recognition device.

AB - Although performance data are often freely cited by vendors of speech recognition devices, the conditions under which these data were collected are seldom specified in sufficient detail to permit comparisons among different systems. To directly compare the performance of six commercially available speech recognition devices, we developed a computer-controlled testing system and a set of standard tests. We carried out these tests to assess the performance of speech recognition devices sold by Texas Instruments, Votan, Dragon, IBM, Interstate, and NEC. The results demonstrate several reliable performance differences among these systems. In general, however, performance differences among these devices are quite small and are reduced by appropriate training. Our results also indicate that the effects of training on performance of a speech recognition device are much greater for difficult vocabularies than they are for discriminable vocabularies. Finally, an examination of the results for recognition of the speech produced by one talker in the testing database suggests that user-specific difficulties in recognition performance may, in some cases, result from interactions among the application vocabulary, the user's speech, and the algorithm of a recognition device.

UR - http://www.scopus.com/inward/record.url?scp=0142085471&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0142085471&partnerID=8YFLogxK

U2 - 10.1016/0885-2308(87)90002-7

DO - 10.1016/0885-2308(87)90002-7

M3 - Article

AN - SCOPUS:0142085471

VL - 2

SP - 87

EP - 108

JO - Computer Speech and Language

JF - Computer Speech and Language

SN - 0885-2308

IS - 2

ER -