Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies

Glebys Gonzalez, Naveen Madapana, Rahul Taneja, Lingsong Zhang, Richard Rodgers, Juan P. Wachs

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The choice of what gestures should be part of a gesture language is a critical step in the design of gesture-based interfaces. This step is especially important when time and accuracy are key factors of the user experience, such as gestural interfaces in vehicle control and sterile control of a picture archiving and communication system (PACS) in the operating room (OR). Agreement studies are commonly used to find the gesture preference of the end users. These studies hypothesize that the best available gesture lexicon is the one preferred by a majority. However, these agreement approaches cannot offer a metric to assess the qualitative aspects of gestures. In this work, we propose an experimental framework to quantify, compare and evaluate gestures. This framework is grounded in the expert knowledge of speech and language professionals (SLPs). The development consisted of three studies: 1) Creation, 2) Evaluation and 3) Validation. In the creation study, we followed an adapted version of the Delphi's interview/discussion procedure with SLPs. The purpose was to obtain the Vocabulary Acceptability Criteria (VAC) to evaluate gestures. Next, in the evaluation study, a modified method of pairwise comparisons was used to rank and quantify the gestures based on each criteria (VAC). Lastly, in the validation study, we formulated an odd one out procedure, to prove that the VAC values of a gesture are representative and sufficiently distinctive, to select that particular gesture from a pool of gestures. We applied this framework to the gestures obtained from a gesture elicitation study conducted with nine neurosurgeons, to control an imaging software. In addition, 29 SLPs comprising of 17 experts and 12 graduate students participated in the VAC study. The best lexicons from the available pool were obtained through both agreement and VAC metrics. We used binomial tests to show that the results obtained from the validation procedure are significantly better than the baseline. These results verify our hypothesis that the VAC are representative of the gestures and the subjects should be able to select the right gesture given its VAC values.

Original languageEnglish (US)
Title of host publication62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
PublisherHuman Factors and Ergonomics Society Inc.
Pages997-1001
Number of pages5
ISBN (Electronic)9781510889538
StatePublished - Jan 1 2018
Event62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018 - Philadelphia, United States
Duration: Oct 1 2018Oct 5 2018

Publication series

NameProceedings of the Human Factors and Ergonomics Society
Volume2
ISSN (Print)1071-1813

Conference

Conference62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
CountryUnited States
CityPhiladelphia
Period10/1/1810/5/18

Fingerprint

vocabulary
Operating rooms
Picture archiving and communication systems
Students
Imaging techniques
language
comparison of methods
expert knowledge
evaluation
communication system
Values
graduate
expert

ASJC Scopus subject areas

  • Human Factors and Ergonomics

Cite this

Gonzalez, G., Madapana, N., Taneja, R., Zhang, L., Rodgers, R., & Wachs, J. P. (2018). Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies. In 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018 (pp. 997-1001). (Proceedings of the Human Factors and Ergonomics Society; Vol. 2). Human Factors and Ergonomics Society Inc..

Looking beyond the gesture : Vocabulary acceptability Criteria for gesture elicitation studies. / Gonzalez, Glebys; Madapana, Naveen; Taneja, Rahul; Zhang, Lingsong; Rodgers, Richard; Wachs, Juan P.

62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018. Human Factors and Ergonomics Society Inc., 2018. p. 997-1001 (Proceedings of the Human Factors and Ergonomics Society; Vol. 2).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gonzalez, G, Madapana, N, Taneja, R, Zhang, L, Rodgers, R & Wachs, JP 2018, Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies. in 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018. Proceedings of the Human Factors and Ergonomics Society, vol. 2, Human Factors and Ergonomics Society Inc., pp. 997-1001, 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018, Philadelphia, United States, 10/1/18.
Gonzalez G, Madapana N, Taneja R, Zhang L, Rodgers R, Wachs JP. Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies. In 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018. Human Factors and Ergonomics Society Inc. 2018. p. 997-1001. (Proceedings of the Human Factors and Ergonomics Society).
Gonzalez, Glebys ; Madapana, Naveen ; Taneja, Rahul ; Zhang, Lingsong ; Rodgers, Richard ; Wachs, Juan P. / Looking beyond the gesture : Vocabulary acceptability Criteria for gesture elicitation studies. 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018. Human Factors and Ergonomics Society Inc., 2018. pp. 997-1001 (Proceedings of the Human Factors and Ergonomics Society).
@inproceedings{f8c480fc531c401c8a35d3396a8b2d1f,
title = "Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies",
abstract = "The choice of what gestures should be part of a gesture language is a critical step in the design of gesture-based interfaces. This step is especially important when time and accuracy are key factors of the user experience, such as gestural interfaces in vehicle control and sterile control of a picture archiving and communication system (PACS) in the operating room (OR). Agreement studies are commonly used to find the gesture preference of the end users. These studies hypothesize that the best available gesture lexicon is the one preferred by a majority. However, these agreement approaches cannot offer a metric to assess the qualitative aspects of gestures. In this work, we propose an experimental framework to quantify, compare and evaluate gestures. This framework is grounded in the expert knowledge of speech and language professionals (SLPs). The development consisted of three studies: 1) Creation, 2) Evaluation and 3) Validation. In the creation study, we followed an adapted version of the Delphi's interview/discussion procedure with SLPs. The purpose was to obtain the Vocabulary Acceptability Criteria (VAC) to evaluate gestures. Next, in the evaluation study, a modified method of pairwise comparisons was used to rank and quantify the gestures based on each criteria (VAC). Lastly, in the validation study, we formulated an odd one out procedure, to prove that the VAC values of a gesture are representative and sufficiently distinctive, to select that particular gesture from a pool of gestures. We applied this framework to the gestures obtained from a gesture elicitation study conducted with nine neurosurgeons, to control an imaging software. In addition, 29 SLPs comprising of 17 experts and 12 graduate students participated in the VAC study. The best lexicons from the available pool were obtained through both agreement and VAC metrics. We used binomial tests to show that the results obtained from the validation procedure are significantly better than the baseline. These results verify our hypothesis that the VAC are representative of the gestures and the subjects should be able to select the right gesture given its VAC values.",
author = "Glebys Gonzalez and Naveen Madapana and Rahul Taneja and Lingsong Zhang and Richard Rodgers and Wachs, {Juan P.}",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
series = "Proceedings of the Human Factors and Ergonomics Society",
publisher = "Human Factors and Ergonomics Society Inc.",
pages = "997--1001",
booktitle = "62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018",

}

TY - GEN

T1 - Looking beyond the gesture

T2 - Vocabulary acceptability Criteria for gesture elicitation studies

AU - Gonzalez, Glebys

AU - Madapana, Naveen

AU - Taneja, Rahul

AU - Zhang, Lingsong

AU - Rodgers, Richard

AU - Wachs, Juan P.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The choice of what gestures should be part of a gesture language is a critical step in the design of gesture-based interfaces. This step is especially important when time and accuracy are key factors of the user experience, such as gestural interfaces in vehicle control and sterile control of a picture archiving and communication system (PACS) in the operating room (OR). Agreement studies are commonly used to find the gesture preference of the end users. These studies hypothesize that the best available gesture lexicon is the one preferred by a majority. However, these agreement approaches cannot offer a metric to assess the qualitative aspects of gestures. In this work, we propose an experimental framework to quantify, compare and evaluate gestures. This framework is grounded in the expert knowledge of speech and language professionals (SLPs). The development consisted of three studies: 1) Creation, 2) Evaluation and 3) Validation. In the creation study, we followed an adapted version of the Delphi's interview/discussion procedure with SLPs. The purpose was to obtain the Vocabulary Acceptability Criteria (VAC) to evaluate gestures. Next, in the evaluation study, a modified method of pairwise comparisons was used to rank and quantify the gestures based on each criteria (VAC). Lastly, in the validation study, we formulated an odd one out procedure, to prove that the VAC values of a gesture are representative and sufficiently distinctive, to select that particular gesture from a pool of gestures. We applied this framework to the gestures obtained from a gesture elicitation study conducted with nine neurosurgeons, to control an imaging software. In addition, 29 SLPs comprising of 17 experts and 12 graduate students participated in the VAC study. The best lexicons from the available pool were obtained through both agreement and VAC metrics. We used binomial tests to show that the results obtained from the validation procedure are significantly better than the baseline. These results verify our hypothesis that the VAC are representative of the gestures and the subjects should be able to select the right gesture given its VAC values.

AB - The choice of what gestures should be part of a gesture language is a critical step in the design of gesture-based interfaces. This step is especially important when time and accuracy are key factors of the user experience, such as gestural interfaces in vehicle control and sterile control of a picture archiving and communication system (PACS) in the operating room (OR). Agreement studies are commonly used to find the gesture preference of the end users. These studies hypothesize that the best available gesture lexicon is the one preferred by a majority. However, these agreement approaches cannot offer a metric to assess the qualitative aspects of gestures. In this work, we propose an experimental framework to quantify, compare and evaluate gestures. This framework is grounded in the expert knowledge of speech and language professionals (SLPs). The development consisted of three studies: 1) Creation, 2) Evaluation and 3) Validation. In the creation study, we followed an adapted version of the Delphi's interview/discussion procedure with SLPs. The purpose was to obtain the Vocabulary Acceptability Criteria (VAC) to evaluate gestures. Next, in the evaluation study, a modified method of pairwise comparisons was used to rank and quantify the gestures based on each criteria (VAC). Lastly, in the validation study, we formulated an odd one out procedure, to prove that the VAC values of a gesture are representative and sufficiently distinctive, to select that particular gesture from a pool of gestures. We applied this framework to the gestures obtained from a gesture elicitation study conducted with nine neurosurgeons, to control an imaging software. In addition, 29 SLPs comprising of 17 experts and 12 graduate students participated in the VAC study. The best lexicons from the available pool were obtained through both agreement and VAC metrics. We used binomial tests to show that the results obtained from the validation procedure are significantly better than the baseline. These results verify our hypothesis that the VAC are representative of the gestures and the subjects should be able to select the right gesture given its VAC values.

UR - http://www.scopus.com/inward/record.url?scp=85072748198&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072748198&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85072748198

T3 - Proceedings of the Human Factors and Ergonomics Society

SP - 997

EP - 1001

BT - 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018

PB - Human Factors and Ergonomics Society Inc.

ER -