Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies

Glebys Gonzalez, Naveen Madapana, Rahul Taneja, Lingsong Zhang, Richard Rodgers, Juan P. Wachs

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The choice of what gestures should be part of a gesture language is a critical step in the design of gesture-based interfaces. This step is especially important when time and accuracy are key factors of the user experience, such as gestural interfaces in vehicle control and sterile control of a picture archiving and communication system (PACS) in the operating room (OR). Agreement studies are commonly used to find the gesture preference of the end users. These studies hypothesize that the best available gesture lexicon is the one preferred by a majority. However, these agreement approaches cannot offer a metric to assess the qualitative aspects of gestures. In this work, we propose an experimental framework to quantify, compare and evaluate gestures. This framework is grounded in the expert knowledge of speech and language professionals (SLPs). The development consisted of three studies: 1) Creation, 2) Evaluation and 3) Validation. In the creation study, we followed an adapted version of the Delphi's interview/discussion procedure with SLPs. The purpose was to obtain the Vocabulary Acceptability Criteria (VAC) to evaluate gestures. Next, in the evaluation study, a modified method of pairwise comparisons was used to rank and quantify the gestures based on each criteria (VAC). Lastly, in the validation study, we formulated an odd one out procedure, to prove that the VAC values of a gesture are representative and sufficiently distinctive, to select that particular gesture from a pool of gestures. We applied this framework to the gestures obtained from a gesture elicitation study conducted with nine neurosurgeons, to control an imaging software. In addition, 29 SLPs comprising of 17 experts and 12 graduate students participated in the VAC study. The best lexicons from the available pool were obtained through both agreement and VAC metrics. We used binomial tests to show that the results obtained from the validation procedure are significantly better than the baseline. These results verify our hypothesis that the VAC are representative of the gestures and the subjects should be able to select the right gesture given its VAC values.

Original languageEnglish (US)
Title of host publication62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
PublisherHuman Factors and Ergonomics Society Inc.
Pages997-1001
Number of pages5
ISBN (Electronic)9781510889538
StatePublished - Jan 1 2018
Event62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018 - Philadelphia, United States
Duration: Oct 1 2018Oct 5 2018

Publication series

NameProceedings of the Human Factors and Ergonomics Society
Volume2
ISSN (Print)1071-1813

Conference

Conference62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
CountryUnited States
CityPhiladelphia
Period10/1/1810/5/18

    Fingerprint

ASJC Scopus subject areas

  • Human Factors and Ergonomics

Cite this

Gonzalez, G., Madapana, N., Taneja, R., Zhang, L., Rodgers, R., & Wachs, J. P. (2018). Looking beyond the gesture: Vocabulary acceptability Criteria for gesture elicitation studies. In 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018 (pp. 997-1001). (Proceedings of the Human Factors and Ergonomics Society; Vol. 2). Human Factors and Ergonomics Society Inc..