A First-Person Mentee Second-Person Mentor AR Interface for Surgical Telementoring

Chengyuan Lin, Daniel Andersen, Voicu Popescu, Edgar Rojas-Munoz, Maria Eugenia Cabrera, Brian Mullis, Ben Zarzaur, Kathryn Anderson, Sherri Marley, Juan Wachs

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

This application paper presents the work of a multidisciplinary group of designing, implementing, and testing an Augmented Reality (AR) surgical telementoring system. The system acquires the surgical field with an overhead camera, the video feed is transmitted to the remote mentor, where it is displayed on a touch-based interaction table, the mentor annotates the video feed, the annotations are sent back to the mentee, where they are displayed into the mentee's field of view using an optical see-through AR head-mounted display (HMD). The annotations are reprojected from the mentor's second-person view of the surgical field to the mentee's first-person view. The mentee sees the annotations with depth perception, and the annotations remain anchored to the surgical field as the mentee moves their head. Average annotation display accuracy is 1.22cm. The system was tested in the context of a user study where surgery residents ($n = 20$) were asked to perform a lower-leg fasciotomy on cadaver models. Participants who benefited from telementoring using our system received a higher Individual Performance Score, and they reported higher usability and self confidence levels.

Original languageEnglish (US)
Title of host publicationAdjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3-8
Number of pages6
ISBN (Electronic)9781538675922
DOIs
StatePublished - Jul 2 2018
Event17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 - Munich, Germany
Duration: Oct 16 2018Oct 20 2018

Publication series

NameAdjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

Conference

Conference17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
CountryGermany
CityMunich
Period10/16/1810/20/18

    Fingerprint

Keywords

  • Human computer interaction
  • Human-centered computing
  • Interaction paradigms
  • Mixed / augmented reality

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this

Lin, C., Andersen, D., Popescu, V., Rojas-Munoz, E., Cabrera, M. E., Mullis, B., Zarzaur, B., Anderson, K., Marley, S., & Wachs, J. (2018). A First-Person Mentee Second-Person Mentor AR Interface for Surgical Telementoring. In Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 (pp. 3-8). [8699305] (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct.2018.00021