Augmented Reality Future Step Visualization for Robust Surgical Telementoring

Daniel S. Andersen, Maria E. Cabrera, Edgar J. Rojas-Muñoz, Voicu S. Popescu, Glebys T. Gonzalez, Brian Mullis, Sherri Marley, Ben L. Zarzaur, Juan P. Wachs

Research output: Contribution to journalReview article

Abstract

Introduction Surgical telementoring connects expert mentors with trainees performing urgent care in austere environments. However, such environments impose unreliable network quality, with significant latency and low bandwidth. We have developed an augmented reality telementoring system that includes future step visualization of the medical procedure. Pregenerated video instructions of the procedure are dynamically overlaid onto the trainee's view of the operating field when the network connection with a mentor is unreliable. Methods Our future step visualization uses a tablet suspended above the patient's body, through which the trainee views the operating field. Before trainee use, an expert records a "future library" of step-by-step video footage of the operation. Videos are displayed to the trainee as semitransparent graphical overlays. We conducted a study where participants completed a cricothyroidotomy under telementored guidance. Participants used one of two telementoring conditions: conventional telestrator or our system with future step visualization. During the operation, the connection between trainee and mentor was bandwidth throttled. Recorded metrics were idle time ratio, recall error, and task performance. Results Participants in the future step visualization condition had 48% smaller idle time ratio (14.5% vs. 27.9%, P < 0.001), 26% less recall error (119 vs. 161, P = 0.042), and 10% higher task performance scores (rater 1 = 90.83 vs. 81.88, P = 0.008; rater 2 = 88.54 vs. 79.17, P = 0.042) than participants in the telestrator condition. Conclusions Future step visualization in surgical telementoring is an important fallback mechanism when trainee/mentor network connection is poor, and it is a key step towards semiautonomous and then completely mentor-free medical assistance systems.

Original languageEnglish (US)
Pages (from-to)59-66
Number of pages8
JournalSimulation in Healthcare
Volume14
Issue number1
DOIs
StatePublished - Feb 1 2019
Externally publishedYes

Fingerprint

Mentors
Augmented reality
Augmented Reality
trainee
visualization
Visualization
Task Performance and Analysis
video
Bandwidth
Medical Assistance
Ambulatory Care
expert
Overlay
Tablets
Libraries
Guidance
Latency
performance
Metric
assistance

Keywords

  • Augmented reality
  • simulator training
  • telemedicine
  • telementoring

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

Andersen, D. S., Cabrera, M. E., Rojas-Muñoz, E. J., Popescu, V. S., Gonzalez, G. T., Mullis, B., ... Wachs, J. P. (2019). Augmented Reality Future Step Visualization for Robust Surgical Telementoring. Simulation in Healthcare, 14(1), 59-66. https://doi.org/10.1097/SIH.0000000000000334

Augmented Reality Future Step Visualization for Robust Surgical Telementoring. / Andersen, Daniel S.; Cabrera, Maria E.; Rojas-Muñoz, Edgar J.; Popescu, Voicu S.; Gonzalez, Glebys T.; Mullis, Brian; Marley, Sherri; Zarzaur, Ben L.; Wachs, Juan P.

In: Simulation in Healthcare, Vol. 14, No. 1, 01.02.2019, p. 59-66.

Research output: Contribution to journalReview article

Andersen, DS, Cabrera, ME, Rojas-Muñoz, EJ, Popescu, VS, Gonzalez, GT, Mullis, B, Marley, S, Zarzaur, BL & Wachs, JP 2019, 'Augmented Reality Future Step Visualization for Robust Surgical Telementoring', Simulation in Healthcare, vol. 14, no. 1, pp. 59-66. https://doi.org/10.1097/SIH.0000000000000334
Andersen, Daniel S. ; Cabrera, Maria E. ; Rojas-Muñoz, Edgar J. ; Popescu, Voicu S. ; Gonzalez, Glebys T. ; Mullis, Brian ; Marley, Sherri ; Zarzaur, Ben L. ; Wachs, Juan P. / Augmented Reality Future Step Visualization for Robust Surgical Telementoring. In: Simulation in Healthcare. 2019 ; Vol. 14, No. 1. pp. 59-66.
@article{755ca9a917d14b1d8060d3e1512ad06f,
title = "Augmented Reality Future Step Visualization for Robust Surgical Telementoring",
abstract = "Introduction Surgical telementoring connects expert mentors with trainees performing urgent care in austere environments. However, such environments impose unreliable network quality, with significant latency and low bandwidth. We have developed an augmented reality telementoring system that includes future step visualization of the medical procedure. Pregenerated video instructions of the procedure are dynamically overlaid onto the trainee's view of the operating field when the network connection with a mentor is unreliable. Methods Our future step visualization uses a tablet suspended above the patient's body, through which the trainee views the operating field. Before trainee use, an expert records a {"}future library{"} of step-by-step video footage of the operation. Videos are displayed to the trainee as semitransparent graphical overlays. We conducted a study where participants completed a cricothyroidotomy under telementored guidance. Participants used one of two telementoring conditions: conventional telestrator or our system with future step visualization. During the operation, the connection between trainee and mentor was bandwidth throttled. Recorded metrics were idle time ratio, recall error, and task performance. Results Participants in the future step visualization condition had 48{\%} smaller idle time ratio (14.5{\%} vs. 27.9{\%}, P < 0.001), 26{\%} less recall error (119 vs. 161, P = 0.042), and 10{\%} higher task performance scores (rater 1 = 90.83 vs. 81.88, P = 0.008; rater 2 = 88.54 vs. 79.17, P = 0.042) than participants in the telestrator condition. Conclusions Future step visualization in surgical telementoring is an important fallback mechanism when trainee/mentor network connection is poor, and it is a key step towards semiautonomous and then completely mentor-free medical assistance systems.",
keywords = "Augmented reality, simulator training, telemedicine, telementoring",
author = "Andersen, {Daniel S.} and Cabrera, {Maria E.} and Rojas-Mu{\~n}oz, {Edgar J.} and Popescu, {Voicu S.} and Gonzalez, {Glebys T.} and Brian Mullis and Sherri Marley and Zarzaur, {Ben L.} and Wachs, {Juan P.}",
year = "2019",
month = "2",
day = "1",
doi = "10.1097/SIH.0000000000000334",
language = "English (US)",
volume = "14",
pages = "59--66",
journal = "Simulation in Healthcare",
issn = "1559-2332",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Augmented Reality Future Step Visualization for Robust Surgical Telementoring

AU - Andersen, Daniel S.

AU - Cabrera, Maria E.

AU - Rojas-Muñoz, Edgar J.

AU - Popescu, Voicu S.

AU - Gonzalez, Glebys T.

AU - Mullis, Brian

AU - Marley, Sherri

AU - Zarzaur, Ben L.

AU - Wachs, Juan P.

PY - 2019/2/1

Y1 - 2019/2/1

N2 - Introduction Surgical telementoring connects expert mentors with trainees performing urgent care in austere environments. However, such environments impose unreliable network quality, with significant latency and low bandwidth. We have developed an augmented reality telementoring system that includes future step visualization of the medical procedure. Pregenerated video instructions of the procedure are dynamically overlaid onto the trainee's view of the operating field when the network connection with a mentor is unreliable. Methods Our future step visualization uses a tablet suspended above the patient's body, through which the trainee views the operating field. Before trainee use, an expert records a "future library" of step-by-step video footage of the operation. Videos are displayed to the trainee as semitransparent graphical overlays. We conducted a study where participants completed a cricothyroidotomy under telementored guidance. Participants used one of two telementoring conditions: conventional telestrator or our system with future step visualization. During the operation, the connection between trainee and mentor was bandwidth throttled. Recorded metrics were idle time ratio, recall error, and task performance. Results Participants in the future step visualization condition had 48% smaller idle time ratio (14.5% vs. 27.9%, P < 0.001), 26% less recall error (119 vs. 161, P = 0.042), and 10% higher task performance scores (rater 1 = 90.83 vs. 81.88, P = 0.008; rater 2 = 88.54 vs. 79.17, P = 0.042) than participants in the telestrator condition. Conclusions Future step visualization in surgical telementoring is an important fallback mechanism when trainee/mentor network connection is poor, and it is a key step towards semiautonomous and then completely mentor-free medical assistance systems.

AB - Introduction Surgical telementoring connects expert mentors with trainees performing urgent care in austere environments. However, such environments impose unreliable network quality, with significant latency and low bandwidth. We have developed an augmented reality telementoring system that includes future step visualization of the medical procedure. Pregenerated video instructions of the procedure are dynamically overlaid onto the trainee's view of the operating field when the network connection with a mentor is unreliable. Methods Our future step visualization uses a tablet suspended above the patient's body, through which the trainee views the operating field. Before trainee use, an expert records a "future library" of step-by-step video footage of the operation. Videos are displayed to the trainee as semitransparent graphical overlays. We conducted a study where participants completed a cricothyroidotomy under telementored guidance. Participants used one of two telementoring conditions: conventional telestrator or our system with future step visualization. During the operation, the connection between trainee and mentor was bandwidth throttled. Recorded metrics were idle time ratio, recall error, and task performance. Results Participants in the future step visualization condition had 48% smaller idle time ratio (14.5% vs. 27.9%, P < 0.001), 26% less recall error (119 vs. 161, P = 0.042), and 10% higher task performance scores (rater 1 = 90.83 vs. 81.88, P = 0.008; rater 2 = 88.54 vs. 79.17, P = 0.042) than participants in the telestrator condition. Conclusions Future step visualization in surgical telementoring is an important fallback mechanism when trainee/mentor network connection is poor, and it is a key step towards semiautonomous and then completely mentor-free medical assistance systems.

KW - Augmented reality

KW - simulator training

KW - telemedicine

KW - telementoring

UR - http://www.scopus.com/inward/record.url?scp=85060942295&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060942295&partnerID=8YFLogxK

U2 - 10.1097/SIH.0000000000000334

DO - 10.1097/SIH.0000000000000334

M3 - Review article

C2 - 30395078

AN - SCOPUS:85060942295

VL - 14

SP - 59

EP - 66

JO - Simulation in Healthcare

JF - Simulation in Healthcare

SN - 1559-2332

IS - 1

ER -