Virtual annotations of the surgical field through an augmented reality transparent display

Daniel Andersen, Voicu Popescu, Maria Eugenia Cabrera, Aditya Shanghavi, Gerardo Gomez, Sherri Marley, Brian Mullis, Juan Wachs

Research output: Contribution to journalArticle

10 Scopus citations


Existing telestrator-based surgical telementoring systems require a trainee surgeon to shift focus frequently between the operating field and a nearby monitor to acquire and apply instructions from a remote mentor. We present a novel approach to surgical telementoring where annotations are superimposed directly onto the surgical field using an augmented reality (AR) simulated transparent display. We present our first steps towards realizing this vision, using two networked conventional tablets to allow a mentor to remotely annotate the operating field as seen by a trainee. Annotations are anchored to the surgical field as the trainee tablet moves and as the surgical field deforms or becomes occluded. The system is built exclusively from compact commodity-level components—all imaging and processing are performed on the two tablets.

Original languageEnglish (US)
Pages (from-to)1481-1498
Number of pages18
JournalVisual Computer
Issue number11
StatePublished - Nov 1 2016



  • Annotation anchoring
  • Augmented reality
  • Telemedicine
  • Telementoring

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this

Andersen, D., Popescu, V., Cabrera, M. E., Shanghavi, A., Gomez, G., Marley, S., Mullis, B., & Wachs, J. (2016). Virtual annotations of the surgical field through an augmented reality transparent display. Visual Computer, 32(11), 1481-1498.