Collaboration with a robotic scrub nurse

Mithun George Jacob, Yu Ting Li, George A. Akingba, Juan P. Wachs

Research output: Contribution to journalArticlepeer-review

21 Scopus citations


The main use of robotics in surgery is not to replace the surgeon or surgical nurses but to work with them during surgery. A significant advantage of gesture-based communication is it requires no special training by the surgeon. Gesturing comes naturally to surgeons since their hands are already their main tools. Moreover, hand signs are the standard method for requesting surgical instruments, and gestures are not affected by ambient noise in the OR. A multimodal solution combining voice and gesture provides redundancy needed to assure proper instrument delivery. Gestures for robotic control have been the focus of much research since the early 1980s. Early work was done with Richard A. Bolt's Put-That-There interface followed by others using magnetic sensors or gloves to encode hand signs. The streaming depth maps captured through the Kinect sensor are processed by the gesture recognition module while a microphone concurrently captures voice commands interpreted by the speech recognition module. Following recognition, a command is transmitted to the robot through an application that controls a Fanuc LR Mate 200iC robotic arm across the network through a Telnet interface.

Original languageEnglish (US)
Pages (from-to)68-75
Number of pages8
JournalCommunications of the ACM
Issue number5
StatePublished - May 1 2013

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Collaboration with a robotic scrub nurse'. Together they form a unique fingerprint.

Cite this