Comparison of automated posttonsillectomy bleed capture with self-report

D. Ryan Phillips, Susan E. Ellsperman, Bruce Matt, Ben L. Zarzaur

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Importance: Tonsillectomy is one of the most common procedures performed by otolaryngologists and is associated with postoperative bleeding. Bleed rates are usually monitored by self-report. Objective: To evaluate whether using automated capture and reporting of pediatric posttonsillectomy bleeding is feasible and accurate compared with traditional self-reporting by the surgical team. Design, Setting, and Participants: An automated complication-reporting algorithmwas designed to query the local health information exchange and then tested against self-reported tonsillectomy complication data collected from January 1, 2014, through December 31, 2015, at a tertiary pediatric hospital. The algorithm identified patients undergoing tonsillectomy and searched their postoperative encounters for a hand-selected set of diagnosis codes from the International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision and free-text words to identify complication events. Five months of the 2014-2015 data set were used to help design the algorithm. Data from the remaining 19 months were compared with self-reported complications. Main Outcomes and Measures: Automated system findings compared with self-reported bleeding events. Results: During the 19-month period, 1017 tonsillectomies were performed. We compared the algorithm's effectiveness in finding tonsillectomy and adenotonsillectomy procedures for the evaluated surgeons with the hand-reviewed master tonsillectomy list. The algorithm reported 51 false-positive (5.01%missed) and 74 false-negative (7.28%misidentified) procedures. The algorithm agreed with self-report for 986 tonsillectomies and disagreed on 31 cases (3.05%) (K = 0.69; 95%CI, 0.66-0.73). The algorithm was found to be sensitive to correctly identifying 60.53%(95%CI, 48.63%-71.34%) of tonsillectomies as having bleeding complications, with a specificity of 98.30% (95%CI, 97.19%-98.99%). Conclusions and Relevance: Capture of posttonsillectomy bleeding is possible through an automatic search of the medical record, although the algorithm will require continued refinement. Leveraging health information exchange data increases the possibilities of capturing complications at hospitals outside the local health system. Use of these algorithms will allow repeatable automated feedback to be provided to surgeons on a cyclical basis.

Original languageEnglish (US)
Pages (from-to)764-768
Number of pages5
JournalJAMA Otolaryngology - Head and Neck Surgery
Volume143
Issue number8
DOIs
StatePublished - Aug 1 2017

Fingerprint

Tonsillectomy
Self Report
Hemorrhage
International Classification of Diseases
Hand
Pediatric Hospitals
Tertiary Care Centers
Medical Records
Outcome Assessment (Health Care)
Pediatrics
Health

ASJC Scopus subject areas

  • Surgery
  • Otorhinolaryngology

Cite this

Comparison of automated posttonsillectomy bleed capture with self-report. / Phillips, D. Ryan; Ellsperman, Susan E.; Matt, Bruce; Zarzaur, Ben L.

In: JAMA Otolaryngology - Head and Neck Surgery, Vol. 143, No. 8, 01.08.2017, p. 764-768.

Research output: Contribution to journalArticle

Phillips, D. Ryan ; Ellsperman, Susan E. ; Matt, Bruce ; Zarzaur, Ben L. / Comparison of automated posttonsillectomy bleed capture with self-report. In: JAMA Otolaryngology - Head and Neck Surgery. 2017 ; Vol. 143, No. 8. pp. 764-768.
@article{1a417c78fbd046659e352a58583d2e89,
title = "Comparison of automated posttonsillectomy bleed capture with self-report",
abstract = "Importance: Tonsillectomy is one of the most common procedures performed by otolaryngologists and is associated with postoperative bleeding. Bleed rates are usually monitored by self-report. Objective: To evaluate whether using automated capture and reporting of pediatric posttonsillectomy bleeding is feasible and accurate compared with traditional self-reporting by the surgical team. Design, Setting, and Participants: An automated complication-reporting algorithmwas designed to query the local health information exchange and then tested against self-reported tonsillectomy complication data collected from January 1, 2014, through December 31, 2015, at a tertiary pediatric hospital. The algorithm identified patients undergoing tonsillectomy and searched their postoperative encounters for a hand-selected set of diagnosis codes from the International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision and free-text words to identify complication events. Five months of the 2014-2015 data set were used to help design the algorithm. Data from the remaining 19 months were compared with self-reported complications. Main Outcomes and Measures: Automated system findings compared with self-reported bleeding events. Results: During the 19-month period, 1017 tonsillectomies were performed. We compared the algorithm's effectiveness in finding tonsillectomy and adenotonsillectomy procedures for the evaluated surgeons with the hand-reviewed master tonsillectomy list. The algorithm reported 51 false-positive (5.01{\%}missed) and 74 false-negative (7.28{\%}misidentified) procedures. The algorithm agreed with self-report for 986 tonsillectomies and disagreed on 31 cases (3.05{\%}) (K = 0.69; 95{\%}CI, 0.66-0.73). The algorithm was found to be sensitive to correctly identifying 60.53{\%}(95{\%}CI, 48.63{\%}-71.34{\%}) of tonsillectomies as having bleeding complications, with a specificity of 98.30{\%} (95{\%}CI, 97.19{\%}-98.99{\%}). Conclusions and Relevance: Capture of posttonsillectomy bleeding is possible through an automatic search of the medical record, although the algorithm will require continued refinement. Leveraging health information exchange data increases the possibilities of capturing complications at hospitals outside the local health system. Use of these algorithms will allow repeatable automated feedback to be provided to surgeons on a cyclical basis.",
author = "Phillips, {D. Ryan} and Ellsperman, {Susan E.} and Bruce Matt and Zarzaur, {Ben L.}",
year = "2017",
month = "8",
day = "1",
doi = "10.1001/jamaoto.2017.0148",
language = "English (US)",
volume = "143",
pages = "764--768",
journal = "JAMA Otolaryngology - Head and Neck Surgery",
issn = "2168-6181",
publisher = "American Medical Association",
number = "8",

}

TY - JOUR

T1 - Comparison of automated posttonsillectomy bleed capture with self-report

AU - Phillips, D. Ryan

AU - Ellsperman, Susan E.

AU - Matt, Bruce

AU - Zarzaur, Ben L.

PY - 2017/8/1

Y1 - 2017/8/1

N2 - Importance: Tonsillectomy is one of the most common procedures performed by otolaryngologists and is associated with postoperative bleeding. Bleed rates are usually monitored by self-report. Objective: To evaluate whether using automated capture and reporting of pediatric posttonsillectomy bleeding is feasible and accurate compared with traditional self-reporting by the surgical team. Design, Setting, and Participants: An automated complication-reporting algorithmwas designed to query the local health information exchange and then tested against self-reported tonsillectomy complication data collected from January 1, 2014, through December 31, 2015, at a tertiary pediatric hospital. The algorithm identified patients undergoing tonsillectomy and searched their postoperative encounters for a hand-selected set of diagnosis codes from the International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision and free-text words to identify complication events. Five months of the 2014-2015 data set were used to help design the algorithm. Data from the remaining 19 months were compared with self-reported complications. Main Outcomes and Measures: Automated system findings compared with self-reported bleeding events. Results: During the 19-month period, 1017 tonsillectomies were performed. We compared the algorithm's effectiveness in finding tonsillectomy and adenotonsillectomy procedures for the evaluated surgeons with the hand-reviewed master tonsillectomy list. The algorithm reported 51 false-positive (5.01%missed) and 74 false-negative (7.28%misidentified) procedures. The algorithm agreed with self-report for 986 tonsillectomies and disagreed on 31 cases (3.05%) (K = 0.69; 95%CI, 0.66-0.73). The algorithm was found to be sensitive to correctly identifying 60.53%(95%CI, 48.63%-71.34%) of tonsillectomies as having bleeding complications, with a specificity of 98.30% (95%CI, 97.19%-98.99%). Conclusions and Relevance: Capture of posttonsillectomy bleeding is possible through an automatic search of the medical record, although the algorithm will require continued refinement. Leveraging health information exchange data increases the possibilities of capturing complications at hospitals outside the local health system. Use of these algorithms will allow repeatable automated feedback to be provided to surgeons on a cyclical basis.

AB - Importance: Tonsillectomy is one of the most common procedures performed by otolaryngologists and is associated with postoperative bleeding. Bleed rates are usually monitored by self-report. Objective: To evaluate whether using automated capture and reporting of pediatric posttonsillectomy bleeding is feasible and accurate compared with traditional self-reporting by the surgical team. Design, Setting, and Participants: An automated complication-reporting algorithmwas designed to query the local health information exchange and then tested against self-reported tonsillectomy complication data collected from January 1, 2014, through December 31, 2015, at a tertiary pediatric hospital. The algorithm identified patients undergoing tonsillectomy and searched their postoperative encounters for a hand-selected set of diagnosis codes from the International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision and free-text words to identify complication events. Five months of the 2014-2015 data set were used to help design the algorithm. Data from the remaining 19 months were compared with self-reported complications. Main Outcomes and Measures: Automated system findings compared with self-reported bleeding events. Results: During the 19-month period, 1017 tonsillectomies were performed. We compared the algorithm's effectiveness in finding tonsillectomy and adenotonsillectomy procedures for the evaluated surgeons with the hand-reviewed master tonsillectomy list. The algorithm reported 51 false-positive (5.01%missed) and 74 false-negative (7.28%misidentified) procedures. The algorithm agreed with self-report for 986 tonsillectomies and disagreed on 31 cases (3.05%) (K = 0.69; 95%CI, 0.66-0.73). The algorithm was found to be sensitive to correctly identifying 60.53%(95%CI, 48.63%-71.34%) of tonsillectomies as having bleeding complications, with a specificity of 98.30% (95%CI, 97.19%-98.99%). Conclusions and Relevance: Capture of posttonsillectomy bleeding is possible through an automatic search of the medical record, although the algorithm will require continued refinement. Leveraging health information exchange data increases the possibilities of capturing complications at hospitals outside the local health system. Use of these algorithms will allow repeatable automated feedback to be provided to surgeons on a cyclical basis.

UR - http://www.scopus.com/inward/record.url?scp=85027677088&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027677088&partnerID=8YFLogxK

U2 - 10.1001/jamaoto.2017.0148

DO - 10.1001/jamaoto.2017.0148

M3 - Article

VL - 143

SP - 764

EP - 768

JO - JAMA Otolaryngology - Head and Neck Surgery

JF - JAMA Otolaryngology - Head and Neck Surgery

SN - 2168-6181

IS - 8

ER -