Decision support from local data

Creating adaptive order menus from past clinician behavior

Jeffrey G. Klann, Peter Szolovits, Stephen Downs, Gunther Schadow

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Objective: Reducing care variability through guidelines has significantly benefited patients. Nonetheless, guideline-based Clinical Decision Support (CDS) systems are not widely implemented or used, are frequently out-of-date, and cannot address complex care for which guidelines do not exist. Here, we develop and evaluate a complementary approach - using Bayesian Network (BN) learning to generate adaptive, context-specific treatment menus based on local order-entry data. These menus can be used as a draft for expert review, in order to minimize development time for local decision support content. This is in keeping with the vision outlined in the US Health Information Technology Strategic Plan, which describes a healthcare system that learns from itself. Materials and methods: We used the Greedy Equivalence Search algorithm to learn four 50-node domain-specific BNs from 11,344 encounters: abdominal pain in the emergency department, inpatient pregnancy, hypertension in the Urgent Visit Clinic, and altered mental state in the intensive care unit. We developed a system to produce situation-specific, rank-ordered treatment menus from these networks. We evaluated this system with a hospital-simulation methodology and computed Area Under the Receiver-Operator Curve (AUC) and average menu position at time of selection. We also compared this system with a similar association-rule-mining approach. Results: A short order menu on average contained the next order (weighted average length 3.91-5.83 items). Overall predictive ability was good: average AUC above 0.9 for 25% of order types and overall average AUC 714-844 (depending on domain). However, AUC had high variance (50-99). Higher AUC correlated with tighter clusters and more connections in the graphs, indicating importance of appropriate contextual data. Comparison with an Association Rule Mining approach showed similar performance for only the most common orders with dramatic divergence as orders are less frequent. Discussion and conclusion: This study demonstrates that local clinical knowledge can be extracted from treatment data for decision support. This approach is appealing because: it reflects local standards; it uses data already being captured; and it produces human-readable treatment-diagnosis networks that could be curated by a human expert to reduce workload in developing localized CDS content. The BN methodology captured transitive associations and co-varying relationships, which existing approaches do not. It also performs better as orders become less frequent and require more context. This system is a step forward in harnessing local, empirical data to enhance decision support.

Original languageEnglish
Pages (from-to)84-93
Number of pages10
JournalJournal of Biomedical Informatics
Volume48
DOIs
StatePublished - 2014

Fingerprint

Association rules
Bayesian networks
Area Under Curve
Intensive care units
Clinical Decision Support Systems
Decision support systems
Information technology
Guidelines
Data acquisition
Health
Medical Informatics
Aptitude
Bayes Theorem
Therapeutics
Ambulatory Care
Workload
Abdominal Pain
Intensive Care Units
Hospital Emergency Service
Inpatients

Keywords

  • Bayesian analysis
  • Clinical Decision Support
  • Data mining

ASJC Scopus subject areas

  • Health Informatics
  • Computer Science Applications
  • Medicine(all)

Cite this

Decision support from local data : Creating adaptive order menus from past clinician behavior. / Klann, Jeffrey G.; Szolovits, Peter; Downs, Stephen; Schadow, Gunther.

In: Journal of Biomedical Informatics, Vol. 48, 2014, p. 84-93.

Research output: Contribution to journalArticle

@article{c6ae7f4524a643f78c5897dc27d51741,
title = "Decision support from local data: Creating adaptive order menus from past clinician behavior",
abstract = "Objective: Reducing care variability through guidelines has significantly benefited patients. Nonetheless, guideline-based Clinical Decision Support (CDS) systems are not widely implemented or used, are frequently out-of-date, and cannot address complex care for which guidelines do not exist. Here, we develop and evaluate a complementary approach - using Bayesian Network (BN) learning to generate adaptive, context-specific treatment menus based on local order-entry data. These menus can be used as a draft for expert review, in order to minimize development time for local decision support content. This is in keeping with the vision outlined in the US Health Information Technology Strategic Plan, which describes a healthcare system that learns from itself. Materials and methods: We used the Greedy Equivalence Search algorithm to learn four 50-node domain-specific BNs from 11,344 encounters: abdominal pain in the emergency department, inpatient pregnancy, hypertension in the Urgent Visit Clinic, and altered mental state in the intensive care unit. We developed a system to produce situation-specific, rank-ordered treatment menus from these networks. We evaluated this system with a hospital-simulation methodology and computed Area Under the Receiver-Operator Curve (AUC) and average menu position at time of selection. We also compared this system with a similar association-rule-mining approach. Results: A short order menu on average contained the next order (weighted average length 3.91-5.83 items). Overall predictive ability was good: average AUC above 0.9 for 25{\%} of order types and overall average AUC 714-844 (depending on domain). However, AUC had high variance (50-99). Higher AUC correlated with tighter clusters and more connections in the graphs, indicating importance of appropriate contextual data. Comparison with an Association Rule Mining approach showed similar performance for only the most common orders with dramatic divergence as orders are less frequent. Discussion and conclusion: This study demonstrates that local clinical knowledge can be extracted from treatment data for decision support. This approach is appealing because: it reflects local standards; it uses data already being captured; and it produces human-readable treatment-diagnosis networks that could be curated by a human expert to reduce workload in developing localized CDS content. The BN methodology captured transitive associations and co-varying relationships, which existing approaches do not. It also performs better as orders become less frequent and require more context. This system is a step forward in harnessing local, empirical data to enhance decision support.",
keywords = "Bayesian analysis, Clinical Decision Support, Data mining",
author = "Klann, {Jeffrey G.} and Peter Szolovits and Stephen Downs and Gunther Schadow",
year = "2014",
doi = "10.1016/j.jbi.2013.12.005",
language = "English",
volume = "48",
pages = "84--93",
journal = "Journal of Biomedical Informatics",
issn = "1532-0464",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Decision support from local data

T2 - Creating adaptive order menus from past clinician behavior

AU - Klann, Jeffrey G.

AU - Szolovits, Peter

AU - Downs, Stephen

AU - Schadow, Gunther

PY - 2014

Y1 - 2014

N2 - Objective: Reducing care variability through guidelines has significantly benefited patients. Nonetheless, guideline-based Clinical Decision Support (CDS) systems are not widely implemented or used, are frequently out-of-date, and cannot address complex care for which guidelines do not exist. Here, we develop and evaluate a complementary approach - using Bayesian Network (BN) learning to generate adaptive, context-specific treatment menus based on local order-entry data. These menus can be used as a draft for expert review, in order to minimize development time for local decision support content. This is in keeping with the vision outlined in the US Health Information Technology Strategic Plan, which describes a healthcare system that learns from itself. Materials and methods: We used the Greedy Equivalence Search algorithm to learn four 50-node domain-specific BNs from 11,344 encounters: abdominal pain in the emergency department, inpatient pregnancy, hypertension in the Urgent Visit Clinic, and altered mental state in the intensive care unit. We developed a system to produce situation-specific, rank-ordered treatment menus from these networks. We evaluated this system with a hospital-simulation methodology and computed Area Under the Receiver-Operator Curve (AUC) and average menu position at time of selection. We also compared this system with a similar association-rule-mining approach. Results: A short order menu on average contained the next order (weighted average length 3.91-5.83 items). Overall predictive ability was good: average AUC above 0.9 for 25% of order types and overall average AUC 714-844 (depending on domain). However, AUC had high variance (50-99). Higher AUC correlated with tighter clusters and more connections in the graphs, indicating importance of appropriate contextual data. Comparison with an Association Rule Mining approach showed similar performance for only the most common orders with dramatic divergence as orders are less frequent. Discussion and conclusion: This study demonstrates that local clinical knowledge can be extracted from treatment data for decision support. This approach is appealing because: it reflects local standards; it uses data already being captured; and it produces human-readable treatment-diagnosis networks that could be curated by a human expert to reduce workload in developing localized CDS content. The BN methodology captured transitive associations and co-varying relationships, which existing approaches do not. It also performs better as orders become less frequent and require more context. This system is a step forward in harnessing local, empirical data to enhance decision support.

AB - Objective: Reducing care variability through guidelines has significantly benefited patients. Nonetheless, guideline-based Clinical Decision Support (CDS) systems are not widely implemented or used, are frequently out-of-date, and cannot address complex care for which guidelines do not exist. Here, we develop and evaluate a complementary approach - using Bayesian Network (BN) learning to generate adaptive, context-specific treatment menus based on local order-entry data. These menus can be used as a draft for expert review, in order to minimize development time for local decision support content. This is in keeping with the vision outlined in the US Health Information Technology Strategic Plan, which describes a healthcare system that learns from itself. Materials and methods: We used the Greedy Equivalence Search algorithm to learn four 50-node domain-specific BNs from 11,344 encounters: abdominal pain in the emergency department, inpatient pregnancy, hypertension in the Urgent Visit Clinic, and altered mental state in the intensive care unit. We developed a system to produce situation-specific, rank-ordered treatment menus from these networks. We evaluated this system with a hospital-simulation methodology and computed Area Under the Receiver-Operator Curve (AUC) and average menu position at time of selection. We also compared this system with a similar association-rule-mining approach. Results: A short order menu on average contained the next order (weighted average length 3.91-5.83 items). Overall predictive ability was good: average AUC above 0.9 for 25% of order types and overall average AUC 714-844 (depending on domain). However, AUC had high variance (50-99). Higher AUC correlated with tighter clusters and more connections in the graphs, indicating importance of appropriate contextual data. Comparison with an Association Rule Mining approach showed similar performance for only the most common orders with dramatic divergence as orders are less frequent. Discussion and conclusion: This study demonstrates that local clinical knowledge can be extracted from treatment data for decision support. This approach is appealing because: it reflects local standards; it uses data already being captured; and it produces human-readable treatment-diagnosis networks that could be curated by a human expert to reduce workload in developing localized CDS content. The BN methodology captured transitive associations and co-varying relationships, which existing approaches do not. It also performs better as orders become less frequent and require more context. This system is a step forward in harnessing local, empirical data to enhance decision support.

KW - Bayesian analysis

KW - Clinical Decision Support

KW - Data mining

UR - http://www.scopus.com/inward/record.url?scp=84899489955&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899489955&partnerID=8YFLogxK

U2 - 10.1016/j.jbi.2013.12.005

DO - 10.1016/j.jbi.2013.12.005

M3 - Article

VL - 48

SP - 84

EP - 93

JO - Journal of Biomedical Informatics

JF - Journal of Biomedical Informatics

SN - 1532-0464

ER -