Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System

Michael S. Phipps, Jeff Fahner, Danielle Sager, Jessica Coffing, Bailey Maryfield, Linda Williams

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Background: The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality. Objective: To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data. Design: Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review. Main Measures: We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed. Key Results: In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 %–99.7 % (denominators) and 87.7 %–97.9 % (numerators). PPVs tended to be higher (range 96.8 %–100 % in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73–0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes. Conclusions: Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 % match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.

Original languageEnglish (US)
Pages (from-to)46-52
Number of pages7
JournalJournal of General Internal Medicine
Volume31
DOIs
StatePublished - Apr 1 2016

Fingerprint

Electronic Health Records
Stroke
Information Storage and Retrieval
Hospice Care
Equipment and Supplies
Hospices
Cross-Sectional Studies
Sensitivity and Specificity

Keywords

  • electronic health records
  • meaningful use
  • process assessment
  • quality assessment
  • stroke

ASJC Scopus subject areas

  • Internal Medicine

Cite this

Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System. / Phipps, Michael S.; Fahner, Jeff; Sager, Danielle; Coffing, Jessica; Maryfield, Bailey; Williams, Linda.

In: Journal of General Internal Medicine, Vol. 31, 01.04.2016, p. 46-52.

Research output: Contribution to journalArticle

Phipps, Michael S. ; Fahner, Jeff ; Sager, Danielle ; Coffing, Jessica ; Maryfield, Bailey ; Williams, Linda. / Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System. In: Journal of General Internal Medicine. 2016 ; Vol. 31. pp. 46-52.
@article{02def78e339f4740968c8aef02bde14c,
title = "Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System",
abstract = "Background: The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality. Objective: To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data. Design: Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review. Main Measures: We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed. Key Results: In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 {\%}–99.7 {\%} (denominators) and 87.7 {\%}–97.9 {\%} (numerators). PPVs tended to be higher (range 96.8 {\%}–100 {\%} in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73–0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes. Conclusions: Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 {\%} match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.",
keywords = "electronic health records, meaningful use, process assessment, quality assessment, stroke",
author = "Phipps, {Michael S.} and Jeff Fahner and Danielle Sager and Jessica Coffing and Bailey Maryfield and Linda Williams",
year = "2016",
month = "4",
day = "1",
doi = "10.1007/s11606-015-3562-5",
language = "English (US)",
volume = "31",
pages = "46--52",
journal = "Journal of General Internal Medicine",
issn = "0884-8734",
publisher = "Springer New York",

}

TY - JOUR

T1 - Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System

AU - Phipps, Michael S.

AU - Fahner, Jeff

AU - Sager, Danielle

AU - Coffing, Jessica

AU - Maryfield, Bailey

AU - Williams, Linda

PY - 2016/4/1

Y1 - 2016/4/1

N2 - Background: The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality. Objective: To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data. Design: Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review. Main Measures: We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed. Key Results: In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 %–99.7 % (denominators) and 87.7 %–97.9 % (numerators). PPVs tended to be higher (range 96.8 %–100 % in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73–0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes. Conclusions: Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 % match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.

AB - Background: The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality. Objective: To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data. Design: Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review. Main Measures: We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed. Key Results: In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 %–99.7 % (denominators) and 87.7 %–97.9 % (numerators). PPVs tended to be higher (range 96.8 %–100 % in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73–0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes. Conclusions: Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 % match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.

KW - electronic health records

KW - meaningful use

KW - process assessment

KW - quality assessment

KW - stroke

UR - http://www.scopus.com/inward/record.url?scp=84960110316&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960110316&partnerID=8YFLogxK

U2 - 10.1007/s11606-015-3562-5

DO - 10.1007/s11606-015-3562-5

M3 - Article

VL - 31

SP - 46

EP - 52

JO - Journal of General Internal Medicine

JF - Journal of General Internal Medicine

SN - 0884-8734

ER -