Are performance measures based on automated medical records valid for physician/practice profiling of asthma care?

Anne Fuhlbrigge, Vincent J. Carey, Jonathan A. Finkelstein, Paula Lozano, Thomas Inui, Kevin B. Weiss

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

BACKGROUND: The use of physician profiles in "pay for performance" initiatives depend on their reliability and accuracy. OBJECTIVES: To evaluate whether health care delivery units (practices) can be reliably differentiated using the Health Employers Data Information System (HEDIS) performance measure. RESEARCH DESIGN: Simulation was used to describe the relationship between practice size (number of children with persistent asthma) and precision of practice measures to estimate performance. SUBJECTS: Children enrolled in 1 of the 39 practice groups from 1 of 3 managed care organizations participating in the Pediatric Asthma Care Patient Outcomes Research Team (PAC PORT). MEASURES: The main outcome was reproducibility of 4 performance measures, the HEDIS measure and 3 additional measures available from automated claims data: the proportion of children with asthma related-hospitalization, emergency department visits and oral steroid dispensings for asthma. RESULTS: The ability to reproducibly rank a practice is dependent on the performance measure, practice size, and the reproducibility threshold chosen. Of measures evaluated, none achieved a reproducibility >85% for practice size of 50 or less. At a practice size of 100 subjects, the HEDIS measure reproducibly ranked practices 89% of the time, compared with 85% for emergency department visits and 83% for hospitalizations. Only at a practice size of 100 children with persistent asthma, was reproducibility of ranking greater than 85% with all performance measures evaluated. CONCLUSIONS: The reliability of ranking medical practices depends on practice size. Only at the level of the health care organization can asthma measures, available within claims data, be used to rank performance reliably.

Original languageEnglish
Pages (from-to)620-626
Number of pages7
JournalMedical Care
Volume46
Issue number6
DOIs
StatePublished - Jun 2008

Fingerprint

Information Systems
Medical Records
Asthma
physician
Physicians
performance
Hospital Emergency Service
Health
Hospitalization
employer
information system
Organizations
Incentive Reimbursement
Delivery of Health Care
hospitalization
Managed Care Programs
ranking
Health Status
health
health care

Keywords

  • Asthma
  • Health policy
  • HEDIS

ASJC Scopus subject areas

  • Nursing(all)
  • Public Health, Environmental and Occupational Health
  • Health(social science)
  • Health Professions(all)

Cite this

Are performance measures based on automated medical records valid for physician/practice profiling of asthma care? / Fuhlbrigge, Anne; Carey, Vincent J.; Finkelstein, Jonathan A.; Lozano, Paula; Inui, Thomas; Weiss, Kevin B.

In: Medical Care, Vol. 46, No. 6, 06.2008, p. 620-626.

Research output: Contribution to journalArticle

Fuhlbrigge, Anne ; Carey, Vincent J. ; Finkelstein, Jonathan A. ; Lozano, Paula ; Inui, Thomas ; Weiss, Kevin B. / Are performance measures based on automated medical records valid for physician/practice profiling of asthma care?. In: Medical Care. 2008 ; Vol. 46, No. 6. pp. 620-626.
@article{1f2193457c48461089f8b909dc570542,
title = "Are performance measures based on automated medical records valid for physician/practice profiling of asthma care?",
abstract = "BACKGROUND: The use of physician profiles in {"}pay for performance{"} initiatives depend on their reliability and accuracy. OBJECTIVES: To evaluate whether health care delivery units (practices) can be reliably differentiated using the Health Employers Data Information System (HEDIS) performance measure. RESEARCH DESIGN: Simulation was used to describe the relationship between practice size (number of children with persistent asthma) and precision of practice measures to estimate performance. SUBJECTS: Children enrolled in 1 of the 39 practice groups from 1 of 3 managed care organizations participating in the Pediatric Asthma Care Patient Outcomes Research Team (PAC PORT). MEASURES: The main outcome was reproducibility of 4 performance measures, the HEDIS measure and 3 additional measures available from automated claims data: the proportion of children with asthma related-hospitalization, emergency department visits and oral steroid dispensings for asthma. RESULTS: The ability to reproducibly rank a practice is dependent on the performance measure, practice size, and the reproducibility threshold chosen. Of measures evaluated, none achieved a reproducibility >85{\%} for practice size of 50 or less. At a practice size of 100 subjects, the HEDIS measure reproducibly ranked practices 89{\%} of the time, compared with 85{\%} for emergency department visits and 83{\%} for hospitalizations. Only at a practice size of 100 children with persistent asthma, was reproducibility of ranking greater than 85{\%} with all performance measures evaluated. CONCLUSIONS: The reliability of ranking medical practices depends on practice size. Only at the level of the health care organization can asthma measures, available within claims data, be used to rank performance reliably.",
keywords = "Asthma, Health policy, HEDIS",
author = "Anne Fuhlbrigge and Carey, {Vincent J.} and Finkelstein, {Jonathan A.} and Paula Lozano and Thomas Inui and Weiss, {Kevin B.}",
year = "2008",
month = "6",
doi = "10.1097/MLR.0b013e3181618ec9",
language = "English",
volume = "46",
pages = "620--626",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "6",

}

TY - JOUR

T1 - Are performance measures based on automated medical records valid for physician/practice profiling of asthma care?

AU - Fuhlbrigge, Anne

AU - Carey, Vincent J.

AU - Finkelstein, Jonathan A.

AU - Lozano, Paula

AU - Inui, Thomas

AU - Weiss, Kevin B.

PY - 2008/6

Y1 - 2008/6

N2 - BACKGROUND: The use of physician profiles in "pay for performance" initiatives depend on their reliability and accuracy. OBJECTIVES: To evaluate whether health care delivery units (practices) can be reliably differentiated using the Health Employers Data Information System (HEDIS) performance measure. RESEARCH DESIGN: Simulation was used to describe the relationship between practice size (number of children with persistent asthma) and precision of practice measures to estimate performance. SUBJECTS: Children enrolled in 1 of the 39 practice groups from 1 of 3 managed care organizations participating in the Pediatric Asthma Care Patient Outcomes Research Team (PAC PORT). MEASURES: The main outcome was reproducibility of 4 performance measures, the HEDIS measure and 3 additional measures available from automated claims data: the proportion of children with asthma related-hospitalization, emergency department visits and oral steroid dispensings for asthma. RESULTS: The ability to reproducibly rank a practice is dependent on the performance measure, practice size, and the reproducibility threshold chosen. Of measures evaluated, none achieved a reproducibility >85% for practice size of 50 or less. At a practice size of 100 subjects, the HEDIS measure reproducibly ranked practices 89% of the time, compared with 85% for emergency department visits and 83% for hospitalizations. Only at a practice size of 100 children with persistent asthma, was reproducibility of ranking greater than 85% with all performance measures evaluated. CONCLUSIONS: The reliability of ranking medical practices depends on practice size. Only at the level of the health care organization can asthma measures, available within claims data, be used to rank performance reliably.

AB - BACKGROUND: The use of physician profiles in "pay for performance" initiatives depend on their reliability and accuracy. OBJECTIVES: To evaluate whether health care delivery units (practices) can be reliably differentiated using the Health Employers Data Information System (HEDIS) performance measure. RESEARCH DESIGN: Simulation was used to describe the relationship between practice size (number of children with persistent asthma) and precision of practice measures to estimate performance. SUBJECTS: Children enrolled in 1 of the 39 practice groups from 1 of 3 managed care organizations participating in the Pediatric Asthma Care Patient Outcomes Research Team (PAC PORT). MEASURES: The main outcome was reproducibility of 4 performance measures, the HEDIS measure and 3 additional measures available from automated claims data: the proportion of children with asthma related-hospitalization, emergency department visits and oral steroid dispensings for asthma. RESULTS: The ability to reproducibly rank a practice is dependent on the performance measure, practice size, and the reproducibility threshold chosen. Of measures evaluated, none achieved a reproducibility >85% for practice size of 50 or less. At a practice size of 100 subjects, the HEDIS measure reproducibly ranked practices 89% of the time, compared with 85% for emergency department visits and 83% for hospitalizations. Only at a practice size of 100 children with persistent asthma, was reproducibility of ranking greater than 85% with all performance measures evaluated. CONCLUSIONS: The reliability of ranking medical practices depends on practice size. Only at the level of the health care organization can asthma measures, available within claims data, be used to rank performance reliably.

KW - Asthma

KW - Health policy

KW - HEDIS

UR - http://www.scopus.com/inward/record.url?scp=44449171476&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=44449171476&partnerID=8YFLogxK

U2 - 10.1097/MLR.0b013e3181618ec9

DO - 10.1097/MLR.0b013e3181618ec9

M3 - Article

C2 - 18520317

AN - SCOPUS:44449171476

VL - 46

SP - 620

EP - 626

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 6

ER -