Improving quality assessment through multilevel modeling

The case of Nursing Home Compare

Gregory Arling, Teresa Lewis, Robert L. Kane, Christine Mueller, Shannon Flood

Research output: Contribution to journalArticle

41 Citations (Scopus)

Abstract

Objective. To demonstrate how multilevel modeling and empirical Bayes (EB) estimates can improve Medicare's Nursing Home Compare quality measures (QMs). Data Sources/Study Setting. Secondary data from July 1 to September 30, 2004. Facility-level QMs were estimated from minimum data set (MDS) assessments for approximately 31,000 Minnesota nursing home residents in 393 facilities. Study Design. Prevalence and incidence rates for 12 nursing facility QMs (e.g., use of physical restraints, pressure sores, and weight loss) were estimated with EB methods and risk adjustment using a hierarchical general linear model. Three sets of rates were developed: Nursing Home Compare's current method, unadjusted EB rates, and risk-adjusted EB rates. Bayesian 90 percent credibility intervals (CIs) were constructed around EB rates, and these were used to flag facilities for potential quality of care problems. Data Collection/Extraction Methods. MDS assessments were performed by nursing facility staff, transmitted electronically to the Minnesota Department of Health, and provided to the investigators. Principal Findings. Facility rates and rankings for the 12 QMs differed substantially using the multilevel models compared with current methods. The EB estimated rates shrank considerably toward the population mean. Risk adjustment had a large impact on some QM rates and a more modest impact on others. When EB CIs were used to flag problem facilities, there was wide variation across QMs in the percentage of facilities flagged. Conclusions. Multilevel modeling should be applied to Nursing Home Compare and more widely in other health care quality assessment systems.

Original languageEnglish
Pages (from-to)1177-1199
Number of pages23
JournalHealth Services Research
Volume42
Issue number3 I
DOIs
StatePublished - Jun 2007

Fingerprint

nursing home
Nursing Homes
Risk Adjustment
Health Care Quality Assurance
Physical Restraint
risk adjustment
empirical method
Pressure Ulcer
Quality of Health Care
Information Storage and Retrieval
Nursing Staff
Medicare
credibility
Weight Loss
Linear Models
Nursing
nursing
Research Personnel
Incidence
Health

Keywords

  • Multilevel models
  • Nursing home
  • Quality
  • Risk adjustment

ASJC Scopus subject areas

  • Nursing(all)
  • Health(social science)
  • Health Professions(all)
  • Health Policy

Cite this

Improving quality assessment through multilevel modeling : The case of Nursing Home Compare. / Arling, Gregory; Lewis, Teresa; Kane, Robert L.; Mueller, Christine; Flood, Shannon.

In: Health Services Research, Vol. 42, No. 3 I, 06.2007, p. 1177-1199.

Research output: Contribution to journalArticle

Arling, Gregory ; Lewis, Teresa ; Kane, Robert L. ; Mueller, Christine ; Flood, Shannon. / Improving quality assessment through multilevel modeling : The case of Nursing Home Compare. In: Health Services Research. 2007 ; Vol. 42, No. 3 I. pp. 1177-1199.
@article{13007bbd0eef4464bfe40796f056e7f1,
title = "Improving quality assessment through multilevel modeling: The case of Nursing Home Compare",
abstract = "Objective. To demonstrate how multilevel modeling and empirical Bayes (EB) estimates can improve Medicare's Nursing Home Compare quality measures (QMs). Data Sources/Study Setting. Secondary data from July 1 to September 30, 2004. Facility-level QMs were estimated from minimum data set (MDS) assessments for approximately 31,000 Minnesota nursing home residents in 393 facilities. Study Design. Prevalence and incidence rates for 12 nursing facility QMs (e.g., use of physical restraints, pressure sores, and weight loss) were estimated with EB methods and risk adjustment using a hierarchical general linear model. Three sets of rates were developed: Nursing Home Compare's current method, unadjusted EB rates, and risk-adjusted EB rates. Bayesian 90 percent credibility intervals (CIs) were constructed around EB rates, and these were used to flag facilities for potential quality of care problems. Data Collection/Extraction Methods. MDS assessments were performed by nursing facility staff, transmitted electronically to the Minnesota Department of Health, and provided to the investigators. Principal Findings. Facility rates and rankings for the 12 QMs differed substantially using the multilevel models compared with current methods. The EB estimated rates shrank considerably toward the population mean. Risk adjustment had a large impact on some QM rates and a more modest impact on others. When EB CIs were used to flag problem facilities, there was wide variation across QMs in the percentage of facilities flagged. Conclusions. Multilevel modeling should be applied to Nursing Home Compare and more widely in other health care quality assessment systems.",
keywords = "Multilevel models, Nursing home, Quality, Risk adjustment",
author = "Gregory Arling and Teresa Lewis and Kane, {Robert L.} and Christine Mueller and Shannon Flood",
year = "2007",
month = "6",
doi = "10.1111/j.1475-6773.2006.00647.x",
language = "English",
volume = "42",
pages = "1177--1199",
journal = "Health Services Research",
issn = "0017-9124",
publisher = "Wiley-Blackwell",
number = "3 I",

}

TY - JOUR

T1 - Improving quality assessment through multilevel modeling

T2 - The case of Nursing Home Compare

AU - Arling, Gregory

AU - Lewis, Teresa

AU - Kane, Robert L.

AU - Mueller, Christine

AU - Flood, Shannon

PY - 2007/6

Y1 - 2007/6

N2 - Objective. To demonstrate how multilevel modeling and empirical Bayes (EB) estimates can improve Medicare's Nursing Home Compare quality measures (QMs). Data Sources/Study Setting. Secondary data from July 1 to September 30, 2004. Facility-level QMs were estimated from minimum data set (MDS) assessments for approximately 31,000 Minnesota nursing home residents in 393 facilities. Study Design. Prevalence and incidence rates for 12 nursing facility QMs (e.g., use of physical restraints, pressure sores, and weight loss) were estimated with EB methods and risk adjustment using a hierarchical general linear model. Three sets of rates were developed: Nursing Home Compare's current method, unadjusted EB rates, and risk-adjusted EB rates. Bayesian 90 percent credibility intervals (CIs) were constructed around EB rates, and these were used to flag facilities for potential quality of care problems. Data Collection/Extraction Methods. MDS assessments were performed by nursing facility staff, transmitted electronically to the Minnesota Department of Health, and provided to the investigators. Principal Findings. Facility rates and rankings for the 12 QMs differed substantially using the multilevel models compared with current methods. The EB estimated rates shrank considerably toward the population mean. Risk adjustment had a large impact on some QM rates and a more modest impact on others. When EB CIs were used to flag problem facilities, there was wide variation across QMs in the percentage of facilities flagged. Conclusions. Multilevel modeling should be applied to Nursing Home Compare and more widely in other health care quality assessment systems.

AB - Objective. To demonstrate how multilevel modeling and empirical Bayes (EB) estimates can improve Medicare's Nursing Home Compare quality measures (QMs). Data Sources/Study Setting. Secondary data from July 1 to September 30, 2004. Facility-level QMs were estimated from minimum data set (MDS) assessments for approximately 31,000 Minnesota nursing home residents in 393 facilities. Study Design. Prevalence and incidence rates for 12 nursing facility QMs (e.g., use of physical restraints, pressure sores, and weight loss) were estimated with EB methods and risk adjustment using a hierarchical general linear model. Three sets of rates were developed: Nursing Home Compare's current method, unadjusted EB rates, and risk-adjusted EB rates. Bayesian 90 percent credibility intervals (CIs) were constructed around EB rates, and these were used to flag facilities for potential quality of care problems. Data Collection/Extraction Methods. MDS assessments were performed by nursing facility staff, transmitted electronically to the Minnesota Department of Health, and provided to the investigators. Principal Findings. Facility rates and rankings for the 12 QMs differed substantially using the multilevel models compared with current methods. The EB estimated rates shrank considerably toward the population mean. Risk adjustment had a large impact on some QM rates and a more modest impact on others. When EB CIs were used to flag problem facilities, there was wide variation across QMs in the percentage of facilities flagged. Conclusions. Multilevel modeling should be applied to Nursing Home Compare and more widely in other health care quality assessment systems.

KW - Multilevel models

KW - Nursing home

KW - Quality

KW - Risk adjustment

UR - http://www.scopus.com/inward/record.url?scp=34247897091&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247897091&partnerID=8YFLogxK

U2 - 10.1111/j.1475-6773.2006.00647.x

DO - 10.1111/j.1475-6773.2006.00647.x

M3 - Article

VL - 42

SP - 1177

EP - 1199

JO - Health Services Research

JF - Health Services Research

SN - 0017-9124

IS - 3 I

ER -