Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression

Patrick Monahan, Colleen A. Mchorney, Timothy E. Stump, Anthony J. Perkins

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

Previous methodological and applied studies that used binary logistic regression (LR) for detection of differential item functioning (DIF) in dichotomously scored items either did not report an effect size or did not employ several useful measures of DIF magnitude derived from the LR model. Equations are provided for these effect size indices. Using two large data sets, the authors demonstrate the usefulness of these effect sizes for judging practical importance: the LR adjusted odds ratio and its conversions to the delta metric, the Educational Testing Service (ETS) classification system, and the p metric; the LR model-based standardization indices, using various weights for averaging stratum-specific differences in fitted probabilities; and a p metric classification system. Pros and cons of these effect sizes are discussed. Recommendations are offered. These LR effect sizes will be valuable to practitioners, particularly for preventing flagging of statistically significant hut practically unimportant DIF in large samples.

Original languageEnglish
Pages (from-to)92-109
Number of pages18
JournalJournal of Educational and Behavioral Statistics
Volume32
Issue number1
DOIs
StatePublished - Mar 2007

Fingerprint

logistics
regression
social stratum

Keywords

  • Differential item functioning
  • Effect sizes
  • Logistic regression

ASJC Scopus subject areas

  • Education
  • Social Sciences (miscellaneous)

Cite this

Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. / Monahan, Patrick; Mchorney, Colleen A.; Stump, Timothy E.; Perkins, Anthony J.

In: Journal of Educational and Behavioral Statistics, Vol. 32, No. 1, 03.2007, p. 92-109.

Research output: Contribution to journalArticle

@article{ec6e52053fdd4b82bddce69ee234262b,
title = "Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression",
abstract = "Previous methodological and applied studies that used binary logistic regression (LR) for detection of differential item functioning (DIF) in dichotomously scored items either did not report an effect size or did not employ several useful measures of DIF magnitude derived from the LR model. Equations are provided for these effect size indices. Using two large data sets, the authors demonstrate the usefulness of these effect sizes for judging practical importance: the LR adjusted odds ratio and its conversions to the delta metric, the Educational Testing Service (ETS) classification system, and the p metric; the LR model-based standardization indices, using various weights for averaging stratum-specific differences in fitted probabilities; and a p metric classification system. Pros and cons of these effect sizes are discussed. Recommendations are offered. These LR effect sizes will be valuable to practitioners, particularly for preventing flagging of statistically significant hut practically unimportant DIF in large samples.",
keywords = "Differential item functioning, Effect sizes, Logistic regression",
author = "Patrick Monahan and Mchorney, {Colleen A.} and Stump, {Timothy E.} and Perkins, {Anthony J.}",
year = "2007",
month = "3",
doi = "10.3102/1076998606298035",
language = "English",
volume = "32",
pages = "92--109",
journal = "Journal of Educational and Behavioral Statistics",
issn = "1076-9986",
publisher = "SAGE Publications Inc.",
number = "1",

}

TY - JOUR

T1 - Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression

AU - Monahan, Patrick

AU - Mchorney, Colleen A.

AU - Stump, Timothy E.

AU - Perkins, Anthony J.

PY - 2007/3

Y1 - 2007/3

N2 - Previous methodological and applied studies that used binary logistic regression (LR) for detection of differential item functioning (DIF) in dichotomously scored items either did not report an effect size or did not employ several useful measures of DIF magnitude derived from the LR model. Equations are provided for these effect size indices. Using two large data sets, the authors demonstrate the usefulness of these effect sizes for judging practical importance: the LR adjusted odds ratio and its conversions to the delta metric, the Educational Testing Service (ETS) classification system, and the p metric; the LR model-based standardization indices, using various weights for averaging stratum-specific differences in fitted probabilities; and a p metric classification system. Pros and cons of these effect sizes are discussed. Recommendations are offered. These LR effect sizes will be valuable to practitioners, particularly for preventing flagging of statistically significant hut practically unimportant DIF in large samples.

AB - Previous methodological and applied studies that used binary logistic regression (LR) for detection of differential item functioning (DIF) in dichotomously scored items either did not report an effect size or did not employ several useful measures of DIF magnitude derived from the LR model. Equations are provided for these effect size indices. Using two large data sets, the authors demonstrate the usefulness of these effect sizes for judging practical importance: the LR adjusted odds ratio and its conversions to the delta metric, the Educational Testing Service (ETS) classification system, and the p metric; the LR model-based standardization indices, using various weights for averaging stratum-specific differences in fitted probabilities; and a p metric classification system. Pros and cons of these effect sizes are discussed. Recommendations are offered. These LR effect sizes will be valuable to practitioners, particularly for preventing flagging of statistically significant hut practically unimportant DIF in large samples.

KW - Differential item functioning

KW - Effect sizes

KW - Logistic regression

UR - http://www.scopus.com/inward/record.url?scp=34147223247&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34147223247&partnerID=8YFLogxK

U2 - 10.3102/1076998606298035

DO - 10.3102/1076998606298035

M3 - Article

AN - SCOPUS:34147223247

VL - 32

SP - 92

EP - 109

JO - Journal of Educational and Behavioral Statistics

JF - Journal of Educational and Behavioral Statistics

SN - 1076-9986

IS - 1

ER -