Bias of exploratory and cross-validated DETECT index under unidimensionality

Patrick Monahan, Timothy E. Stump, Holmes Finch, Ronald K. Hambleton

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

DETECT is a nonparametric ''full'' dimensionality assessment procedure that clusters dichotomously scored items into dimensions and provides a DETECT index of magnitude of multidimensionality. Four factors (test length, sample size, item response theory [IRT] model, and DETECT index) were manipulated in a Monte Carlo study of bias, standard error, and root mean square error (RMSE) under the condition of unidimensionality. Bias, standard error, and RMSE of both DETECT indices increased as test length and sample size decreased. Results suggest that the cross-validated index should always be preferred over the exploratory index, even for 100 examinees and five items. Bias, standard error, and RMSE may be problematic for both indices under certain conditions of small samples or short tests. A Monte Carlo procedure could be built into DETECT to estimate and adjust for potential bias.

Original languageEnglish
Pages (from-to)483-503
Number of pages21
JournalApplied Psychological Measurement
Volume31
Issue number6
DOIs
StatePublished - Nov 2007

Fingerprint

Sample Size
trend
model theory

Keywords

  • Bias
  • DETECT
  • Dimensionality
  • DIMTEST
  • Monte Carlo

ASJC Scopus subject areas

  • Psychology(all)
  • Psychology (miscellaneous)
  • Social Sciences (miscellaneous)

Cite this

Bias of exploratory and cross-validated DETECT index under unidimensionality. / Monahan, Patrick; Stump, Timothy E.; Finch, Holmes; Hambleton, Ronald K.

In: Applied Psychological Measurement, Vol. 31, No. 6, 11.2007, p. 483-503.

Research output: Contribution to journalArticle

Monahan, Patrick ; Stump, Timothy E. ; Finch, Holmes ; Hambleton, Ronald K. / Bias of exploratory and cross-validated DETECT index under unidimensionality. In: Applied Psychological Measurement. 2007 ; Vol. 31, No. 6. pp. 483-503.
@article{bda6a4a491754f53a8d3c27448d6deb9,
title = "Bias of exploratory and cross-validated DETECT index under unidimensionality",
abstract = "DETECT is a nonparametric ''full'' dimensionality assessment procedure that clusters dichotomously scored items into dimensions and provides a DETECT index of magnitude of multidimensionality. Four factors (test length, sample size, item response theory [IRT] model, and DETECT index) were manipulated in a Monte Carlo study of bias, standard error, and root mean square error (RMSE) under the condition of unidimensionality. Bias, standard error, and RMSE of both DETECT indices increased as test length and sample size decreased. Results suggest that the cross-validated index should always be preferred over the exploratory index, even for 100 examinees and five items. Bias, standard error, and RMSE may be problematic for both indices under certain conditions of small samples or short tests. A Monte Carlo procedure could be built into DETECT to estimate and adjust for potential bias.",
keywords = "Bias, DETECT, Dimensionality, DIMTEST, Monte Carlo",
author = "Patrick Monahan and Stump, {Timothy E.} and Holmes Finch and Hambleton, {Ronald K.}",
year = "2007",
month = "11",
doi = "10.1177/0146621606292216",
language = "English",
volume = "31",
pages = "483--503",
journal = "Applied Psychological Measurement",
issn = "0146-6216",
publisher = "SAGE Publications Inc.",
number = "6",

}

TY - JOUR

T1 - Bias of exploratory and cross-validated DETECT index under unidimensionality

AU - Monahan, Patrick

AU - Stump, Timothy E.

AU - Finch, Holmes

AU - Hambleton, Ronald K.

PY - 2007/11

Y1 - 2007/11

N2 - DETECT is a nonparametric ''full'' dimensionality assessment procedure that clusters dichotomously scored items into dimensions and provides a DETECT index of magnitude of multidimensionality. Four factors (test length, sample size, item response theory [IRT] model, and DETECT index) were manipulated in a Monte Carlo study of bias, standard error, and root mean square error (RMSE) under the condition of unidimensionality. Bias, standard error, and RMSE of both DETECT indices increased as test length and sample size decreased. Results suggest that the cross-validated index should always be preferred over the exploratory index, even for 100 examinees and five items. Bias, standard error, and RMSE may be problematic for both indices under certain conditions of small samples or short tests. A Monte Carlo procedure could be built into DETECT to estimate and adjust for potential bias.

AB - DETECT is a nonparametric ''full'' dimensionality assessment procedure that clusters dichotomously scored items into dimensions and provides a DETECT index of magnitude of multidimensionality. Four factors (test length, sample size, item response theory [IRT] model, and DETECT index) were manipulated in a Monte Carlo study of bias, standard error, and root mean square error (RMSE) under the condition of unidimensionality. Bias, standard error, and RMSE of both DETECT indices increased as test length and sample size decreased. Results suggest that the cross-validated index should always be preferred over the exploratory index, even for 100 examinees and five items. Bias, standard error, and RMSE may be problematic for both indices under certain conditions of small samples or short tests. A Monte Carlo procedure could be built into DETECT to estimate and adjust for potential bias.

KW - Bias

KW - DETECT

KW - Dimensionality

KW - DIMTEST

KW - Monte Carlo

UR - http://www.scopus.com/inward/record.url?scp=35348885610&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=35348885610&partnerID=8YFLogxK

U2 - 10.1177/0146621606292216

DO - 10.1177/0146621606292216

M3 - Article

VL - 31

SP - 483

EP - 503

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

SN - 0146-6216

IS - 6

ER -