Preliminary Factor Analyses Raise Concerns about Script Concordance Test Utility

Exploring SCT constructs

Adam B. Wilson, Gary R. Pike, Aloysius Humbert

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Purpose: Script concordance tests (SCTs) are assessments purported to measure one’s ability to interpret clinical data, a key component of the clinical diagnostic reasoning process. The aim of this research was to directly examine the construct validity/dimensionality of SCTs using factor analysis. Method: At Indiana University School of Medicine, a confirmatory factor analysis was conducted on three SCT datasets to assess whether SCT scores represented a single dimension of clinical reasoning. Exploratory factor analyses were also performed to determine if the SCTs represented multiple dimensions and to examine the effects of various SCT scoring methods on test dimensionality. Results: Results of the confirmatory factor analysis revealed that the SCTs did not have a unidimensional factor structure. Subsequent exploratory factor analyses of six different scoring methods confirmed these results. Factor loadings were weak and the factors explained only a small portion of the total variance in SCT scores. Conclusions: The results of this study challenge the assertion that SCTs measure one dimension of clinical reasoning. Because the outcomes of this study raise questions about the latent constructs SCTs measure, recommendations concerning the utility of SCT scores are also presented.

Original languageEnglish (US)
Pages (from-to)51-58
Number of pages8
JournalMedical Science Educator
Volume24
Issue number1
DOIs
StatePublished - Mar 1 2014

Fingerprint

Statistical Factor Analysis
Research Design
factor analysis
Medicine
Outcome Assessment (Health Care)
Research
construct validity
diagnostic
medicine

Keywords

  • Clinical reasoning
  • Factor analysis
  • Scoring
  • Script concordance test

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Education

Cite this

Preliminary Factor Analyses Raise Concerns about Script Concordance Test Utility : Exploring SCT constructs. / Wilson, Adam B.; Pike, Gary R.; Humbert, Aloysius.

In: Medical Science Educator, Vol. 24, No. 1, 01.03.2014, p. 51-58.

Research output: Contribution to journalArticle

@article{6f3f6c29266f40419e53cbc6dac49c02,
title = "Preliminary Factor Analyses Raise Concerns about Script Concordance Test Utility: Exploring SCT constructs",
abstract = "Purpose: Script concordance tests (SCTs) are assessments purported to measure one’s ability to interpret clinical data, a key component of the clinical diagnostic reasoning process. The aim of this research was to directly examine the construct validity/dimensionality of SCTs using factor analysis. Method: At Indiana University School of Medicine, a confirmatory factor analysis was conducted on three SCT datasets to assess whether SCT scores represented a single dimension of clinical reasoning. Exploratory factor analyses were also performed to determine if the SCTs represented multiple dimensions and to examine the effects of various SCT scoring methods on test dimensionality. Results: Results of the confirmatory factor analysis revealed that the SCTs did not have a unidimensional factor structure. Subsequent exploratory factor analyses of six different scoring methods confirmed these results. Factor loadings were weak and the factors explained only a small portion of the total variance in SCT scores. Conclusions: The results of this study challenge the assertion that SCTs measure one dimension of clinical reasoning. Because the outcomes of this study raise questions about the latent constructs SCTs measure, recommendations concerning the utility of SCT scores are also presented.",
keywords = "Clinical reasoning, Factor analysis, Scoring, Script concordance test",
author = "Wilson, {Adam B.} and Pike, {Gary R.} and Aloysius Humbert",
year = "2014",
month = "3",
day = "1",
doi = "10.1007/s40670-014-0013-6",
language = "English (US)",
volume = "24",
pages = "51--58",
journal = "Medical Science Educator",
issn = "2156-8650",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - Preliminary Factor Analyses Raise Concerns about Script Concordance Test Utility

T2 - Exploring SCT constructs

AU - Wilson, Adam B.

AU - Pike, Gary R.

AU - Humbert, Aloysius

PY - 2014/3/1

Y1 - 2014/3/1

N2 - Purpose: Script concordance tests (SCTs) are assessments purported to measure one’s ability to interpret clinical data, a key component of the clinical diagnostic reasoning process. The aim of this research was to directly examine the construct validity/dimensionality of SCTs using factor analysis. Method: At Indiana University School of Medicine, a confirmatory factor analysis was conducted on three SCT datasets to assess whether SCT scores represented a single dimension of clinical reasoning. Exploratory factor analyses were also performed to determine if the SCTs represented multiple dimensions and to examine the effects of various SCT scoring methods on test dimensionality. Results: Results of the confirmatory factor analysis revealed that the SCTs did not have a unidimensional factor structure. Subsequent exploratory factor analyses of six different scoring methods confirmed these results. Factor loadings were weak and the factors explained only a small portion of the total variance in SCT scores. Conclusions: The results of this study challenge the assertion that SCTs measure one dimension of clinical reasoning. Because the outcomes of this study raise questions about the latent constructs SCTs measure, recommendations concerning the utility of SCT scores are also presented.

AB - Purpose: Script concordance tests (SCTs) are assessments purported to measure one’s ability to interpret clinical data, a key component of the clinical diagnostic reasoning process. The aim of this research was to directly examine the construct validity/dimensionality of SCTs using factor analysis. Method: At Indiana University School of Medicine, a confirmatory factor analysis was conducted on three SCT datasets to assess whether SCT scores represented a single dimension of clinical reasoning. Exploratory factor analyses were also performed to determine if the SCTs represented multiple dimensions and to examine the effects of various SCT scoring methods on test dimensionality. Results: Results of the confirmatory factor analysis revealed that the SCTs did not have a unidimensional factor structure. Subsequent exploratory factor analyses of six different scoring methods confirmed these results. Factor loadings were weak and the factors explained only a small portion of the total variance in SCT scores. Conclusions: The results of this study challenge the assertion that SCTs measure one dimension of clinical reasoning. Because the outcomes of this study raise questions about the latent constructs SCTs measure, recommendations concerning the utility of SCT scores are also presented.

KW - Clinical reasoning

KW - Factor analysis

KW - Scoring

KW - Script concordance test

UR - http://www.scopus.com/inward/record.url?scp=84928275803&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928275803&partnerID=8YFLogxK

U2 - 10.1007/s40670-014-0013-6

DO - 10.1007/s40670-014-0013-6

M3 - Article

VL - 24

SP - 51

EP - 58

JO - Medical Science Educator

JF - Medical Science Educator

SN - 2156-8650

IS - 1

ER -