A pilot experience with competency-based clinical skills assessment in a surgical clerkship

Gary Dunnington, Kelly Wright, Kaaren Hoffman

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

This investigation examines a competencybased clinical skills assessment program for surgical clerks using checklists and rating forms for precise measurement of physical exam (PE) skills, physician-patient interaction (PPI) skills, and patient write-up (PW) skills. Analysis of variance demonstrated improvement in PW skills across the academic year when measured by the rating instrument, but this improvement was not detected on traditional subjective rating forms (SRF). PPI skills improved between first rotations across 2 academic years with the addition of orientation to expectations (mean, 79% versus 92%, P = 0.000). Poor correlation was noted between the National Board of Medical Examiners Surgery Subtest scores and PE skills (r = .19), PW skills (r = .20), and PPI skills (r = .15). While the overall ratings given by faculty on SRF correlated with the SRF ratings of PE skills (r = .77) and PPI skills (r = .58), these same faculty ratings correlated poorly with these skills as assessed by checklist (r = .16 and r = .14, respectively). This pilot experience demonstrates that PE skills, PW skills, and PPI skills (1) improve only with orientation to expectations and feedback, (2) correlate poorly with fund of knowledge assessment, and (3) are best assessed with precise measurement (eg, checklist, direct observation), which avoids the halo effect of overall evaluation that occurs with subjective rating forms.

Original languageEnglish (US)
Pages (from-to)604-607
Number of pages4
JournalAmerican Journal of Surgery
Volume167
Issue number6
DOIs
StatePublished - 1994
Externally publishedYes

Fingerprint

Clinical Competence
Physicians
Checklist
Epidemiologic Effect Modifiers
Coroners and Medical Examiners
Financial Management
Analysis of Variance
Observation

ASJC Scopus subject areas

  • Surgery

Cite this

A pilot experience with competency-based clinical skills assessment in a surgical clerkship. / Dunnington, Gary; Wright, Kelly; Hoffman, Kaaren.

In: American Journal of Surgery, Vol. 167, No. 6, 1994, p. 604-607.

Research output: Contribution to journalArticle

@article{8245dc7d1ff84929a33d5483f02ed183,
title = "A pilot experience with competency-based clinical skills assessment in a surgical clerkship",
abstract = "This investigation examines a competencybased clinical skills assessment program for surgical clerks using checklists and rating forms for precise measurement of physical exam (PE) skills, physician-patient interaction (PPI) skills, and patient write-up (PW) skills. Analysis of variance demonstrated improvement in PW skills across the academic year when measured by the rating instrument, but this improvement was not detected on traditional subjective rating forms (SRF). PPI skills improved between first rotations across 2 academic years with the addition of orientation to expectations (mean, 79{\%} versus 92{\%}, P = 0.000). Poor correlation was noted between the National Board of Medical Examiners Surgery Subtest scores and PE skills (r = .19), PW skills (r = .20), and PPI skills (r = .15). While the overall ratings given by faculty on SRF correlated with the SRF ratings of PE skills (r = .77) and PPI skills (r = .58), these same faculty ratings correlated poorly with these skills as assessed by checklist (r = .16 and r = .14, respectively). This pilot experience demonstrates that PE skills, PW skills, and PPI skills (1) improve only with orientation to expectations and feedback, (2) correlate poorly with fund of knowledge assessment, and (3) are best assessed with precise measurement (eg, checklist, direct observation), which avoids the halo effect of overall evaluation that occurs with subjective rating forms.",
author = "Gary Dunnington and Kelly Wright and Kaaren Hoffman",
year = "1994",
doi = "10.1016/0002-9610(94)90107-4",
language = "English (US)",
volume = "167",
pages = "604--607",
journal = "American Journal of Surgery",
issn = "0002-9610",
publisher = "Elsevier Inc.",
number = "6",

}

TY - JOUR

T1 - A pilot experience with competency-based clinical skills assessment in a surgical clerkship

AU - Dunnington, Gary

AU - Wright, Kelly

AU - Hoffman, Kaaren

PY - 1994

Y1 - 1994

N2 - This investigation examines a competencybased clinical skills assessment program for surgical clerks using checklists and rating forms for precise measurement of physical exam (PE) skills, physician-patient interaction (PPI) skills, and patient write-up (PW) skills. Analysis of variance demonstrated improvement in PW skills across the academic year when measured by the rating instrument, but this improvement was not detected on traditional subjective rating forms (SRF). PPI skills improved between first rotations across 2 academic years with the addition of orientation to expectations (mean, 79% versus 92%, P = 0.000). Poor correlation was noted between the National Board of Medical Examiners Surgery Subtest scores and PE skills (r = .19), PW skills (r = .20), and PPI skills (r = .15). While the overall ratings given by faculty on SRF correlated with the SRF ratings of PE skills (r = .77) and PPI skills (r = .58), these same faculty ratings correlated poorly with these skills as assessed by checklist (r = .16 and r = .14, respectively). This pilot experience demonstrates that PE skills, PW skills, and PPI skills (1) improve only with orientation to expectations and feedback, (2) correlate poorly with fund of knowledge assessment, and (3) are best assessed with precise measurement (eg, checklist, direct observation), which avoids the halo effect of overall evaluation that occurs with subjective rating forms.

AB - This investigation examines a competencybased clinical skills assessment program for surgical clerks using checklists and rating forms for precise measurement of physical exam (PE) skills, physician-patient interaction (PPI) skills, and patient write-up (PW) skills. Analysis of variance demonstrated improvement in PW skills across the academic year when measured by the rating instrument, but this improvement was not detected on traditional subjective rating forms (SRF). PPI skills improved between first rotations across 2 academic years with the addition of orientation to expectations (mean, 79% versus 92%, P = 0.000). Poor correlation was noted between the National Board of Medical Examiners Surgery Subtest scores and PE skills (r = .19), PW skills (r = .20), and PPI skills (r = .15). While the overall ratings given by faculty on SRF correlated with the SRF ratings of PE skills (r = .77) and PPI skills (r = .58), these same faculty ratings correlated poorly with these skills as assessed by checklist (r = .16 and r = .14, respectively). This pilot experience demonstrates that PE skills, PW skills, and PPI skills (1) improve only with orientation to expectations and feedback, (2) correlate poorly with fund of knowledge assessment, and (3) are best assessed with precise measurement (eg, checklist, direct observation), which avoids the halo effect of overall evaluation that occurs with subjective rating forms.

UR - http://www.scopus.com/inward/record.url?scp=0028102954&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028102954&partnerID=8YFLogxK

U2 - 10.1016/0002-9610(94)90107-4

DO - 10.1016/0002-9610(94)90107-4

M3 - Article

C2 - 8209937

AN - SCOPUS:0028102954

VL - 167

SP - 604

EP - 607

JO - American Journal of Surgery

JF - American Journal of Surgery

SN - 0002-9610

IS - 6

ER -