Do Individual Attendings' Post-rotation Performance Ratings Detect Residents' Clinical Performance Deficiencies?

Cathy J. Schwind, Reed G. Williams, Margaret L. Boehler, Gary Dunnington

Research output: Contribution to journalArticle

38 Citations (Scopus)

Abstract

Purpose. To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. Method. Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. Results. Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. Conclusions. A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's pre-existing convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

Original languageEnglish (US)
Pages (from-to)453-457
Number of pages5
JournalAcademic Medicine
Volume79
Issue number5
DOIs
StatePublished - May 2004
Externally publishedYes

Fingerprint

performance disorder
deficit
rating
resident
Physicians
physician
performance
surgery
evaluation
knowledge
Medicine
medicine

ASJC Scopus subject areas

  • Nursing(all)
  • Public Health, Environmental and Occupational Health
  • Education

Cite this

Do Individual Attendings' Post-rotation Performance Ratings Detect Residents' Clinical Performance Deficiencies? / Schwind, Cathy J.; Williams, Reed G.; Boehler, Margaret L.; Dunnington, Gary.

In: Academic Medicine, Vol. 79, No. 5, 05.2004, p. 453-457.

Research output: Contribution to journalArticle

Schwind, Cathy J. ; Williams, Reed G. ; Boehler, Margaret L. ; Dunnington, Gary. / Do Individual Attendings' Post-rotation Performance Ratings Detect Residents' Clinical Performance Deficiencies?. In: Academic Medicine. 2004 ; Vol. 79, No. 5. pp. 453-457.
@article{f94bbb3d8ad64bb2970dce611ff67a6f,
title = "Do Individual Attendings' Post-rotation Performance Ratings Detect Residents' Clinical Performance Deficiencies?",
abstract = "Purpose. To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. Method. Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. Results. Thirteen of 1,986 individual post-rotation ratings (0.7{\%}) nominally noted a deficit. Post-rotation ratings of {"}good{"} or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. Conclusions. A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's pre-existing convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.",
author = "Schwind, {Cathy J.} and Williams, {Reed G.} and Boehler, {Margaret L.} and Gary Dunnington",
year = "2004",
month = "5",
doi = "10.1097/00001888-200405000-00016",
language = "English (US)",
volume = "79",
pages = "453--457",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "5",

}

TY - JOUR

T1 - Do Individual Attendings' Post-rotation Performance Ratings Detect Residents' Clinical Performance Deficiencies?

AU - Schwind, Cathy J.

AU - Williams, Reed G.

AU - Boehler, Margaret L.

AU - Dunnington, Gary

PY - 2004/5

Y1 - 2004/5

N2 - Purpose. To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. Method. Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. Results. Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. Conclusions. A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's pre-existing convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

AB - Purpose. To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. Method. Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. Results. Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. Conclusions. A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's pre-existing convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

UR - http://www.scopus.com/inward/record.url?scp=2142755861&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=2142755861&partnerID=8YFLogxK

U2 - 10.1097/00001888-200405000-00016

DO - 10.1097/00001888-200405000-00016

M3 - Article

VL - 79

SP - 453

EP - 457

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 5

ER -