Evaluating resident operative performance: A qualitative analysis of expert opinions

Hilary Sanfey, Reed G. Williams, Xiaodong Chen, Gary L. Dunnington

Research output: Contribution to journalArticle

20 Citations (Scopus)

Abstract

Purpose: To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings. Methods: Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills. Results: Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28%) of 300 post rating comments with majority agreement on another 83 (28%). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P <.05). Conclusion: Fewer than 30% of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56% of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.

Original languageEnglish (US)
Pages (from-to)759-770
Number of pages12
JournalSurgery
Volume150
Issue number4
DOIs
StatePublished - Oct 1 2011

Fingerprint

Expert Testimony
Laparoscopic Cholecystectomy
Ventral Hernia
Process Assessment (Health Care)
Groin
Herniorrhaphy
Thyroidectomy
Patient Safety
Mental Competency
Consensus

ASJC Scopus subject areas

  • Surgery

Cite this

Evaluating resident operative performance : A qualitative analysis of expert opinions. / Sanfey, Hilary; Williams, Reed G.; Chen, Xiaodong; Dunnington, Gary L.

In: Surgery, Vol. 150, No. 4, 01.10.2011, p. 759-770.

Research output: Contribution to journalArticle

Sanfey, Hilary ; Williams, Reed G. ; Chen, Xiaodong ; Dunnington, Gary L. / Evaluating resident operative performance : A qualitative analysis of expert opinions. In: Surgery. 2011 ; Vol. 150, No. 4. pp. 759-770.
@article{ab523cac733a4619af3c5851508fd612,
title = "Evaluating resident operative performance: A qualitative analysis of expert opinions",
abstract = "Purpose: To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings. Methods: Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills. Results: Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28{\%}) of 300 post rating comments with majority agreement on another 83 (28{\%}). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P <.05). Conclusion: Fewer than 30{\%} of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56{\%} of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.",
author = "Hilary Sanfey and Williams, {Reed G.} and Xiaodong Chen and Dunnington, {Gary L.}",
year = "2011",
month = "10",
day = "1",
doi = "10.1016/j.surg.2011.07.058",
language = "English (US)",
volume = "150",
pages = "759--770",
journal = "Surgery",
issn = "0039-6060",
publisher = "Mosby Inc.",
number = "4",

}

TY - JOUR

T1 - Evaluating resident operative performance

T2 - A qualitative analysis of expert opinions

AU - Sanfey, Hilary

AU - Williams, Reed G.

AU - Chen, Xiaodong

AU - Dunnington, Gary L.

PY - 2011/10/1

Y1 - 2011/10/1

N2 - Purpose: To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings. Methods: Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills. Results: Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28%) of 300 post rating comments with majority agreement on another 83 (28%). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P <.05). Conclusion: Fewer than 30% of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56% of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.

AB - Purpose: To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings. Methods: Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills. Results: Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28%) of 300 post rating comments with majority agreement on another 83 (28%). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P <.05). Conclusion: Fewer than 30% of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56% of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.

UR - http://www.scopus.com/inward/record.url?scp=80054100947&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80054100947&partnerID=8YFLogxK

U2 - 10.1016/j.surg.2011.07.058

DO - 10.1016/j.surg.2011.07.058

M3 - Article

C2 - 22000189

AN - SCOPUS:80054100947

VL - 150

SP - 759

EP - 770

JO - Surgery

JF - Surgery

SN - 0039-6060

IS - 4

ER -