Reliability and validity of reflection exercises for obstetrics and gynecology residents

Lee A. Learman, Amy M. Autry, Patricia O'Sullivan

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

Objective: Self-evaluation is an essential skill throughout a physician's career, and reflection is thought to be a necessary mechanism for effective self-evaluation. The aim of our study was to establish the reliability and validity of structured assessments of critical self-reflection. Study Design: Thirty-two residents completed 6 exercises that were scored from 0 (no description of event) to 6 (deep reflection). We calculated interrater and internal consistency reliability for the exercises and compared scores by postgraduate year and with other competency assessments. Results: Residents completed 183 reflections. Interrater reliability was 0.89. Surgical skill reflections scored highest (score, 3.2 ± 0.91 [SD]). Five exercises had adequate internal consistency reliability (0.62). Senior residents received higher reflection scores than junior residents; the magnitude of difference was similar for other competency measures and not statistically significant. Reflection scores were correlated with professionalism and communication skill assessments (score, 0.36-0.37; P <.01) but not with medical knowledge. Conclusion: Self-reflection can be assessed reliably with scored exercises that demonstrate concurrent validity with other assessments. We encourage further research that should include multiple training programs to further evaluate our approach for the assessment of reflection in postgraduate education.

Original languageEnglish (US)
JournalAmerican Journal of Obstetrics and Gynecology
Volume198
Issue number4
DOIs
StatePublished - Apr 2008
Externally publishedYes

Fingerprint

Gynecology
Reproducibility of Results
Obstetrics
Exercise
Diagnostic Self Evaluation
Education
Communication
Physicians
Research

Keywords

  • graduate medical education
  • reflection
  • self-assessment

ASJC Scopus subject areas

  • Medicine(all)
  • Obstetrics and Gynecology

Cite this

Reliability and validity of reflection exercises for obstetrics and gynecology residents. / Learman, Lee A.; Autry, Amy M.; O'Sullivan, Patricia.

In: American Journal of Obstetrics and Gynecology, Vol. 198, No. 4, 04.2008.

Research output: Contribution to journalArticle

Learman, Lee A. ; Autry, Amy M. ; O'Sullivan, Patricia. / Reliability and validity of reflection exercises for obstetrics and gynecology residents. In: American Journal of Obstetrics and Gynecology. 2008 ; Vol. 198, No. 4.
@article{4975e526aec347eca745dfec09ead369,
title = "Reliability and validity of reflection exercises for obstetrics and gynecology residents",
abstract = "Objective: Self-evaluation is an essential skill throughout a physician's career, and reflection is thought to be a necessary mechanism for effective self-evaluation. The aim of our study was to establish the reliability and validity of structured assessments of critical self-reflection. Study Design: Thirty-two residents completed 6 exercises that were scored from 0 (no description of event) to 6 (deep reflection). We calculated interrater and internal consistency reliability for the exercises and compared scores by postgraduate year and with other competency assessments. Results: Residents completed 183 reflections. Interrater reliability was 0.89. Surgical skill reflections scored highest (score, 3.2 ± 0.91 [SD]). Five exercises had adequate internal consistency reliability (0.62). Senior residents received higher reflection scores than junior residents; the magnitude of difference was similar for other competency measures and not statistically significant. Reflection scores were correlated with professionalism and communication skill assessments (score, 0.36-0.37; P <.01) but not with medical knowledge. Conclusion: Self-reflection can be assessed reliably with scored exercises that demonstrate concurrent validity with other assessments. We encourage further research that should include multiple training programs to further evaluate our approach for the assessment of reflection in postgraduate education.",
keywords = "graduate medical education, reflection, self-assessment",
author = "Learman, {Lee A.} and Autry, {Amy M.} and Patricia O'Sullivan",
year = "2008",
month = "4",
doi = "10.1016/j.ajog.2007.12.021",
language = "English (US)",
volume = "198",
journal = "American Journal of Obstetrics and Gynecology",
issn = "0002-9378",
publisher = "Mosby Inc.",
number = "4",

}

TY - JOUR

T1 - Reliability and validity of reflection exercises for obstetrics and gynecology residents

AU - Learman, Lee A.

AU - Autry, Amy M.

AU - O'Sullivan, Patricia

PY - 2008/4

Y1 - 2008/4

N2 - Objective: Self-evaluation is an essential skill throughout a physician's career, and reflection is thought to be a necessary mechanism for effective self-evaluation. The aim of our study was to establish the reliability and validity of structured assessments of critical self-reflection. Study Design: Thirty-two residents completed 6 exercises that were scored from 0 (no description of event) to 6 (deep reflection). We calculated interrater and internal consistency reliability for the exercises and compared scores by postgraduate year and with other competency assessments. Results: Residents completed 183 reflections. Interrater reliability was 0.89. Surgical skill reflections scored highest (score, 3.2 ± 0.91 [SD]). Five exercises had adequate internal consistency reliability (0.62). Senior residents received higher reflection scores than junior residents; the magnitude of difference was similar for other competency measures and not statistically significant. Reflection scores were correlated with professionalism and communication skill assessments (score, 0.36-0.37; P <.01) but not with medical knowledge. Conclusion: Self-reflection can be assessed reliably with scored exercises that demonstrate concurrent validity with other assessments. We encourage further research that should include multiple training programs to further evaluate our approach for the assessment of reflection in postgraduate education.

AB - Objective: Self-evaluation is an essential skill throughout a physician's career, and reflection is thought to be a necessary mechanism for effective self-evaluation. The aim of our study was to establish the reliability and validity of structured assessments of critical self-reflection. Study Design: Thirty-two residents completed 6 exercises that were scored from 0 (no description of event) to 6 (deep reflection). We calculated interrater and internal consistency reliability for the exercises and compared scores by postgraduate year and with other competency assessments. Results: Residents completed 183 reflections. Interrater reliability was 0.89. Surgical skill reflections scored highest (score, 3.2 ± 0.91 [SD]). Five exercises had adequate internal consistency reliability (0.62). Senior residents received higher reflection scores than junior residents; the magnitude of difference was similar for other competency measures and not statistically significant. Reflection scores were correlated with professionalism and communication skill assessments (score, 0.36-0.37; P <.01) but not with medical knowledge. Conclusion: Self-reflection can be assessed reliably with scored exercises that demonstrate concurrent validity with other assessments. We encourage further research that should include multiple training programs to further evaluate our approach for the assessment of reflection in postgraduate education.

KW - graduate medical education

KW - reflection

KW - self-assessment

UR - http://www.scopus.com/inward/record.url?scp=41449108879&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=41449108879&partnerID=8YFLogxK

U2 - 10.1016/j.ajog.2007.12.021

DO - 10.1016/j.ajog.2007.12.021

M3 - Article

VL - 198

JO - American Journal of Obstetrics and Gynecology

JF - American Journal of Obstetrics and Gynecology

SN - 0002-9378

IS - 4

ER -