Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses

Malathi Srinivasan, Debra Litzelman, Roopa Seshadri, Kathleen Lane, Wei Zhou, Stephen Bogdewic, Margaret Gaffney, Matt Galvin, Gary Mitchell, Patricia Treadwell, Lynn Willis

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Purpose. To develop an instrument for measuring medical educators' responses to learners' lapses in professional behavior. Method. In 1999, at the Indiana University School of Medicine, a 22-item checklist of behaviors was developed to describe common responses used by educators responding to learners' lapses in professional behavior. Four medical students were trained to portray lapses in professional behaviors. These students and seven clinical observers trained to categorize behaviors as present or absent. Interrater reliability was assessed during 18 objective structured teaching evaluations (OSTEs). Videotaped OSTEs were coded twice at a one-month interval for test-retest reliability. Items were classified as low, moderate, or high inference behaviors. Script realism and educator effectiveness were assessed. Results. Educators rated OSTE scripts as realistic. Raters observed an average of 6 ± 2 educator behaviors in reaction to learners' lapses in professional behavior. Educators' responses were rated as moderately effective. More experienced educators attempted more interventions and were more effective. Agreement was high among raters (86% ± 7%), while intraclass correlation coefficients decreased with increasing inference level. From video-taped OSTEs, raters scored each behavior identically 86% of the time. Conclusions. Accurate feedback on educators' interactions in addressing learners' professionalism is essential for faculty development. Traditionally, educators have felt that faculty's responses to learners' lapses in professional behavior were difficult to observe and categorize. These data suggest that educators' responses to learners' lapses in professional behavior can be defined and reliably coded. This work will help provide objective feedback to faculty when engaging learners about lapses in professional behavior.

Original languageEnglish
Pages (from-to)888-896
Number of pages9
JournalAcademic Medicine
Volume79
Issue number9
DOIs
StatePublished - Sep 2004

Fingerprint

Teaching
educator
evaluation
Checklist
Medical Students
Reproducibility of Results
Medicine
Students
realism
medical student
video
medicine
present
interaction

ASJC Scopus subject areas

  • Nursing(all)
  • Public Health, Environmental and Occupational Health
  • Education

Cite this

Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses. / Srinivasan, Malathi; Litzelman, Debra; Seshadri, Roopa; Lane, Kathleen; Zhou, Wei; Bogdewic, Stephen; Gaffney, Margaret; Galvin, Matt; Mitchell, Gary; Treadwell, Patricia; Willis, Lynn.

In: Academic Medicine, Vol. 79, No. 9, 09.2004, p. 888-896.

Research output: Contribution to journalArticle

Srinivasan, M, Litzelman, D, Seshadri, R, Lane, K, Zhou, W, Bogdewic, S, Gaffney, M, Galvin, M, Mitchell, G, Treadwell, P & Willis, L 2004, 'Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses', Academic Medicine, vol. 79, no. 9, pp. 888-896. https://doi.org/10.1097/00001888-200409000-00017
Srinivasan, Malathi ; Litzelman, Debra ; Seshadri, Roopa ; Lane, Kathleen ; Zhou, Wei ; Bogdewic, Stephen ; Gaffney, Margaret ; Galvin, Matt ; Mitchell, Gary ; Treadwell, Patricia ; Willis, Lynn. / Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses. In: Academic Medicine. 2004 ; Vol. 79, No. 9. pp. 888-896.
@article{74668a10962440d8a989a0511d0a1266,
title = "Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses",
abstract = "Purpose. To develop an instrument for measuring medical educators' responses to learners' lapses in professional behavior. Method. In 1999, at the Indiana University School of Medicine, a 22-item checklist of behaviors was developed to describe common responses used by educators responding to learners' lapses in professional behavior. Four medical students were trained to portray lapses in professional behaviors. These students and seven clinical observers trained to categorize behaviors as present or absent. Interrater reliability was assessed during 18 objective structured teaching evaluations (OSTEs). Videotaped OSTEs were coded twice at a one-month interval for test-retest reliability. Items were classified as low, moderate, or high inference behaviors. Script realism and educator effectiveness were assessed. Results. Educators rated OSTE scripts as realistic. Raters observed an average of 6 ± 2 educator behaviors in reaction to learners' lapses in professional behavior. Educators' responses were rated as moderately effective. More experienced educators attempted more interventions and were more effective. Agreement was high among raters (86{\%} ± 7{\%}), while intraclass correlation coefficients decreased with increasing inference level. From video-taped OSTEs, raters scored each behavior identically 86{\%} of the time. Conclusions. Accurate feedback on educators' interactions in addressing learners' professionalism is essential for faculty development. Traditionally, educators have felt that faculty's responses to learners' lapses in professional behavior were difficult to observe and categorize. These data suggest that educators' responses to learners' lapses in professional behavior can be defined and reliably coded. This work will help provide objective feedback to faculty when engaging learners about lapses in professional behavior.",
author = "Malathi Srinivasan and Debra Litzelman and Roopa Seshadri and Kathleen Lane and Wei Zhou and Stephen Bogdewic and Margaret Gaffney and Matt Galvin and Gary Mitchell and Patricia Treadwell and Lynn Willis",
year = "2004",
month = "9",
doi = "10.1097/00001888-200409000-00017",
language = "English",
volume = "79",
pages = "888--896",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "9",

}

TY - JOUR

T1 - Developing an OSTE to address lapses in learners' professional behavior and an instrument to code educators' responses

AU - Srinivasan, Malathi

AU - Litzelman, Debra

AU - Seshadri, Roopa

AU - Lane, Kathleen

AU - Zhou, Wei

AU - Bogdewic, Stephen

AU - Gaffney, Margaret

AU - Galvin, Matt

AU - Mitchell, Gary

AU - Treadwell, Patricia

AU - Willis, Lynn

PY - 2004/9

Y1 - 2004/9

N2 - Purpose. To develop an instrument for measuring medical educators' responses to learners' lapses in professional behavior. Method. In 1999, at the Indiana University School of Medicine, a 22-item checklist of behaviors was developed to describe common responses used by educators responding to learners' lapses in professional behavior. Four medical students were trained to portray lapses in professional behaviors. These students and seven clinical observers trained to categorize behaviors as present or absent. Interrater reliability was assessed during 18 objective structured teaching evaluations (OSTEs). Videotaped OSTEs were coded twice at a one-month interval for test-retest reliability. Items were classified as low, moderate, or high inference behaviors. Script realism and educator effectiveness were assessed. Results. Educators rated OSTE scripts as realistic. Raters observed an average of 6 ± 2 educator behaviors in reaction to learners' lapses in professional behavior. Educators' responses were rated as moderately effective. More experienced educators attempted more interventions and were more effective. Agreement was high among raters (86% ± 7%), while intraclass correlation coefficients decreased with increasing inference level. From video-taped OSTEs, raters scored each behavior identically 86% of the time. Conclusions. Accurate feedback on educators' interactions in addressing learners' professionalism is essential for faculty development. Traditionally, educators have felt that faculty's responses to learners' lapses in professional behavior were difficult to observe and categorize. These data suggest that educators' responses to learners' lapses in professional behavior can be defined and reliably coded. This work will help provide objective feedback to faculty when engaging learners about lapses in professional behavior.

AB - Purpose. To develop an instrument for measuring medical educators' responses to learners' lapses in professional behavior. Method. In 1999, at the Indiana University School of Medicine, a 22-item checklist of behaviors was developed to describe common responses used by educators responding to learners' lapses in professional behavior. Four medical students were trained to portray lapses in professional behaviors. These students and seven clinical observers trained to categorize behaviors as present or absent. Interrater reliability was assessed during 18 objective structured teaching evaluations (OSTEs). Videotaped OSTEs were coded twice at a one-month interval for test-retest reliability. Items were classified as low, moderate, or high inference behaviors. Script realism and educator effectiveness were assessed. Results. Educators rated OSTE scripts as realistic. Raters observed an average of 6 ± 2 educator behaviors in reaction to learners' lapses in professional behavior. Educators' responses were rated as moderately effective. More experienced educators attempted more interventions and were more effective. Agreement was high among raters (86% ± 7%), while intraclass correlation coefficients decreased with increasing inference level. From video-taped OSTEs, raters scored each behavior identically 86% of the time. Conclusions. Accurate feedback on educators' interactions in addressing learners' professionalism is essential for faculty development. Traditionally, educators have felt that faculty's responses to learners' lapses in professional behavior were difficult to observe and categorize. These data suggest that educators' responses to learners' lapses in professional behavior can be defined and reliably coded. This work will help provide objective feedback to faculty when engaging learners about lapses in professional behavior.

UR - http://www.scopus.com/inward/record.url?scp=4243086969&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=4243086969&partnerID=8YFLogxK

U2 - 10.1097/00001888-200409000-00017

DO - 10.1097/00001888-200409000-00017

M3 - Article

C2 - 15326017

AN - SCOPUS:4243086969

VL - 79

SP - 888

EP - 896

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 9

ER -