Evaluation Tools in Simulation Learning: Performance and Self-Efficacy in Emergency Response

Jacqueline J. Arnold, LeAnn M. Johnson, Sharon J. Tucker, James F. Malec, Sarah E. Henrickson, William F. Dunn

Research output: Contribution to journalArticle

24 Citations (Scopus)

Abstract

Introduction: High performance in emergency response is critical for high-quality patient outcomes but is not assessed in some traditional critical care nurse internships. Recognition of the opportunity to assess these skills in a simulated environment is growing, but validated measurement tools are needed. This study develops and evaluates two tools to assess confidence in emergency response and psychomotor performance during a simulated emergency response experience. Method: Using constructs from basic life support and advanced cardiac life support, an emergency response performance tool (ERPT) and a confidence tool were developed. The ERPT consisted of 11 categorical items and six timed tasks. The confidence tool comprised 17 items and required participants to rate their confidence on a scale from 0% to 100%. A total of 41 registered nurses were assigned to one of three groups: (a) those experienced in critical care without formal training, (b) those with < 13 months experience with formal training, and (c) those with no critical care experience or training. All participants completed the knowledge test and confidence tool. From the 41 participants, 12 were selected randomly and scheduled to participate in one simulation event and then rate their confidence a second time. Trained raters viewed video recordings of the participants' performance and rated them using the ERPT. Results: The statistical findings from this study support reliability and validity for the ERPT performance measure and reliability and internal consistency of the confidence tool. Interrater reliability on the ERPT was high for the majority of items. Comparisons among the three groups found significant differences in knowledge and confidence among the groups. Conclusions: The confidence tool and the ERPT show a basic level of validity, reliability, and usability. Findings from the study have guided the development of more-robust performance and confidence instruments to use in a future comparison study to examine three different simulation teaching methodologies.

Original languageEnglish
JournalClinical Simulation in Nursing
Volume5
Issue number1
DOIs
StatePublished - Jan 2009

Fingerprint

Self-efficacy
Emergency Response
learning performance
Self Efficacy
self-efficacy
Emergencies
Learning
Confidence
simulation
confidence
Evaluation
evaluation
Simulation
performance
Critical Care
Reproducibility of Results
Advanced Cardiac Life Support
Nurses
Psychomotor Performance
Video Recording

Keywords

  • confidence
  • emergency response
  • performance measurement
  • simulation

ASJC Scopus subject areas

  • Nursing (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

Arnold, J. J., Johnson, L. M., Tucker, S. J., Malec, J. F., Henrickson, S. E., & Dunn, W. F. (2009). Evaluation Tools in Simulation Learning: Performance and Self-Efficacy in Emergency Response. Clinical Simulation in Nursing, 5(1). https://doi.org/10.1016/j.ecns.2008.10.003

Evaluation Tools in Simulation Learning : Performance and Self-Efficacy in Emergency Response. / Arnold, Jacqueline J.; Johnson, LeAnn M.; Tucker, Sharon J.; Malec, James F.; Henrickson, Sarah E.; Dunn, William F.

In: Clinical Simulation in Nursing, Vol. 5, No. 1, 01.2009.

Research output: Contribution to journalArticle

Arnold, Jacqueline J. ; Johnson, LeAnn M. ; Tucker, Sharon J. ; Malec, James F. ; Henrickson, Sarah E. ; Dunn, William F. / Evaluation Tools in Simulation Learning : Performance and Self-Efficacy in Emergency Response. In: Clinical Simulation in Nursing. 2009 ; Vol. 5, No. 1.
@article{7f06c03452d94a2984e4322ecd77d42d,
title = "Evaluation Tools in Simulation Learning: Performance and Self-Efficacy in Emergency Response",
abstract = "Introduction: High performance in emergency response is critical for high-quality patient outcomes but is not assessed in some traditional critical care nurse internships. Recognition of the opportunity to assess these skills in a simulated environment is growing, but validated measurement tools are needed. This study develops and evaluates two tools to assess confidence in emergency response and psychomotor performance during a simulated emergency response experience. Method: Using constructs from basic life support and advanced cardiac life support, an emergency response performance tool (ERPT) and a confidence tool were developed. The ERPT consisted of 11 categorical items and six timed tasks. The confidence tool comprised 17 items and required participants to rate their confidence on a scale from 0{\%} to 100{\%}. A total of 41 registered nurses were assigned to one of three groups: (a) those experienced in critical care without formal training, (b) those with < 13 months experience with formal training, and (c) those with no critical care experience or training. All participants completed the knowledge test and confidence tool. From the 41 participants, 12 were selected randomly and scheduled to participate in one simulation event and then rate their confidence a second time. Trained raters viewed video recordings of the participants' performance and rated them using the ERPT. Results: The statistical findings from this study support reliability and validity for the ERPT performance measure and reliability and internal consistency of the confidence tool. Interrater reliability on the ERPT was high for the majority of items. Comparisons among the three groups found significant differences in knowledge and confidence among the groups. Conclusions: The confidence tool and the ERPT show a basic level of validity, reliability, and usability. Findings from the study have guided the development of more-robust performance and confidence instruments to use in a future comparison study to examine three different simulation teaching methodologies.",
keywords = "confidence, emergency response, performance measurement, simulation",
author = "Arnold, {Jacqueline J.} and Johnson, {LeAnn M.} and Tucker, {Sharon J.} and Malec, {James F.} and Henrickson, {Sarah E.} and Dunn, {William F.}",
year = "2009",
month = "1",
doi = "10.1016/j.ecns.2008.10.003",
language = "English",
volume = "5",
journal = "Clinical Simulation in Nursing",
issn = "1876-1399",
publisher = "Elsevier Inc.",
number = "1",

}

TY - JOUR

T1 - Evaluation Tools in Simulation Learning

T2 - Performance and Self-Efficacy in Emergency Response

AU - Arnold, Jacqueline J.

AU - Johnson, LeAnn M.

AU - Tucker, Sharon J.

AU - Malec, James F.

AU - Henrickson, Sarah E.

AU - Dunn, William F.

PY - 2009/1

Y1 - 2009/1

N2 - Introduction: High performance in emergency response is critical for high-quality patient outcomes but is not assessed in some traditional critical care nurse internships. Recognition of the opportunity to assess these skills in a simulated environment is growing, but validated measurement tools are needed. This study develops and evaluates two tools to assess confidence in emergency response and psychomotor performance during a simulated emergency response experience. Method: Using constructs from basic life support and advanced cardiac life support, an emergency response performance tool (ERPT) and a confidence tool were developed. The ERPT consisted of 11 categorical items and six timed tasks. The confidence tool comprised 17 items and required participants to rate their confidence on a scale from 0% to 100%. A total of 41 registered nurses were assigned to one of three groups: (a) those experienced in critical care without formal training, (b) those with < 13 months experience with formal training, and (c) those with no critical care experience or training. All participants completed the knowledge test and confidence tool. From the 41 participants, 12 were selected randomly and scheduled to participate in one simulation event and then rate their confidence a second time. Trained raters viewed video recordings of the participants' performance and rated them using the ERPT. Results: The statistical findings from this study support reliability and validity for the ERPT performance measure and reliability and internal consistency of the confidence tool. Interrater reliability on the ERPT was high for the majority of items. Comparisons among the three groups found significant differences in knowledge and confidence among the groups. Conclusions: The confidence tool and the ERPT show a basic level of validity, reliability, and usability. Findings from the study have guided the development of more-robust performance and confidence instruments to use in a future comparison study to examine three different simulation teaching methodologies.

AB - Introduction: High performance in emergency response is critical for high-quality patient outcomes but is not assessed in some traditional critical care nurse internships. Recognition of the opportunity to assess these skills in a simulated environment is growing, but validated measurement tools are needed. This study develops and evaluates two tools to assess confidence in emergency response and psychomotor performance during a simulated emergency response experience. Method: Using constructs from basic life support and advanced cardiac life support, an emergency response performance tool (ERPT) and a confidence tool were developed. The ERPT consisted of 11 categorical items and six timed tasks. The confidence tool comprised 17 items and required participants to rate their confidence on a scale from 0% to 100%. A total of 41 registered nurses were assigned to one of three groups: (a) those experienced in critical care without formal training, (b) those with < 13 months experience with formal training, and (c) those with no critical care experience or training. All participants completed the knowledge test and confidence tool. From the 41 participants, 12 were selected randomly and scheduled to participate in one simulation event and then rate their confidence a second time. Trained raters viewed video recordings of the participants' performance and rated them using the ERPT. Results: The statistical findings from this study support reliability and validity for the ERPT performance measure and reliability and internal consistency of the confidence tool. Interrater reliability on the ERPT was high for the majority of items. Comparisons among the three groups found significant differences in knowledge and confidence among the groups. Conclusions: The confidence tool and the ERPT show a basic level of validity, reliability, and usability. Findings from the study have guided the development of more-robust performance and confidence instruments to use in a future comparison study to examine three different simulation teaching methodologies.

KW - confidence

KW - emergency response

KW - performance measurement

KW - simulation

UR - http://www.scopus.com/inward/record.url?scp=58449121644&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=58449121644&partnerID=8YFLogxK

U2 - 10.1016/j.ecns.2008.10.003

DO - 10.1016/j.ecns.2008.10.003

M3 - Article

AN - SCOPUS:58449121644

VL - 5

JO - Clinical Simulation in Nursing

JF - Clinical Simulation in Nursing

SN - 1876-1399

IS - 1

ER -