Systematic Heuristic Evaluation of Computerized Consultation Order Templates: Clinicians’ and Human Factors Engineers’ Perspectives

April Savoy, Himalaya Patel, Mindy E. Flanagan, Michael Weiner, Alissa L. Russ

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians’ statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers’ information needs (10%). Severe violations were attributed mostly to meet referrers’ information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants’ requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates’ inability to support referring clinicians’ information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

Original languageEnglish (US)
Article number129
JournalJournal of Medical Systems
Volume41
Issue number8
DOIs
StatePublished - Aug 1 2017

Fingerprint

Human engineering
Referral and Consultation
Engineers
Efficiency
Communication
Safety
Patient Safety
Consultants
Heuristics
Esthetics

Keywords

  • Electronic health records
  • Heuristics
  • Human engineering
  • Medical order entry systems
  • Referral and consultation

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Information Systems
  • Health Informatics
  • Health Information Management

Cite this

Systematic Heuristic Evaluation of Computerized Consultation Order Templates : Clinicians’ and Human Factors Engineers’ Perspectives. / Savoy, April; Patel, Himalaya; Flanagan, Mindy E.; Weiner, Michael; Russ, Alissa L.

In: Journal of Medical Systems, Vol. 41, No. 8, 129, 01.08.2017.

Research output: Contribution to journalArticle

@article{ac113ae42d7841148f723caa95e1996e,
title = "Systematic Heuristic Evaluation of Computerized Consultation Order Templates: Clinicians’ and Human Factors Engineers’ Perspectives",
abstract = "We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians’ statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57{\%}) to 62 (31{\%}). Approximately 68{\%} of violations were linked to 5 heuristics: aesthetic and minimalist design (17{\%}), error prevention (16{\%}), consistency and standards (14{\%}), recognition rather than recall (11{\%}), and meet referrers’ information needs (10{\%}). Severe violations were attributed mostly to meet referrers’ information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80{\%} agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants’ requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates’ inability to support referring clinicians’ information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.",
keywords = "Electronic health records, Heuristics, Human engineering, Medical order entry systems, Referral and consultation",
author = "April Savoy and Himalaya Patel and Flanagan, {Mindy E.} and Michael Weiner and Russ, {Alissa L.}",
year = "2017",
month = "8",
day = "1",
doi = "10.1007/s10916-017-0775-7",
language = "English (US)",
volume = "41",
journal = "Journal of Medical Systems",
issn = "0148-5598",
publisher = "Springer New York",
number = "8",

}

TY - JOUR

T1 - Systematic Heuristic Evaluation of Computerized Consultation Order Templates

T2 - Clinicians’ and Human Factors Engineers’ Perspectives

AU - Savoy, April

AU - Patel, Himalaya

AU - Flanagan, Mindy E.

AU - Weiner, Michael

AU - Russ, Alissa L.

PY - 2017/8/1

Y1 - 2017/8/1

N2 - We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians’ statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers’ information needs (10%). Severe violations were attributed mostly to meet referrers’ information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants’ requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates’ inability to support referring clinicians’ information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

AB - We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians’ statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers’ information needs (10%). Severe violations were attributed mostly to meet referrers’ information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants’ requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates’ inability to support referring clinicians’ information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

KW - Electronic health records

KW - Heuristics

KW - Human engineering

KW - Medical order entry systems

KW - Referral and consultation

UR - http://www.scopus.com/inward/record.url?scp=85025431683&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85025431683&partnerID=8YFLogxK

U2 - 10.1007/s10916-017-0775-7

DO - 10.1007/s10916-017-0775-7

M3 - Article

C2 - 28735373

AN - SCOPUS:85025431683

VL - 41

JO - Journal of Medical Systems

JF - Journal of Medical Systems

SN - 0148-5598

IS - 8

M1 - 129

ER -