A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation

April Savoy, Laura G. Militello, Himalaya Patel, Mindy E. Flanagan, Alissa L. Russ, Joanne Daggy, Michael Weiner, Jason J. Saleem

Research output: Contribution to journalArticle

Abstract

Background: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results: Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p <.001. Conclusion: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

LanguageEnglish (US)
Pages138-148
Number of pages11
JournalJournal of Biomedical Informatics
Volume85
DOIs
StatePublished - Sep 1 2018

Fingerprint

Cognitive systems
Systems engineering
Referral and Consultation
Medical Electronics
User interfaces
Electronic medical equipment
Guidelines
Physicians
Communication
Communication Barriers
Medical Informatics
Health Services Accessibility
Ambulatory Care Facilities
Knowledge acquisition
Primary Health Care
Decision Making
Odds Ratio
Health care
Interviews
Information technology

Keywords

  • Cognitive systems engineering
  • Human factors
  • Medical order entry systems
  • Referral and consultation
  • Usability evaluation

ASJC Scopus subject areas

  • Computer Science Applications
  • Health Informatics

Cite this

A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation. / Savoy, April; Militello, Laura G.; Patel, Himalaya; Flanagan, Mindy E.; Russ, Alissa L.; Daggy, Joanne; Weiner, Michael; Saleem, Jason J.

In: Journal of Biomedical Informatics, Vol. 85, 01.09.2018, p. 138-148.

Research output: Contribution to journalArticle

Savoy, April ; Militello, Laura G. ; Patel, Himalaya ; Flanagan, Mindy E. ; Russ, Alissa L. ; Daggy, Joanne ; Weiner, Michael ; Saleem, Jason J. / A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation. In: Journal of Biomedical Informatics. 2018 ; Vol. 85. pp. 138-148.
@article{b6023a4e96134977bfb115edf33c6eb7,
title = "A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation",
abstract = "Background: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results: Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95{\%} CI = [9.2, 19.8]), p <.001. Conclusion: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.",
keywords = "Cognitive systems engineering, Human factors, Medical order entry systems, Referral and consultation, Usability evaluation",
author = "April Savoy and Militello, {Laura G.} and Himalaya Patel and Flanagan, {Mindy E.} and Russ, {Alissa L.} and Joanne Daggy and Michael Weiner and Saleem, {Jason J.}",
year = "2018",
month = "9",
day = "1",
doi = "10.1016/j.jbi.2018.07.021",
language = "English (US)",
volume = "85",
pages = "138--148",
journal = "Journal of Biomedical Informatics",
issn = "1532-0464",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation

AU - Savoy, April

AU - Militello, Laura G.

AU - Patel, Himalaya

AU - Flanagan, Mindy E.

AU - Russ, Alissa L.

AU - Daggy, Joanne

AU - Weiner, Michael

AU - Saleem, Jason J.

PY - 2018/9/1

Y1 - 2018/9/1

N2 - Background: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results: Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p <.001. Conclusion: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

AB - Background: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results: Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p <.001. Conclusion: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

KW - Cognitive systems engineering

KW - Human factors

KW - Medical order entry systems

KW - Referral and consultation

KW - Usability evaluation

UR - http://www.scopus.com/inward/record.url?scp=85051653248&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051653248&partnerID=8YFLogxK

U2 - 10.1016/j.jbi.2018.07.021

DO - 10.1016/j.jbi.2018.07.021

M3 - Article

VL - 85

SP - 138

EP - 148

JO - Journal of Biomedical Informatics

T2 - Journal of Biomedical Informatics

JF - Journal of Biomedical Informatics

SN - 1532-0464

ER -