Comparative study of heuristic evaluation and usability testing methods

Thankam Paul Thyvalikakath, Valerie Monaco, Himabindu Thambuganipalle, Titus Schleyer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Usability methods, such as heuristic evaluation, cognitive walk-throughs and user testing, are increasingly used to evaluate and improve the design of clinical software applications. There is still some uncertainty, however, as to how those methods can be used to support the development process and evaluation in the most meaningful manner. In this study, we compared the results of a heuristic evaluation with those of formal user tests in order to determine which usability problems were detected by both methods. We conducted heuristic evaluation and usability testing on four major commercial dental computer-based patient records (CPRs), which together cover 80% of the market for chairside computer systems among general dentists. Both methods yielded strong evidence that the dental CPRs have significant usability problems. An average of 50% of empirically-determined usability problems were identified by the preceding heuristic evaluation. Some statements of heuristic violations were specific enough to precisely identify the actual usability problem that study participants encountered. Other violations were less specific, but still manifested themselves in usability problems and poor task outcomes. In this study, heuristic evaluation identified a significant portion of problems found during usability testing. While we make no assumptions about the generalizability of the results to other domains and software systems, heuristic evaluation may, under certain circumstances, be a useful tool to determine design problems early in the development cycle.

Original languageEnglish (US)
Title of host publicationAdvances in Information Technology and Communication in Health
PublisherIOS Press
Pages322-327
Number of pages6
ISBN (Print)9781586039790
DOIs
StatePublished - Jan 1 2009
Event2009 Conference on Advances in Information Technology and Communication in Health: Revolutionizing Health Care with Informatics - From Research to Practice, ITCH 2009 - Victoria, BC, Canada
Duration: Feb 19 2009Feb 22 2009

Publication series

NameStudies in Health Technology and Informatics
Volume143
ISSN (Print)0926-9630
ISSN (Electronic)1879-8365

Other

Other2009 Conference on Advances in Information Technology and Communication in Health: Revolutionizing Health Care with Informatics - From Research to Practice, ITCH 2009
CountryCanada
CityVictoria, BC
Period2/19/092/22/09

    Fingerprint

Keywords

  • Dental CPR
  • Heuristic evaluation
  • Human-computer interaction
  • Usability testing

ASJC Scopus subject areas

  • Biomedical Engineering
  • Health Informatics
  • Health Information Management

Cite this

Thyvalikakath, T. P., Monaco, V., Thambuganipalle, H., & Schleyer, T. (2009). Comparative study of heuristic evaluation and usability testing methods. In Advances in Information Technology and Communication in Health (pp. 322-327). (Studies in Health Technology and Informatics; Vol. 143). IOS Press. https://doi.org/10.3233/978-1-58603-979-0-322