+ Site Statistics
+ Search Articles
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ PDF Full Text
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Translate
+ Recently Requested

Revisiting heuristic evaluation methods to improve the reliability of findings

Revisiting heuristic evaluation methods to improve the reliability of findings

Studies in Health Technology and Informatics 205: 930-934

The heuristic evaluation (HE) method is one of the most common in the suite of tools for usability evaluations because it is a fast, inexpensive and resource-efficient process in relation to the many usability issues it generates. The method emphasizes completely independent initial expert evaluations. Inter-rater reliability and agreement coefficients are not calculated. The variability across evaluators, even dual domain experts, can be significant as is seen in the case study here. The implications of this wide variability mean that results are unique to each HE, results are not readily reproducible and HE research on usability is not yet creating a uniform body of knowledge. We offer recommendations to improve the science by incorporating selected techniques from qualitative research: calculating inter-rater reliability and agreement scores, creating a codebook to define concepts/categories and offering crucial information about raters' backgrounds, agreement techniques and the evaluation setting.

(PDF emailed within 1 workday: $29.90)

Accession: 055565677

Download citation: RISBibTeXText

PMID: 25160324

Related references

Energy management and design of centralized air-conditioning systems through the non-revisiting strategy for heuristic optimization methods. Applied Energy 87(11): 3494-3506, 2010

Clinical evaluation in prosthodontics: practical methods to improve validity and reliability at the undergraduate level. Journal of Prosthetic Dentistry 75(6): 675-680, 1996

Reliability of change in lumbar MRI findings over time in patients with and without disc prosthesis--comparing two different image evaluation methods. Skeletal Radiology 41(12): 1547-1557, 2013

Using Heuristic Evaluation to Improve Sepsis Alert Usability. Critical Care Nursing Clinics of North America 30(2): 297-309, 2018

Using reproductive evaluation to improve genetic search and heuristic discovery. Genetic algorithms and their applications: proceedings of the second International Conference on Genetic Algorithms: July 28-31-1987 at the Massachusetts Institute of Technology Cambridge MA: 115, 1987

Applying heuristic evaluation to improve the usability of a telemedicine system. Telemedicine Journal and E-Health 12(1): 24-34, 2006

Validity and reliability of existing and modified clinical methods of measuring femoral and tibiofibular torsion in healthy subjects: use of different reference axes may improve reliability. Clinical Anatomy 18(1): 46-55, 2004

An efficient approach to improve the usability of e-learning resources: the role of heuristic evaluation. Advances in Physiology Education 37(3): 242-248, 2014

Applying heuristic evaluation to observation chart design to improve the detection of patient deterioration. Applied Ergonomics 44(4): 544-556, 2013

A comparison of usability evaluation methods: heuristic evaluation versus end-user think-aloud protocol - an example from a web-based communication tool for nurse scheduling. AMIA ... Annual Symposium Proceedings. AMIA Symposium 2009: 714-718, 2011

Comparative study of heuristic evaluation and usability testing methods. Studies in Health Technology and Informatics 143: 322-327, 2009

An evaluation of heuristic methods for determining the best table mix in full-service restaurants. Journal of Operations Management 23(6): 599-617, 2005

Comparison of two heuristic evaluation methods for evaluating the usability of health information systems. Journal of Biomedical Informatics 80: 37-42, 2018

Vibration testing: Optimising methods to improve reliability. Muscle and Nerve 2018, 2018

Methods to improve the reliability of histopathological diagnoses in the placenta. Paediatric and Perinatal Epidemiology 14(2): 172-178, 2000