Share Email Print

Proceedings Paper

Influencing clinicians and healthcare managers: can ROC be more persuasive?
Author(s): S. Taylor-Phillips; M. G. Wallis; A. Duncan; A. G. Gale
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Receiver Operating Characteristic analysis provides a reliable and cost effective performance measurement tool, without using full clinical trials. However, when ROC analysis shows that performance is statistically superior in one condition than another it is difficult to relate this result to effects in practice, or even to determine whether it is clinically significant. In this paper we present two concurrent analyses: using ROC methods alongside single threshold recall rate data, and suggest that reporting both provides complimentary data. Four mammographers read 160 difficult cases (41% malignant) twice, with and without prior mammograms. Lesion location and probability of malignancy was reported for each case and analyzed using JAFROC. Concurrently each participant chose recall or return to screen for each case. JAFROC analysis showed that the presence of prior mammograms improved performance (p<.05). Single threshold data showed a trend towards a 26% increase in the number of false positive recalls without prior mammograms (p=.056). If this trend were present throughout the NHS Breast Screening Programme then discarding prior mammograms would correspond to an increase in recall rate from 4.6% to 5.3%, and 12,414 extra women recalled annually for assessment. Whilst ROC methods account for all possible thresholds of recall and have higher power, providing a single threshold example of false positive, false negative, and recall rates when reporting results could be more influential for clinicians. This paper discusses whether this is a useful additional method of presenting data, or whether it is misleading and inaccurate.

Paper Details

Date Published: 23 February 2010
PDF: 9 pages
Proc. SPIE 7627, Medical Imaging 2010: Image Perception, Observer Performance, and Technology Assessment, 76270X (23 February 2010); doi: 10.1117/12.843784
Show Author Affiliations
S. Taylor-Phillips, Loughborough Univ. (United Kingdom)
M. G. Wallis, Addenbrooke's Hospital (United Kingdom)
A. Duncan, Univ. Hospital Coventry (United Kingdom)
A. G. Gale, Loughborough Univ. (United Kingdom)

Published in SPIE Proceedings Vol. 7627:
Medical Imaging 2010: Image Perception, Observer Performance, and Technology Assessment
David J. Manning; Craig K. Abbey, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?