Share Email Print

Proceedings Paper

Efficiency gain of paired split-plot designs in MRMC ROC studies
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The widely used multi-reader multi-case ROC study design for comparing imaging modalities is the fullycrossed (FC) design: every reader reads every case of both modalities. In this work, we investigate the paired split-plot (PSP) designs that allow for reduced cost and increased flexibility compared to the FC design. In the PSP design, patient images from two modalities are read by the same readers, thereby the readings are paired across modalities. However, within each modality, not every reader reads every case. Instead, both the readers and the cases are partitioned into a number of groups and each group of readers read their own group of cases - a split-plot design. Using the U-statistic based variance analysis for AUC (i.e., area under the ROC curve), we show analytically that, with a fixed number of readings per reader, substantial statistical efficiency can be gained by the PSP design as compared to the FC design. Equivalently, we show that the PSP design can achieve the same statistical power as the FC design with substantially reduced number of readings. However, the efficiency/power gain of the PSP design comes with the increased cost of collecting a larger number of truthverified patient cases than the FC design. This means that one can trade off between different sources of cost and choose a least burdensome design. We demonstrate the advantages of the PSP design with a real-world reader study for the comparison of full field digital mammography with screen-film mammography.

Paper Details

Date Published: 7 March 2018
PDF: 8 pages
Proc. SPIE 10577, Medical Imaging 2018: Image Perception, Observer Performance, and Technology Assessment, 105770F (7 March 2018); doi: 10.1117/12.2293741
Show Author Affiliations
Weijie Chen, U.S. Food and Drug Administration (United States)
Qi Gong, U.S. Food and Drug Administration (United States)
Brandon D. Gallas, U.S. Food and Drug Administration (United States)

Published in SPIE Proceedings Vol. 10577:
Medical Imaging 2018: Image Perception, Observer Performance, and Technology Assessment
Robert M. Nishikawa; Frank W. Samuelson, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?