Share Email Print

Proceedings Paper

An analysis of two ground truth estimation methods
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

An estimation of the so called Ground Truth (GT), i.e. the actual lesion region, can minimize readers' subjectivity if multiple readers' markings are combined. Two methods perform this estimate by considering the spatial location of voxels: Thresholded Probability-Map (TPM) and Simultaneous Truth and Performance Level Estimation (STAPLE). An analysis of these two methods has already been performed. The purpose of this study, however, is gaining a new insight into the method outcomes by comparing the estimated regions. A subset of the publicly available Lung Image Database Consortium archive was used, selecting pulmonary nodules documented by all four radiologists. The TPM estimator was computed by assigning to each voxel a value equal to average number of readers that included such voxel in their markings and then applying a threshold of 0.5. Our STAPLE implementation is loosely based on a version from ITK, to which we added the graph cut post-processing. The pair-wise similarities between the estimated ground truths were analyzed by computing the respective Jaccard coefficients. Then, the sign test of the differences between the volumes of TPM and STAPLE was performed. A total of 35 nodules documented on 26 scans by all four radiologists were available. The spatial agreement had a one-sided 90% Confidence Interval of [0.92, 1.00]. The sign test of the differences had a p-value less than 0.001. We found that (a) the differences in their volume estimates are statistically significant, (b) the spatial disagreement between the two estimators is almost completely due to the exclusion of voxels marked by exactly two readers, (c) STAPLE tends to weight more, in its GT estimate, readers marking broader regions.

Paper Details

Date Published: 27 February 2009
PDF: 8 pages
Proc. SPIE 7260, Medical Imaging 2009: Computer-Aided Diagnosis, 72600E (27 February 2009); doi: 10.1117/12.813440
Show Author Affiliations
A. M. Biancardi, Cornell Univ. (United States)
A. C. Jirapatnakul, Cornell Univ. (United States)
S. Fotin, Cornell Univ. (United States)
T. Apanasovich, Thomas Jefferson Univ. (United States)
A. P. Reeves, Cornell Univ. (United States)

Published in SPIE Proceedings Vol. 7260:
Medical Imaging 2009: Computer-Aided Diagnosis
Nico Karssemeijer; Maryellen L. Giger, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?