Share Email Print

Proceedings Paper

Performance analysis of 3-class classifiers: properties of the 3D ROC surface and the normalized volume under the surface
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Many computer-aided diagnostic problems involve three-class classification (e.g., abnormal (malignant), normal, and benign classes). A general treatment of the performance of a 3-class classifier results in a complex sixdimensional (6D) ROC space for which no analysis tools are available at present. A practical paradigm for performance analysis and a figure of merit (FOM) are yet to be developed. The area Az under the ROC curve plays an important role in the evaluation of two-class classifiers. An analogous FOM for three-class classification is highly desirable. We have been investigating conditions for reducing the 6D ROC space to 3D by constraining the utilities of some of the decisions in the classification task. These assumptions lead to a 3D ROC space in which the true-positive fraction (TPF) of one class can be expressed as a function of the two types of false-positive fractions (FPFs) from the other two classes. In this study, we investigated the properties of the 3D ROC surface for a maximum-likelihood classifier under the condition that the three class distributions in the feature space are multivariate normal with equal covariance matrices. Under the same conditions, we studied the properties of the normalized volume under the surface (NVUS) that was previously proposed as an FOM for a 3-class classifier. Under the equal covariance Gaussian assumption, it can be shown that the probability density functions of the decision variables follow a bivariate log-normal distribution. By considering these probability density functions, we can express the TPF in terms of the FPFs, and numerically integrate over the domain of the TPF to obtain the NVUS. We have compared the NVUS value obtained by directly integrating the probability density functions to that obtained from a Monte Carlo simulation study in which the 3D ROC surface was generated by empirical "optimal" classification of case samples in the multi-dimensional feature space following the assumed distributions. The NVUS values obtained from the direct integration method and the Monte Carlo simulation were found to be in good agreement. Our results indicate that the NVUS exhibits many of the intuitive properties that a proper FOM should satisfy and may be used as a performance index under the conditions that we imposed on the utilities.

Paper Details

Date Published: 17 March 2006
PDF: 7 pages
Proc. SPIE 6146, Medical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment, 61460C (17 March 2006); doi: 10.1117/12.654581
Show Author Affiliations
Berkman Sahiner, Univ. of Michigan (United States)
Heang-Ping Chan, Univ. of Michigan (United States)
Lubomir M. Hadjiiski, Univ. of Michigan (United States)

Published in SPIE Proceedings Vol. 6146:
Medical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment
Yulei Jiang; Miguel P. Eckstein, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?