Nationwide colposcopy assessment system goes online

Colposcopy is a clinical examination performed mostly in high-resource countries for detection and management of disorders of the female lower genital tract, including cancer and precancer in particular. A physician or other advanced medical practitioner examines a woman's cervix and surrounding tissue using a colposcope, a special microscope that provides illuminated, low-power magnification of the cervical region. Abnormal-appearing areas are biopsied, and the biopsy result determines the appropriate clinical management.¹ A woman is typically referred to colposcopy as a result of an abnormal cytology-screening (Papanicolaou or Pap test) result. Practitioners of colposcopy include nurse practitioners, family physicians, gynecologists, and gynecologic oncologists. Training and evaluation of professional expertise in colposcopy is not well standardized, but it has been improved in recent years by the incorporation of computer technology using image-based learning and testing modules. However, even these technological advances have been hampered by a shortage of expert-graded image examples, lack of interactivity, and brittle system architectures that made content modification difficult.

To tackle these problems, engineers from the National Library of Medicine (NLM) teamed with National Cancer Institute (NCI) and American Society for Colposcopy and Cervical Pathology (ASCCP) gynecologic-oncology experts to create the Teaching Tool—an online system for administering and automatically grading image-based exams—and develop the content required for two proficiency exams in colposcopy. Figure 1 shows a sample question, where the test taker is asked to respond with a visual diagnosis of the cervix condition.

Figure 1. Diagnosis question from a colposcopy exam.

The images used in these exams were mainly taken from the 100,000-image collection of cervicography pictures acquired by the NCI in Costa Rica and at several sites in the US. (Cervicography, which uses a handheld camera to acquire pictures, produces images comparable to those seen with a colposcope.) NCI and ASCCP doctors carried out a meticulous image review to select example pictures of good visual quality, which represent the range of possible cervix diagnoses and for which expert diagnostic consensus could be obtained.

One of the important questions asked on the exams is, where should a biopsy be taken? The test taker using the Teaching Tool provides an answer with a mouse click. Hence, once the images for the exams had been selected, it was necessary to establish ‘consensus biopsy regions’ for scoring a response as correct or incorrect. To create these, approximately 20 experts in oncologic gynecology used web-based custom software, also developed by NLM, to individually view the cervix images and graphically annotate biopsy regions. This annotated image data was transmitted back to an NLM database and subsequently reviewed to achieve consensus to establish the correct biopsy answers.

ASCCP doctors are particularly happy with the interactive biopsy capability. After the test taker has decided where to biopsy, the Teaching Tool mimics real life by providing the correct biopsy result, so that he or she can decide on a management option. The first of the two ASCCP exams implemented in the Teaching Tool, the Residents' Online Exam, is used in residency training in colposcopy through a national program. This exam tests general medical knowledge in the field of lower-genital-tract cancer and precancer, proficiency at visual diagnosis and clinical-feature recognition, and the ability to diagnose, biopsy, and manage a set of 12 patient cases. There are now 52 residency programs in obstetrics/gynecology or family practice that use the Teaching Tool for this ASCCP exam nationwide, including George Washington University, The University of Texas Southwestern Medical Center, and Duke University Medical Center.

The second exam provided by the Teaching Tool is the Colposcopy Mentorship Program exam, a more extensive version of the residents' exam, which is given in the final tier of the ASCCP's three-tiered mentorship program, which is intended to further skills and understanding of the beginning colposcopist. Since the Teaching Tool became operational in May 2010, more than 250 online exams have been administered, and there are more than 650 users of the system, including resident-program administrators, users who have taken exams, and users signed up but not yet tested.

In addition to its capability to present exam questions and collect responses online, the Teaching Tool allows administrators to add and delete users, assign exams, and generate exam-results reports. It also offers a practice exam to acquaint users with the system and provides immediate exam results to test takers in resident programs. Planned future work includes implementation of the highest-level ASCCP exam, the Colposcopy Recognition Award exam, and creation of online self-training modules.

The Teaching Tool is the result of close collaboration of Rodney Long, Scott Budihas, and Lakeyser Phillips of NLM's Communications Engineering Branch, Mark Schiffman and Nicolas Wentzensen of the NCI, Jose Jeronimo of the Program for Appropriate Technology in Health, ASCCP doctors Alex Waxman and Dennis O'Connor, and Kathy Poole, executive director of the ASCCP. This work was supported by the Intramural Research Program of the National Institutes of Health, the NLM, and the Lister Hill National Center for Biomedical Communications.