Share Email Print

Proceedings Paper

Video-mosaicking of in vivo reflectance confocal microscopy images for noninvasive examination of skin lesion (Conference Presentation)
Author(s): Kivanc Kose; Mengran Gou; Oriol Yelamos; Miguel A. Cordova; Anthony Rossi; Kishwer S. Nehal; Octavia I. Camps; Jennifer G. Dy; Dana H. Brooks; Milind Rajadhyaksha

Paper Abstract

In this report we describe a computer vision based pipeline to convert in-vivo reflectance confocal microscopy (RCM) videos collected with a handheld system into large field of view (FOV) mosaics. For many applications such as imaging of hard to access lesions, intraoperative assessment of MOHS margins, or delineation of lesion margins beyond clinical borders, raster scan based mosaicing techniques have clinically significant limitations. In such cases, clinicians often capture RCM videos by freely moving a handheld microscope over the area of interest, but the resulting videos lose large-scale spatial relationships. Videomosaicking is a standard computational imaging technique to register, and stitch together consecutive frames of videos into large FOV high resolution mosaics. However, mosaicing RCM videos collected in-vivo has unique challenges: (i) tissue may deform or warp due to physical contact with the microscope objective lens, (ii) discontinuities or “jumps” between consecutive images and motion blur artifacts may occur, due to manual operation of the microscope, and (iii) optical sectioning and resolution may vary between consecutive images due to scattering and aberrations induced by changes in imaging depth and tissue morphology. We addressed these challenges by adapting or developing new algorithmic methods for videomosaicking, specifically by modeling non-rigid deformations, followed by automatically detecting discontinuities (cut locations) and, finally, applying a data-driven image stitching approach that fully preserves resolution and tissue morphologic detail without imposing arbitrary pre-defined boundaries. We will present example mosaics obtained by clinical imaging of both melanoma and non-melanoma skin cancers. The ability to combine freehand mosaicing for handheld microscopes with preserved cellular resolution will have high impact application in diverse clinical settings, including low-resource healthcare systems.

Paper Details

Date Published: 19 April 2017
PDF: 1 pages
Proc. SPIE 10037, Photonics in Dermatology and Plastic Surgery, 100370D (19 April 2017); doi: 10.1117/12.2253085
Show Author Affiliations
Kivanc Kose, Memorial Sloan-Kettering Cancer Ctr. (United States)
Mengran Gou, Northeastern Univ. (United States)
Oriol Yelamos, Memorial Sloan-Kettering Cancer Ctr. (United States)
Miguel A. Cordova, Memorial Sloan-Kettering Cancer Ctr. (United States)
Anthony Rossi, Memorial Sloan-Kettering Cancer Ctr. (United States)
Kishwer S. Nehal, Memorial Sloan-Kettering Cancer Ctr. (United States)
Octavia I. Camps, Northeastern Univ. (United States)
Jennifer G. Dy, Northeastern Univ. (United States)
Dana H. Brooks, Northeastern Univ. (United States)
Milind Rajadhyaksha, Memorial Sloan-Kettering Cancer Ctr. (United States)

Published in SPIE Proceedings Vol. 10037:
Photonics in Dermatology and Plastic Surgery
Bernard Choi; Haishan Zeng; Nikiforos Kollias, Editor(s)

© SPIE. Terms of Use
Back to Top