Improved segmentation of damages on high-resolution coating images using CNN-based ensemble learning

The automation of inspection processes in aircraft engines comprises challenging computer vision tasks. In particular, the inspection of coating damages in confined spaces with hand-held endoscopes is based on image data acquired under dynamic operating conditions (illumination, position and orientation of the sensor, etc.). In this study, 2D RGB video data is processed to quantify damages in large coating areas. Therefore, the video frames are pre-processed by feature tracking and stitching algorithms to generate high-resolution overview images. For the subsequent analysis of the whole coating area and to overcome the challenges posed by the diverse image data, Convolutional Neural Networks (CNNs) are applied. In a preliminary study, it was found that the image analysis is advantageous when executed on different scales. Here, one CNN is applied on small image patches without down-scaling, while a second CNN is applied on larger down-scaled image patches. This multi-scale approach raises the challenge to combine the predictions of both networks. Therefore, this study presents a novel method to increase the segmentation accuracy by interpreting the network results to derive a final segmentation mask. This ensemble method consists of a CNN, which is applied on the predictions of the given patches from the overview images. The evaluation of this method comprises different pre-processing techniques regarding the logit outputs of the preceding networks as well as additional information such as RGB image data. Further, different network structures are evaluated, which include own structures specifically designed for this task. Finally, these approaches are compared against state-of-the-art network structures.