04/01/2021
By Xinzi Sun

Title: Colorectal Polyp Detection and Segmentation in Real-world Scenario

Ph.D. Candidate: Xinzi Sun
Date: Friday, April 16, 2021
Time: 10 a.m.
Location: This will be a virtual defense via Zoom.

Committee Members:

  • Yu Cao (advisor), Professor, Department of Computer Science
  • Benyuan Liu (advisor), Professor, Department of Computer Science
  • Hengyong Yu (member), Professor, Department of Electrical and Computer Engineering

Abstract:

Over the past decade, we have witnessed the rapid advancement of the Convolutional Neural Network (CNN). Nowadays, CNN has been widely applied for solving various problems in different domains, such as Computer Vision (CV) and Natural language processing (NLP). CNN has also demonstrated its huge potential in medical image analysis to help physicians make diagnoses. In this dissertation, we will investigate CNN-based image segmentation techniques for colorectal polyp detection and their applications in the real-world scenario.

Colorectal polyps are abnormal tissues growing on the intima of the colon or rectum with a high risk of developing into colorectal cancer, the third leading cause of cancer death worldwide. Early detection and removal of colon polyps via colonoscopy have proved to be an effective approach to prevent colorectal cancer. To help physicians detect polyps in colonoscopy procedures, we devise two CNN-based polyp detectors including a U-Net with Dilation Convolution detector and an Anchor Free Polyp (AFP-Net) detector. The U-Net with Dilation Convolution detector consists of an encoder to extract multi-scale semantic features and a decoder to expand the feature maps to a polyp segmentation map. We improve the feature representation ability of the encoder by introducing the dilated convolution to learn high-level semantic features without resolution reduction. The AFP-Net detector is a novel anchor-free detector that can localize polyps without using predefined anchor boxes. To further strengthen the model, we leverage a Context Enhancement Module and Cosine Ground truth Projection. Both of these two polyp detectors achieve state-of-the-art results on two mainstream public datasets.

While the aforementioned two approaches achieve excellent performance on public datasets, they do not perform well in real-world colonoscopy operations due to the significant difference between images in a real colonoscopy and those in the public datasets. Unlike the well-chosen clear images with obvious polyps in the public datasets, images from a colonoscopy are often blurry and contain various artifacts such as fluid, debris, bubbles, reflection, specularity, contrast, saturation, and medical instruments, with a wide variety of polyps of different sizes, shapes, and textures. All these factors pose a significant challenge to effective polyp detection in a colonoscopy. To this end, we collect a private dataset that contains 7,311 images from 336 complete colonoscopy procedures. This dataset represents realistic operation scenarios and thus can be used to better train the models and evaluate a system's performance in practice. We also propose an integrated system architecture that consists of a blurry image detector, U-Net with Dilation Convolution detector, and Anchor Free Polyp (AFP-Net) detector to address the performance reduction for polyp detection in real-world colonoscopy operations. Extensive experimental results show that our system can effectively detect polyps in real-world colonoscopy operations with excellent performance in a real-time fashion.

While the performance of the computer-aided colorectal polyp detection system has been significantly improved, there are still many remaining challenges for colorectal polyp detection and segmentation. Our future research direction would be to focus on instance segmentation and pathology prediction. The most common types of colorectal polyps are adenomatous, hyperplastic, and inflammatory. We will collect a colonoscopy dataset with both segmentation mask annotations and pathology information. We will also improve the network architecture by modifying the existing backbone, neck, or head structure to extract strong feature representation for polyp segmentation and pathology prediction.