11/17/2021
By Xinzi Sun

Title: Colorectal Polyp Detection and Segmentation in Real-world Scenario

Ph.D. Candidate: Xinzi Sun
Time: Thursday, Dec. 2, 2021, 2 p.m.
Location: This will be a virtual defense via Zoom

Committee Members:

  • Benyuan Liu (advisor), Professor, Department of Computer Science
  • Yu Cao (advisor), Professor, Department of Computer Science
  • Hengyong Yu (member), Professor, Department of Electrical and Computer Engineering

Abstract:

Over the past decade, we have witnessed the rapid advancement of the Convolutional Neural Network (CNN). Nowadays, CNN has been widely applied for solving various problems in different domains, such as Computer Vision (CV) and Natural language processing (NLP). CNN has also demonstrated its huge potential in medical image analysis to help physicians make diagnoses. In this dissertation, we will investigate CNN-based object detection and segmentation techniques for colorectal polyp detection and segmentation and their applications in real-world scenarios.

Colorectal polyps are abnormal tissues growing on the intima of the colon or rectum with a high risk of developing into colorectal cancer, the third leading cause of cancer death worldwide. Early detection and removal of colon polyps via colonoscopy have proved to be an effective approach to prevent colorectal cancer. To help physicians detect polyps in colonoscopy procedures, we devise two CNN-based polyp detectors including a U-Net with Dilation Convolution detector and an Anchor Free Polyp (AFP-Net) detector. The U-Net with Dilation Convolution detector consists of an encoder to extract multi-scale semantic features and a decoder to expand the feature maps to a polyp segmentation map. We improve the feature representation ability of the encoder by introducing the dilated convolution to learn high-level semantic features without resolution reduction. The AFP-Net detector is a novel anchor-free detector that can localize polyps without using predefined anchor boxes. To further strengthen the model, we leverage a Context Enhancement Module and Cosine Ground truth Projection. Both of these two polyp detectors achieve state-of-the-art results on two mainstream public datasets.

While the aforementioned two approaches achieve excellent performance on public datasets, they do not perform well in real-world colonoscopy operations due to the significant difference between images in a real colonoscopy and those in the public datasets. Unlike the well-chosen clear images with obvious polyps in the public datasets, images from a colonoscopy are often blurry and contain various artifacts such as fluid, debris, bubbles, reflection, specularity, contrast, saturation, and medical instruments, with a wide variety of polyps of different sizes, shapes, and textures. All these factors pose a significant challenge to effective polyp detection in a colonoscopy. To this end, we collect a private dataset that contains 7,311 images from 336 complete colonoscopy procedures. This dataset represents realistic operation scenarios and thus can be used to better train the models and evaluate a system's performance in practice. We also propose an integrated system architecture that consists of a blurry image detector, U-Net with Dilation Convolution detector, and Anchor Free Polyp (AFP-Net) detector to address the performance reduction for polyp detection in real-world colonoscopy operations. Extensive experimental results show that our system can effectively detect polyps in real-world colonoscopy operations with excellent performance in a real-time fashion.

The most common types of colorectal polyps are inflammatory, hyperplastic, and adenomatous. The adenomatous polyp is the most dangerous type of polyp. About 70 percent of all polyps are adenomatous, making it the most common type of colon polyp. However, there is no public colonoscopy dataset with both localization annotations and pathology information. To this end, we collect a large-scale polyp pathology dataset. The dataset contains 6,059 images with object-level bounding boxes, segmentation, and three-category pathology annotations. Additionally, we propose a multi-branch convolutional network for polyp detection and classification. It consists of an anchor-free polyp detection branch for detecting and localizing polyps and a classification branch for global feature extraction and pathology prediction. We also propose a foreground attention module to generate local features that are combined with the global feature in the classification branch to enhance the pathology prediction performance.