Ph.D. Dissertation Proposal Defense in Computer Science: Zubin Bhuyan 7/19

Abstract:
Intelligent Transportation Systems (ITS) encompass a broad range of technologies aimed at improving the safety, efficiency, and sustainability of transportation and infrastructure. These systems manage traffic flow, enhance safety, and improve public transit efficiency through innovations such as real-time tracking and scheduling. Additionally, ITS rely on the continuous collection and intelligent analysis of data, enabling proactive management and informed decision-making in response to dynamic traffic conditions. Leveraging deep learning, significant advancements have been made in the areas of traffic control, congestion reduction, and autonomous driving technologies, thus enhancing system efficiency and safety. However, there remains considerable untapped potential for deep learning within ITS, particularly in handling complex environments and meeting the increasing safety and equity requirements of modern transportation systems.

This doctoral thesis proposal focuses on applying deep learning and computer vision (CV) techniques to a comprehensive set of datasets, including both ground-level and aerial imagery in RGB and thermal modalities. The primary aim is to enhance vehicle detection, tracking, and analysis, particularly focusing on highway traffic. Data collection includes multiple locations in Massachusetts and New Hampshire, covering various general traffic environments and specifically targeted work zones to improve the dataset’s relevance to real-world scenarios. Notably, three of these locations include active work zones with lane closures, two of which are nighttime work zones. These specific sites provide both aerial datasets, captured via drones, and ground-level thermal footage. The aerial datasets are annotated with Oriented Bounding Boxes (OBB), while segmentation masks are used in the ground-level thermal videos. We developed three key datasets: the MA Highway Aerial Dataset, the MA Nighttime UAV Thermal Dataset, and the MA-NH Highway Thermal Dataset. Each dataset includes vehicle annotations as well as segmentation of roadways and various infrastructure elements, such as channelizer drums and cones.

A novel deep learning model was developed for segmenting and detecting vehicles using OBBs, along with an OBB tracking algorithm that extracts vehicle trajectories in aerial videos. Additionally, we introduced a CNN architecture, based on YOLOv8 enhanced with Group Shuffle Convolution modules, which significantly improved the detection and segmentation performance in thermal drone videos. Furthermore, we developed models to segment and analyze roadways and related infrastructures, such as highways, exit lanes, and channelizer drums. Leveraging these models enables accurate identification of vehicles near work zones and detection of those making last-minute lane changes in active work zones with lane closures. This comprehensive strategy of simultaneously segmenting roadways and vehicles not only yields more detailed information about highway and work zone scenarios, but also enhances the predictive capabilities of our system. Such improvements can potentially increase the responsiveness of traffic management systems in real-time scenarios.

Moving forward, our efforts will be directed towards enhancing the effectiveness of our deep learning pipelines, with a specific focus on improving detection performance. This enhancement involves implementing a modified mosaic data augmentation strategy that utilizes segmentation-mask areas and targets “hard examples” based on missed detections in tracked trajectories. Additionally, we propose an enhancement to the Slicing Aided Hyper Inference (SHAI) framework, which we have named Selective-SHAI. This approach is designed for rapid inference and includes a fine-tuning pipeline optimized for the detection of small objects. It specifically concentrates on relevant regions, such as highways in our case. Further efforts will include expanded trajectory analysis to demonstrate how these results can facilitate a deeper understanding of driver behavior, such as merging analysis and the identification of risky driving behaviors.