03/31/2023
By Dalila Megherbi
James Gallagher will defend his MS thesis in Computer Engineering, titled “Finding Neural. Network Lottery Tickets with Smallest Delta Pruning and Analysis” on Monday, April 10, 2023, at 11:30 a.m.
Location: This will be a virtual defense via Zoom. Those interested in attending should contact Committee Chair Dalila_Megherbi@uml.edu at and james_gallagher1@student.uml.edu at least 24 hours before the defense to request access to the meeting.
Committee Chair: D. B. Megherbi, ECE Department, (CMINDS), UML, Thesis Advisor
Committee Members:
- Xuejun Lu, ECE Department, UML
- Kanti Prasad, ECE Department, UML
Abstract:
Large neural networks are not readily deployable to applications with limited resources and are costly to train in terms of time, computational resources, and energy. Neural network pruning provides a promising pathway to creating sparse subnetworks that require substantially fewer computational resources to train and run but are still able to achieve high levels of performance. A fairly new Lottery Ticket Hypothesis posits that large, dense neural networks have a sparse subnetwork that is able to meet or exceed the accuracy of the original, unpruned network once trained. In this thesis, we investigate novel alternative methods for finding these subnetworks. We propose the smallest delta pruning that outperforms simple iterative magnitude pruning for a LeNet-type network using the MNIST dataset. We also test the effectiveness of the method on a modified form of VGG-19 with the CIFAR-10 dataset. We are able to train the VGG-19 network to within 2.3% of target test accuracy at 8% sparsity in the worst case.
All interested students, faculty members, and staff are invited to attend the online defense via remote access.