11/10/2023
By Danielle Fretwell

The Francis College of Engineering, Department of Electrical and Computer Engineering, invites you to attend a Doctoral Dissertation defense by Shilong Wang on: GPU Accelerated Latent Dirichlet Allocation.

Candidate Name: Shilong Wang
Degree: Doctoral
Defense Date: Wednesday, Nov. 22, 2023
Time: 9-10:30 a.m.
Location: Perry Hall 215

Committee:

  • Advisor: Hengyong Yu, Professor, Electrical and Computer Engineering, University of Massachusetts Lowell
  • Yu Cao, Professor, Computer Science, UMass Lowell
  • Yan Luo, Professor, Electrical and Computer Engineering, UMass Lowell
  • SeungWoo Son, Associate Professor, Electrical and Computer Engineering, UMass Lowell

Brief Abstract:
Latent Dirichlet Allocation (LDA) is a statistical approach for topic modeling with a wide range of applications. LDA can be subdivided into flatten model and hierarchical model. Graphics Processing Units (GPUs) exhibit significant advantages over CPUs owing to their exceptional computing and memory throughput capabilities. This dissertation presents two GPU-based LDA methods: ezLDA and GHLDA, which corresponds to flatten model and hierarchical model respectively. EZLDA achieves efficient and scalable LDA training on GPUs with the following three contributions: First, ezLDA introduces three-branch sampling method which takes advantage of the convergence heterogeneity of various tokens to reduce the redundant sampling task. Second, to enable sparsity-aware format for both D and W on GPUs with fast sampling and updating, we introduce hybrid format for W along with corresponding token partition to T and inverted index designs. Third, we design a hierarchical workload balancing solution to address the extremely skewed workload imbalance problem on GPU and scale ezLDA across multiple GPUs. Taken together, ezLDA achieves superior performance over the state-of-the-art attempts with lower memory consumption. Hierarchical Latent Dirichlet Allocation (HLDA) is an enhanced version of the conventional Latent Dirichlet Allocation (LDA) text model, designed to uncover underlying semantic patterns in text corpora. The HLDA introduces a hierarchical structure that captures topic hierarchies across various levels of detail, resulting in more precise outcomes. However, the complexity of the tree data structure and the dynamically growing memory requirements present obstacles to the scalability of HLDA when implemented on GPUs. This dissertation introduces a GPU-based HLDA (GHLDA), first known-to-all GPU-based implementation of HLDA. Our innovations are twofold: we implement the HLDA on GPU at a systematic level; and we handle the complex tree structures by matrix on GPU.