03/23/2022
By Gregory LeMasurier

The Kennedy College of Sciences, Department of Computer Science, invites you to attend a Master’s thesis defense by Gregory LeMasurier on “Shopping assistance for people who are blind or have low vision."

Thesis/Dissertation Title: Shopping assistance for people who are blind or have low vision
Candidate Name: Gregory LeMasurier
Degree: Master’s
Defense Date: Tuesday, April 5, 2022
Time: 11:30 a.m.
Location: Via Zoom and in Person: Shah Hall 306

Committee:

  • Holly A. Yanco (advisor), Professor, Computer Science Department, University of Massachusetts Lowell
  • Mohammad Arif Ul Alam, Assistant Professor, Computer Science Department, University of Massachusetts Lowell
  • Aaron Steinfeld, Associate Research Professor, Robotics Institute, Carnegie Mellon University

Brief Abstract:
According to the World Health Organization in 2019, approximately 2.2 billion people are blind or have visual impairments. People who are blind or have low vision (B/LV) rely on assistance from others as well as from assistive technology to complete many everyday tasks, which require visual sensory information.

We initially conducted a survey of devices for people who are B/LV which focused on the sensor packages used that enable indoor and outdoor sensing, the information communicated to the user through different feedback methods, and the tasks these devices are designed to assist people with. After discovering a need for object finding devices in our survey, we distributed a questionnaire to local agencies for people who are B/LV. Our first questionnaire focused on learning about assistive devices that are commonly used by this population, as well as challenges that they face while shopping. From this questionnaire, we found that there is a need for a device to assist with various tasks while shopping, including navigating to desired products, reading information on labels, and helping with tasks while purchasing items. In a follow up questionnaire, we asked questions to gather feedback on our initial design plans for this device including how beneficial they thought it would be, how the device should communicate information to them, and any privacy concerns they had with sharing data to build a map shared between users.

The system that we designed and implemented uses a smartphone and its built in sensors to provide shopping assistance, primarily using speech. Labels are read to identify products and to provide answers to questions users may ask regarding the products. Simultaneous Localization and Mapping (SLAM) is used to map the stores and provide navigation instructions to users. These maps are community updated, where all devices contribute data to ensure that the maps are kept up to date.