07/17/2023
By Amin Majdi

The Kennedy College of Sciences, Department of Computer Science, announces the master's thesis defense of Amin Majdi entitled "Enhancing Human-Exoskeleton Performance through Real-Time Parameter Tuning: A Reinforcement Learning Framework for Assistive Upper-Limb Assistive Robots."

Date: Tuesday, July 25, 2023
Time: 3 p.m. Eastern Time 
Location: Dandeneau 309, North Campus and via Zoom

Committee Members:

  • Reza Ahmadzadeh (Advisor)
  • Holly Yanco (Co-Advisor)
  • Maru Cabrera (Member)

Abstract:

Upper-limb assistive robots provide a promising solution for enhancing the daily lives of individuals with disabilities and the elderly in regaining independence and performing daily tasks. Despite notable advancements in Upper-limb assistive robot developments, several technological and theoretical challenges still need to be addressed. One of the most significant challenges is using preset parameters that cannot be updated in real-time and hence fail to accommodate the dynamic requirements of individual users. This thesis addresses the real-time parameter tuning challenge by developing an innovative model-based adaptive learning framework. This framework enables real-time adaptation of robot parameters to align the robot movements more closely with the human user's intentions. Consequently, this alignment enables a more personalized and efficient interaction with the robotic aid. The first step in developing our learning framework included the construction of an accurate dynamic model of the system. We used muscle efforts (i.e., human action), the robot configuration parameters (i.e., robot action), and the robot's angle and angular velocity as inputs to our supervised learning approach. We implemented six methods for constructing the dynamic model of the system. Our comparisons revealed that XGBoost resulted in a more accurate model with better prediction capabilities. We performed real-time data collection and synchronization and incorporated a Data Acquisition System (DAQ) into the robot to achieve real-time data synchronization, feeding directly into our learning framework. We integrated two pressure sensors into the device using an electronic circuit to measure the alignment between the robot's actions and human intentions. These sensors provide a signal to assess the alignment between the robot's movement and the user's intended action. Our framework uses the learned robot dynamic model together with a reward function in a model-based Reinforcement Learning setup that receives inputs from the actual robot for tuning parameters on the fly. We performed real-world experiments with the MyoPro robot, which is an EMG-based assistive robot on a pick-and-place task designed to assess the efficacy of our framework. Our experiments consist of 212 separate repetitions of the pick-and-place task, collectively generating approximately 64 minutes of data at a sampling rate of 10 Hz. Our results show the distinct advantage of our learning framework over the robot's default fixed policy, as evidenced by the cumulative reward collected during the task execution. Furthermore, we discovered that pre-training the framework using simulated human inputs and pressure signals reduces the required interaction time for real-time tuning and slightly improves the algorithm's overall performance.