Engineering and Physical Therapy Students Collaborate on Augmented Reality for Hand Rehabilitation

Let’s face it: Physical therapy would be a lot more fun and effective if it felt like playing a game. 

To achieve that, clinicians are increasingly looking at the possibility of using augmented reality (AR) – technology that enhances real-world experiences by adding digital information including images, text or sound to physical therapy sessions. The technology has the potential to deliver remote care with engaging, measurable movement experiences that clinicians can monitor and evaluate in real time. 

However, for AR to be effective in practice, it must measure movement accurately and in ways that matter clinically. 

A team of engineering and physical therapy faculty and students is collaborating to address that challenge. The group is conducting research to evaluate the accuracy of how AR technology tracks joint motion and hand gestures.

“This research takes a holistic view to enable the research on AR application for physical therapy to be applied not only in a clinical setting, but ultimately within the larger health ecosystem of both the clients and the health care providers,” says Professor of Electrical and Computer Engineering Kavitha Chandra, who is advising the student researchers. 

Gayathri Boopathy, who initiated the research for her master’s thesis in the computer engineering program, is continuing the research for her doctoral dissertation. The team also includes Ph.D. electrical engineering candidates Emi Aoki and Flore Stecie Norceide as well as Ravi Kkumaar Sivasankar Venkata, who is pursuing a master’s in computer science. 

To incorporate clinical expertise, the engineering students partnered with Associate Professor Erika Lewis of the Department of Physical Therapy and Kinesiology, who is a physical therapist and certified hand therapist, and Kyle Callahan, a doctoral student in the physical therapy program. 

“Collaboration with physical therapists and students is critical, because their domain expertise helps to ensure the technology addresses real clinical needs, is safe and practical and aligns with rehabilitation goals,” says Boopathy. “This interdisciplinary approach significantly strengthens both the research design and outcomes.” 

At Rehab Health 360, a neurological and orthopedic physical therapy clinic in Acton, the research team evaluated the accuracy of the technology among clients with injured and non-injured hands. The clients wore augmented reality headsets with a hand-tracking device. Surrounded by cameras, motion and light sensors, they completed activities while the technology overlaid virtual markers on their finger joints, the center of their wrists and their fingertips to measure range of motion. 

Lewis and Callahan manually measured the range of motion in each joint in each finger using a goniometer as clients held positions such as a fist, claw and open hand with straight, extended fingers. This data provided a benchmark that the engineering students used to compare and validate the accuracy of the augmented reality system’s digital measurements. The research results found that the technology can capture meaningful movement data, an essential first step before extending the study to a larger group. 

“While augmented reality gamification devices can create a more motivating experience for the patient in the clinic and at home, the system that the UML students developed has the potential to also measure how much of an increase in range of motion patients have gained and if they are moving in a better functional pattern,” says Lewis. 

For Callahan, the opportunity to participate in a research study has enriched his training as a physical therapist. 

“I wanted to get more involved in research because it’s such a vital part of our decision-making process as physical therapists,” he says. “I appreciated the willingness of the engineering students to learn about physical therapy and the hand's anatomy.”