Research Aims to Help AI Gauge Job Performance, Problems

Prof. Holly Yanco with a Baxter robot
UMass Lowell Computer Science Prof. Holly Yanco poses with a "Baxter" robot, which will be used in her research to develop in autonomous systems the ability to evaluate their job performance.


Contact: Nancy Cicco, 978-934-4944 or and Christine Gillette, 978-934-2209 or

LOWELL, Mass. – Robots can help do a lot of things – assemble cars, search for explosives, cook a meal or aid in surgery. But one thing they can’t do is tell you how they’re doing – yet.

UMass Lowell researchers are aiming to change that, working to give humanoid robots and other autonomous systems the ability to evaluate how well they can perform tasks or why they can’t complete jobs. This assessment in real time is vital as robots become increasingly independent and are tasked with performing in remote, hostile or unpredictable environments with minimal human supervision or intervention. 

“Right now, robots can’t gauge how well they are able to perform tasks, how a job is progressing or tell you what their limitations or capabilities are. Our goal is to develop methods and metrics that would enable autonomous systems to assess their own performance,” said UMass Lowell Computer Science Prof. Holly Yanco, a renowned robotics expert who established the university’s Robotics Lab and the New England Robotics Validation and Experimentation (NERVE) Center at UMass Lowell, one of the country’s most advanced robotics testing facilities. 

The project – called SUCCESS, which stands for Self-assessment and Understanding of Competence and Conditions to Ensure System Success – is funded with a $7.5 million grant from the U.S. Department of Defense to UMass Lowell and three other institutions that will be working in tandem on the initiative over the next five years.  

Yanco – whose exemplary teaching, research and other contributions to the campus community earned her the designation of UMass Lowell Distinguished University Professor – is leading the university’s research on SUCCESS. 

For the project, Yanco and her colleagues will evaluate the performance of a pair of “Baxter” robots that will complete assembly tasks, problem-solving scenarios and games. Each robot is a two-armed, humanoid machine that is 3 feet tall and can stand more than 6 feet tall when attached to its base. A computer screen on each device enables it to display facial expressions while it completes tasks. Researchers working at the NERVE Center and Carnegie Mellon University’s Robotics Institute will put the robots through their paces, testing their ability to maneuver around obstacles, to find and investigate hidden items and to manipulate objects to open them. 

The team will then build a software database that lays out all of the variables the robots could encounter and ways they could execute tasks based on their previous behavior. By looking at the robots’ track record, researchers hope to predict how well they will perform in the future. The data could then be used by operators in the field to help them anticipate how the machines will behave and to design and build the next generation of enhanced robotics.

“Hopefully, the study will lead to better human-robot teamwork and increase the level of trust, expectation and efficiency between the two,” Yanco said. 

Other institutions participating in the research include Brigham Young University and Tufts University. 

UMass Lowell is a national research university located on a high-energy campus in the heart of a global community. The university offers its more than 18,000 students bachelors, masters and doctoral degrees in business, education, engineering, fine arts, health, humanities, sciences and social sciences. UMass Lowell delivers high-quality educational programs, vigorous hands-on learning and personal attention from leading faculty and staff, all of which prepare graduates to be ready for work, for life and for all the world offers.