Most artificial intelligence (AI) systems are programmed to adapt to individual human users, aligning with their values and reinforcing their beliefs and decisions – right or wrong.

Such alignment is considered important to foster human trust in AI agents. But Criminology Professor Neil Shortland says AI alignment can be a handicap in high-stakes decision-making, where thoughtful disagreement can lead to better outcomes. 

“Nowhere in the study of foreign policy disasters is there someone who complained there was too much diversity of views,” Shortland says. On the other hand, “You can find lots of disasters where a lack of diversity of viewpoints was a huge problem.”

Now, he and three other UMass Lowell faculty members have won a $971,874 grant from the Army Research Office to explore what happens when AI is programmed to “misalign” with its human users, based on previous research results that found considering opposing points of view leads to better decisions.

“You don’t always want alignment,” Shortland says. “Task conflict is usually a good thing. Where it’s a bad thing is when we have task conflict, but we also dislike each other.”

Shortland, a forensic psychologist who is the principal investigator on the grant, says that the research team will also look at human personality traits and cognitive styles that can lead to better or worse decision-making in tandem with AI.

“Not all people will react equally to misalignment. Some people will ignore AI, some people will say, ‘OK, we’ll do it your way because you’re super-smart,’ and some will say, ‘Let’s figure this out together,’” he says. “We are interested in what is it about someone that explains why they behave this way, and can you train that into an algorithm to improve their partnership?”

To try to answer that question, Shortland and his colleagues – Electrical and Computer Engineering Associate Professor Paul Robinette, Mechanical Engineering Assistant Professor Richard Nuckols and Computer Science Assistant Professor Samantha Reig, all of whom do research into human-robot interactions – plan to develop realistic scenarios with equal and conflicting priorities.

For example, they might create a simulation in which a field commander has to decide how to balance the imperative to rescue wounded American soldiers with the need to capture a high-value opponent with important information, Shortland says.

“There’s no right and wrong in there,” he says. “There is individual difference in what’s the first thing you should be doing and what’s the most important thing for you to accomplish.”

In the absence of other personnel of equal rank and experience with whom to hash out differences, having an AI partner that balances the decision-maker’s cognitive style and preferences instead of mimicking them could result in a higher likelihood of achieving both goals, he says.

“If I’m a person who makes fast and crude decisions, I want pushback from someone who’s slow and evaluative, because that’s what I don’t do well,” Shortland says. In such situations, he adds, “Misalignment is a complement.”

While AI agents do need to be aligned with larger organizational rules and priorities – minimizing civilian deaths, for example – it’s important to understand how and why different people react when an AI agent is programmed to challenge them, Shortland says.

“What happens when you force people to interact with AI when it’s programmed to do the opposite of what they want?” he asks.

The research team will also look at how people in different roles could be affected by the use of AI agents: the field commander who relies on it to make a decision, the soldier who must carry out the order even if she doesn’t agree with it, and the general who does the after-action review.

“We want to think about it within an organizational system,” he says.