This page features products that are the result of the UA-1 DECISIVE: Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations project by the University of Massachusetts Lowell. This project is sponsored by the Department of the Army, U.S. Army Combat Capabilities Development Command Soldier Center (W911QY-18- 2-0006).
DECISIVE Test Methods Handbook
This handbook outlines all test methods developed under the DECISIVE project for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. Specifications for 20 test methods are included:
Test Methods
- Non-Line-of-Sight (NLOS) Communications
- Non-Line-of-Sight (NLOS) Video Latency
- Interference Reaction
- Runtime Endurance
- Takeoff and Land/Perch
- Room Clearing
- Indoor Noise Level
- Logistics Characterization
- Operator Control Unit (OCU) Characterization
- Obstacle Avoidance and Collision Resilience
- Position and Traversal Accuracy
- Navigation Through Apertures
- Navigation Through Confined Spaces
- Indoor Mapping Resolution
- Indoor Mapping Accuracy
- Non-Contextual Autonomy Ranking
- Contextual Autonomy Ranking
- Characterizing Factors of Trust
- Interface-Afforded Attention Allocation
- Situation Awareness (SA) Survey Comparison
The DECISIVE Test Methods Handbook (v1.1) is available on arXiv:
- Adam Norton, Reza Ahmadzadeh, Kshitij Jerath, Paul Robinette, Jay Weitzen, Thanuka Wickramarathne, Holly Yanco, Minseop Choi, Ryan Donald, Brendan Donoghue, Christian Dumas, Peter Gavriel, Alden Giedraitis, Brendan Hertel, Jack Houle, Nathan Letteri, Edwin Meriaux, Zahra Rezaei Khavas, Rakshith Singh, Gregg Willcox, and Naye Yoni. "DECISIVE Test Methods Handbook: Test Methods for Evaluating sUAS in Subterranean and Constrained Indoor Environments, Version 1.1." arXiv preprint arXiv:2211.01801, November 2022.
DECISIVE Benchmarking Data Report
This report includes the results of performance benchmarking of a set of 8 small unmanned aerial systems (sUAS) using the test methods specified in the DECISIVE test methods handbook (see above). The following sUAS platforms were evaluated: Cleo Robotics Dronut X1P (P = prototype), FLIR Black Hornet PRS, Flyability Elios 2 GOV, Lumenier Nighthawk V3, Parrot ANAFI USA GOV, Skydio X2D, Teal Golden Eagle, and Vantage Robotics Vesper. The data contained in the report is the result of conducting over 230 tests in-house at the NERVE Center and off-site at Fort Devens and Muscatatuck Urban Training Center, and analyses of best-in-class systems per test method category.
The DECISIVE Benchmarking Data Report (results from Phase I) is available on arXiv:
- Adam Norton, Reza Ahmadzadeh, Kshitij Jerath, Paul Robinette, Jay Weitzen, Thanuka Wickramarathne, Holly Yanco, Minseop Choi, Ryan Donald, Brendan Donoghue, Christian Dumas, Peter Gavriel, Alden Giedraitis, Brendan Hertel, Jack Houle, Nathan Letteri, Edwin Meriaux, Zahra Rezaei Khavas, Rakshith Singh, Gregg Willcox, and Naye Yoni. "DECISIVE Benchmarking Data Report: sUAS Performance Results from Phase I." arXiv preprint arXiv:2301.07853, January 2023.
Publications
- Zahra Rezaei Khavas, Edwin Meriaux, Amin Majdi, and Paul Robinette. "Validating Drone Trust Testing in Navigation Deviation Cases in Simulation." arXiv preprint arXiv:2310.04794, October 2023.
- Edwin Meriaux, Jay Weitzen, and Adam Norton. "Simulation of the Effect of Correlated Packet Loss for sUAS Platforms Operating in Non-Line-of-Sight Indoor Environments." Drones, Volume 7, Issue 7, July 2023.
- Zahra Rezaei Khavas, Amin Majdi, S. Reza Ahmadzadeh, and Paul Robinette. "Human Trust After Drone Failure: Study of the Effects of Drone Type and Failure Type on Human-Drone Trust." In Proceedings of the 20th International Conference on Ubiquitous Robots (UR) 2023, pp. 685-692. IEEE, June 2023.
- Edwin Meriaux, Jay Weitzen, Naye Yoni, and Adam Norton. "Test Methodologies for Evaluating the Effectiveness of sUAS Communication Links for Operation in Indoor and Subterranean Environments." ASTM Journal of Testing and Evaluation, Volume 51, Issue 6, June 2023.
- Minseop Choi, and Thanuka L. Wickramarathne. "On the Use of Real-Time Multi-Object Detection and Classification for Automated Assessment of Operator Visual Attention Allocation." In Proceedings of the 26th International Conference on Information Fusion (FUSION) 2023, pp. 1-7. IEEE, June 2023.
- Ryan Donald, Peter Gavriel, Adam Norton, and S. Reza Ahmadzadeh. "Contextual Autonomy Evaluation of Unmanned Aerial Vehicles in Subterranean Environments." In Proceedings of the 9th International Conference on Automation, Robotics and Applications (ICARA) 2023, February 2023.
- Edwin Meriaux, and Kshitij Jerath. "Evaluation of Navigation and Trajectory-following Capabilities of Small Unmanned Aerial Systems." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST), IEEE, November 2022.
- Minseop Choi, John Houle, and Thanuka L. Wickramarathne. "On the Development of Quantitative Operator Situational Awareness Assessment Methods for Small-Scale Unmanned Aircraft Systems." In Proceedings of the 25th International Conference on Information Fusion (FUSION) 2022, pp. 1-8. IEEE, July 2022.
- Brendan Hertel, Ryan Donald, Christian Dumas, and S. Reza Ahmadzadeh. "Methods for Combining and Representing Non-Contextual Autonomy Scores for Unmanned Aerial Systems." In Proceedings of the 8th International Conference on Automation, Robotics and Applications (ICARA) 2022, pp. 135-139. IEEE, February 2022.
- Adam Norton, Peter Gavriel, Brendan Donoghue, and Holly Yanco. "Test Methods to Evaluate Mapping Capabilities of Small Unmanned Aerial Systems in Constrained Indoor and Subterranean Environments." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST) 2021, IEEE, November 2021.
- Minseop Choi, John Houle, Nathan Letteri, and Thanuka L. Wickramarathne. "On The Use of Small-Scale Unmanned Autonomous Systems for Decision-Support in Subterranean Environments: The Case of Operator Situational Awareness Assessment." In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST) 2021, IEEE, November 2021.
Advancing and Evaluating UAS for Defense
The capstone event for the DECISIVE project, "Advancing and Evaluating UAS for Defense," was held on October 12, 2022. The event feature research briefs on the UA-1 and UA-2 projects funded by the UMass Lowell (UML)/DEVCOM HEROES program. All presentations given at the event are available for download via a publicly accessible Google Drive folder. Below is a list of the presentations given:
- HEROES Program Overview, Christopher Drew
- UA-1 Project Review: DECISIVE: Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations, Adam Norton
- UA-2 Project Review: Drone-Based Wireless Sensor Network for Multimodal Underground Situational Awareness, Xuejun Lu
- UMass Lowell UAS Project Transition, Adam Norton
- Test Methodologies to Evaluate UAS Communication and the Effects of Packet Loss in NLOS Indoor Environments, Jay Weitzen
- Embedding IR Sensors to Increase UAS Situational Awareness, Xuejun Lu
- Artificial Intelligence for Image Recognition During UAS Operations, Yuanchang Xie
- Methods for Combining and Representing Autonomy Ranking Scores of UAS, Reza Ahmadzadeh
- Characterizing and Evaluating Field Readiness of UAS for Subterranean and Indoor Environments, Adam Norton
- Evaluation of Indoor Navigation and Obstacle Avoidance and Resilience Capabilities of UAS, Kshitij Jerath
- Field Testing of UAS Navigation Through Confined Spaces and Apertures, Adam Norton
- Test Methods to Evaluate Indoor and Subterranean Mapping Capabilities of UAS, Adam Norton
- Trustworthy and Safe Human-UAS Interaction for Subterranean Mapping, Paul Robinette
- Assessing Operator Situational Awareness of UAS for Decision-Support in Subterranean Environments, Thanuka Wickramarathne