Artificial Intelligence (AI) has a growing influence on modern warfare, affecting both the strategic and active capabilities of military personnel on the ground or remotely. Able to be configured into systems with considerable autonomy and limited human oversight, the range of applications for AI includes reconnaissance missions, fire support, and finding optimal routes for assisting medical evacuations. AI is currently being used to considerable effect in conflicts in Ukraine and Gaza [1].
One form of AI system with significant influence in current conflicts is vision AI which focuses on analysing real-time visual data. Vision AI systems can recognise and classify (i.e. add a label to) objects in images and video feeds on a screen [2]. Examples of object detection and action recognition include the identification of weapons, vehicles, buildings and persons. Some of these vision AI systems can detect and classify protected symbols like the Red Cross, or to capture gestures like ‘hands-up’ [3]. Military vision AI systems can also distinguish between military and civilian objects or things that otherwise deserve special attention.
While vision AI systems can increase speed and precision of military decision-making, their influence on ethical decision-making is still unclear. In Occasional Paper 40, we aim to uncover how vision AI systems affect the critical judgments military personnel makes in response to military scenarios. Interestingly, our research indicates that AI may neither help nor hinder efforts to make ethical battlefield decisions.
Ongoing Efforts to Regulate Military AI
With AI’s increasing presence on the battlefield, there have been growing efforts to regulate its use. These efforts are being led, for example, by the ‘United Nations’ Convention on Certain Conventional Weapons Group of Governmental Experts on Lethal Autonomous Weapons Systems’ and the ‘Responsible AI in the Military Domain’ framework which Australia endorses and in which it actively participates. Further, several countries and international organisations (including NATO) have established ethical guidelines for the use of AI in warfare. These frameworks focus on ensuring that military AI not only operates within the boundaries of international humanitarian law, but is safeguarded by principles such as ‘governability’, ‘traceability’, ‘trustworthiness’, ‘reliability’, etc. Despite these crucial efforts and guidelines, there remains a gap in the empirical data and actual, real-world insights that help illuminate the effects of AI on military ethics and decision-making. High-level ethical guidelines need to be grounded and informed by such data if accurate assessments are to be made as to the opportunities and challenges inherent in the military use of AI systems.
Our Research on AI and Ethical Decision-Making
The authors’ research work at the ‘Military Ethics Research Lab and Innovation Network’ within The University of New South Wales, Canberra, seeks to provide this much-needed empirical insight. This work focussed on vision-based AI systems that assist military personnel to make rapid decisions in response to critical situations. In conducting the research, our aim was to test concerns that users of AI may suffer ‘ethical deskilling’. This is a term coined by Shannon Vallor meaning that soldiers can become too dependent on the system—in this case the AI—gradually losing their ability to make complex ethical judgments. While ethical deskilling is a risk, others have argued that ethical aspects of conflict become more (rather than less) salient to operators who work with emerging technology [4], so much so that their risk of moral injury can be aggravated.
In our research, we tested the effects of vision AI systems in response to scenarios to understand their impact on decision makers. To do this, we pre-recorded various video clips of military personnel, vehicles and objects to create a series of ‘shoot/no-shoot’ decision-making scenarios. These scenarios were overlayed with AI-annotated labels/prompts in a similar fashion to modern vision-based AI systems. We then set up an experiment, with volunteers participating at a computer screen that mimicked a remote weapons system interface. Participants (comprising military trainee officers) were presented with a series of scenarios to elicit their decisions about whether or not to fire a weapon. Across the series of scenarios, various factors were systematically varied including the device an unknown person was carrying (e.g. a rifle or a camera), that person’s type of clothing (civilian, military) and the AI labelling. Across the scenarios, the AI labels were presented either correctly or incorrectly or were not shown at all. This experimental design allowed us to measure whether/how an AI system influences ‘shoot-no shoot’ decisions.
A total of 54 volunteers participated in the first round of lab experiments. Results from this round showed that the AI labelling had no effect on participants’ decisions on whether or not to shoot. The research team considered that this outcome may have been caused by the fact that the video clips offered clear vision of the scenarios, making the decisions unambiguous. Hence, a second round of experiments was conducted in which the clarity of the video clips was reduced. Twenty-seven volunteers participated in this second round.
Key Learnings
The main finding of the experiments was that the type of AI labelling (whether the label was correct, incorrect, or not shown at all) did not affect whether the participant made a ‘correct’ firing decision. In other words, participants relied on their own interpretation of what happened in the video clips (not the AI label) in deciding whether to engage a potential threat with lethal force. This was the case in both rounds of experiments, i.e. reducing vision clarity did not make a difference to the lack of effect of AI labelling. Participants did make, however, incorrect decisions on some occasions. For example, many participants ‘shot’ a person holding a long-lens camera who was not a threat in the scenario. In our research, we tentatively concluded that participants confused the long-lens camera with a rifle (shown in other clips).
Relevance to the Australian Army
As evidenced by conflicts in Ukraine and Gaza, AI is rapidly becoming integral to military decision-making, and vision-based AI is at the forefront of this transformation. The fact that the International Committee of the Red Cross devoted a chapter on vision AI in their recent publication on the topic (ICRC 2024) underscores the growing international interest in vision AI. For the Australian Army, understanding how military personnel interacts with AI in real-world situations will be essential to inform training, operations, and for maintaining compliance with international humanitarian law. The findings of our initial research shows that vision AI has no grand effect on decision making and further research is needed to generate in-depth insights into how AI systems may shape the future of ethical decision-making on the battlefield. Such research is needed to ensure that military personnel not only rely on AI for tactical advantage, but do so without compromising the ethical standards that govern military operations.
Endnotes
[1] Anna Nadibaidze e.a., AI in Military Decision Support Systems A Review of Developments and Debates (University of Southern Denmark, Center for War Studies, 2024); Elke Schwarz, ‘Gaza War: Israel Using AI to Identify Human Targets Raising Fears That Innocents Are Being Caught in the Net’, The Conversation, 12 april 2024, http://theconversation.com/gaza-war-israel-using-ai-to-identify-human-targets-raising-fears-that-innocents-are-being-caught-in-the-net-227422; Bianca Baggiarini, ‘Algorithmic War and the Dangers of In-Visibility, Anonymity, and Fragmentation’, Australian Journal of International Affairs 78, nr. 2 (2024): 257-65, https://doi.org/10.1080/10357718.2024.2333824.
[2] Jim Gallagher en Edward Oughton, ‘Transforming the Multidomain Battlefield with AI’, Military Review, september 2024, https://www.armyupress.army.mil/Journals/Military-Review/Online-Exclusive/2024-OLE/Multidomain-Battlefield-AI/.
[3] S. Kate Devitt e.a., ‘Developing a Trusted Human-AI Network for Humanitarian Benefit’, Digital War 4, nr. 1 (2023): 1-17, https://doi.org/10.1057/s42984-023-00063-y.
[4] Lieutenant Colonel Wayne Phelps, On Killing Remotely: The Psychology of Killing with Drones (Hachette, 2021); S. Kate Devitt, ‘Bad, Mad, and Cooked’, in Responsible Use of AI in Military Systems, 1ste dr., door Jan Maarten Schraagen (Chapman and Hall/CRC, 2024), https://doi.org/10.1201/9781003410379-16.
