Industry Development of Military AI
Operationalising International Humanitarian Law Across the Lifecycle
Author: Dr Taylor Kate Woodcock
Introduction
Military innovations today are driven by technology companies. While the military-industrial complex is by no means new, the advent of data-driven applications of Artificial Intelligence (AI) – such as machine learning (ML) and large language models (LLMs) – has brought these companies to the fore in unprecedented ways. Whilst the wide array of contemporary research on AI acknowledges that algorithms ‘shape how we understand the world and they do work in and make the world through their execution as software, with profound consequences’,[1] it is crucial to also examine who shapes these technologies. For militaries integrating AI capabilities, this question has legal implications. The choices made by industry actors in developing these systems shape their technical function, how they can be used, and the military practices in which they are situated, with direct consequences for compliance with international humanitarian law (IHL). This spans a wide range of interconnected, multi-purpose systems that support various capabilities, from intelligence data fusion and analysis, to target production and civilian detection, operational decision support, and logistical processing.
The development of AI capabilities is a distributed process involving myriad industry and military actors in various roles across different stages of the lifecycle. Whilst States may and increasingly do rely on industry for the development, evaluation, and maintenance of military AI systems, the concentration of knowledge and technical expertise within the private sector creates a structural challenge for discharging IHL obligations.
This article develops this argument in two parts. The first explores the role of industry in developing AI capabilities and the implications this holds for States’ fulfilment of obligations under IHL. It argues that given the central role of industry to the development of military AI capabilities and that choices by industry actors shape the technical function of these capabilities, how they can be used, and the broader practices in which they are situated, States have a legal obligation to operationalise IHL in development practices throughout the lifecycle. However, this obligation risks being constrained by dependency on the knowledge and technical expertise concentrated within industry, requiring efforts to ensure independence and capacity to respect IHL. The second traces this argument concretely across selected lifecycle stages, demonstrating how development choices by industry actors can be a source of risk for IHL violations and constrain how States are able to fulfil their due diligence obligations. For military organisations navigating relationships with industry, the ability to conduct operationally sound and lawful operations is at stake, requiring critical attention to not only how AI capabilities are used, by also how they are developed.
States’ Obligations to Operationalise IHL in Industry-Led Development of Military AI
This section explores the role of industry in developing AI capabilities and the implications this holds for States’ fulfilment of obligations under IHL. It argues that given the central role of industry to the development of military AI capabilities and that choices by industry actors shape the technical function of these capabilities, how they can be used, and the broader practices in which they are situated, States have a legal obligation to operationalise IHL in development practices throughout the lifecycle, an obligation constrained by dependency on the knowledge and technical expertise of industry. First, section 2.1 outlines how industry is central to the development of military AI capabilities, though this is heterogenous and diffuse, with corporations of different sizes and activities contributing in diverse ways. Next, section 2.2 demonstrates how AI capabilities are the product of development practices in which diverse private actors iteratively make choices that shape the technical function of systems, how they can be used, and the broader practices in which they are situated, with implications for the lawfulness of military operations. As development choices by industry shape AI capabilities and military operations more broadly, section 2.3 argues that States are legally obligated to operationalise IHL throughout development practices across the entire lifecycle. Finally, section 2.4 highlights that State dependency on industry knowledge and technical expertise risks constraining their ability to ensure IHL compliance throughout the lifecycle and must structure the relationship with industry to ensure the independence and capacity to operationalise IHL throughout development.
Industry and the Development of Military AI Capabilities
Today, industry has become central to the development of military AI capabilities, though its role is heterogenous and diffuse with corporations of different sizes and activities contributing in diverse ways. AI itself is a broad umbrella term capturing a range of technologies. The companies involved in the development of military AI are therefore not homogenous but vary in size and role.[2] The development of military AI involves not just large multi-national corporations, but also small- and medium-sized enterprises (SMEs), often being funded by venture capital.[3] This ranges from the behemoths Google and Microsoft, Palantir and OpenAI, to much smaller start-ups such as Anduril, Shield AI, and Skydio.[4] It also encompasses a number of different types of tech companies, including data companies, platform providers, model and software developers, and cloud services. Amid these various types of companies involved in the development of military AI, there are also different models of development. This includes the co-development of AI systems in collaboration with militaries, the production of products sold off-the-shelf, or the filtering down and redevelopment of civilian products into the military domain.
The scale of State investment in military AI at present is significant. A 2024 Brookings Institute study found that US Defense spending for AI increased threefold in 2022-2023 with a 1200% increase in the value of contracts.[5] Most recently, this year Palantir entered an enterprise agreement worth up to $10 billion by the US army, which consolidates 75 contracts to provide ‘rapid access to cutting-edge data integration, analytics, and AI tools’.[6] Anduril was likewise awarded a $99.9 million contract by the US Army for a ‘next generation command and control prototype’.[7] This year, NATO also acquired Maven Smart System from Palantir.[8] The growing prominence of LLMs with the promulgation of Open AI’s ChatGPT series has also garnered interest in defence, with the launching of Project Lima in the US to develop military applications of LLMs,[9] and OpenAI also being awarded a $200 million contract with the US in 2025.[10] These trends are also repeated in the Australian context, where in January 2026 it was reported that the Australian Defence Force (ADF) is investing 40 million AUD in emerging technologies and AI through the Advanced Strategic Capabilities Accelerator’s (ASCA) Emerging and Disruptive Technologies program.[11] Fourteen contracts were awarded to both universities and private companies, including Cortisonic and Swordfish Computing. An ADF press release on these contracts stated that the investment is intended to ensure that it can ‘make decision with speed and accuracy in an increasingly complex information environment’.[12] Additionally, it was reported that the Australian Department of Defence awarded Palantir a 7.6 million AUD contract in February 2026,[13] though there are growing concerns around Australia’s investment in the company.[14]
The current landscape of military AI highlights the central role of industry in shaping emergent AI capabilities. Critical development choices are primarily made by diverse industry actors, at times in collaboration with defence, but largely outside of military institutions. It is therefore important to pay attention to how industry contributes to the development of military AI capabilities and whether this has consequences for international law, as examined in the following sections.
AI capabilities as the product of development choices
AI systems are the product of a multitude of cumulative development choices made primarily by industry actors across the lifecycle. In the context of practices of development distributed across the range of corporations involved in producing AI capabilities, these choices are made by a diverse actors, from engineers and teams of data scientists to product managers. These development choices throughout the lifecycle iteratively shape technical systems, how they can be used, and the broader practices into which they are situated. These choices can therefore contribute to shaping military decision-making and thus the lawfulness of operations.
As AI systems are the product of numerous development choices, they reflect and embed human decision making. AI systems are often considered as objective and neutral due to their statistical and data-driven nature.[15] This reflects ‘a technologically-inflected promise of mechanical neutrality’, which is often promoted by industry developing these systems.[16] Through development choices, these systems embed a number of human subjectivities, including biases, values, interests, and assumptions.[17] For example, how data is labelled and prepared has a significant impact on the output of an AI system. The tendency to over-trust and uncritically rely on AI systems, referred to as automation bias,[18] further masks that AI systems are the product of human development choices that inform the outcomes they produce. As such, the technical features of AI systems should not be considered separately from their development practices.[19]
As AI capabilities become increasingly integrated into military processes, development practices – and the inherent subjectivities embedded within them – can have a significant impact on how militaries conduct their operations. Indeed, these systems are being introduced in order to ‘revolutionise’ warfare.[20] Yet, reliance on AI capabilities for situational awareness and operational decision-making implicates how militaries conduct lawful operations. The risk of IHL violations flows not just from the technical features of these systems, but the development choices that shape them, how military practitioners interact with them, and broader targeting processes as a whole. However, connecting outcomes in the battlespace with the many development choices made by heterogenous and diffuse industry actors is extremely complicated. As put by Seaver, this involves conceptualising AI not as ‘standalone little boxes, but massive, networked ones with hundreds of hands reaching into them, tweaking and tuning, swapping out parts and experimenting with new arrangements’, in which ‘the tendencies of an engineering team are as significant as the tendencies of a sorting algorithm’.[21]
Consequently, whilst concerns for control and human judgment characterise debates on military AI, ultimately there is no single human ‘in-the-loop’, but various decision points distributed across different actors at lifecycle stages. This raises an urgent challenge for militaries seeking to adopt AI capabilities, requiring them to consider not just how international law can be respected for a ‘final’ AI system or during use, but also in the numerous choices made by industry developers throughout the AI lifecycle.
State obligations to operationalise IHL in industry development practices
Whilst international law obligations for military operations primarily relate to operational decisions, the impact of development choices made by industry on legal compliance means that consideration of IHL only at the point of final review or deployment is already too late. The risk of violations of IHL are not only a result of the technical features of AI systems and how they influence decision-making, but also the development choices that shape these systems, how they can be used by military practitioners, and the broader military processes in which they are situated. As such, IHL must be operationalised throughout the development of AI capabilities, including decisions on how technical systems are designed, capabilities are specified, and whether or not certain applications or functionalities should be developed at all. This section argues that it is a legal obligation for States to ensure IHL is operationalised throughout industry-driven development practices.
While States are the primary bearers of international law obligations,[22] the role of private military AI developers nevertheless has direct implications for IHL compliance and the risk of violations. Core obligations owed under the core IHL targeting principles of distinction, precautions, and proportionality may be undermined through the use of AI systems,[23] particularly regarding limited accuracy, reliability, and transparency of AI systems leading to unintended military engagements and challenging civilian harm mitigation.[24] Concrete examples of risks for IHL compliance based on specific development choices made by industry across the lifecycle are traced in more detail in Section 3.
The characteristics of AI systems suggest that the fulfillment of core IHL targeting principles must be considered already during the development of AI-enabled military capabilities. AI systems can shape decision-making by creating conditions – including of speed, complexity, and opacity – that limit deliberation, frame information for decision-makers, and prompt uncritical reliance on AI outputs through automation biases.[25] These systems have limited performance in terms of accuracy (including errors and hallucinations) and reliability (particularly in complex and dynamic environments such as in conflict situations).[26] Combined, these characteristics can challenge the ability of military practitioners relying on these systems to take reasonable efforts to distinguish lawful targets from protected persons, to take constant care throughout operations and feasible precautions in attacks to mitigate civilian harm, including the verification of targets, and to assess proportionality.[27] Ensuring military practitioners have the capacity to deliberate over these legal determinations and critically assess AI outputs must therefore already be considered when AI capabilities are developed, including in the design of technical systems, the organisation of targeting processes into which they are integrated, and the configuration of human-machine interactions. Fulfilling core IHL obligations cannot be operationalised through a final review of an AI system or throughout deployment alone. Development choices shaping the limits of accuracy and reliability, and human-machine interactions – including the conditions of operational decision-making, the framing of information, and biases in end users – are already fixed through development. To the extent that this limits the capacity of practitioners to make lawful targeting decisions, the moment for intervention has already passed. Respect for the principles of distinction, precautions, and proportionality therefore necessitates States to ensure that development choices made by industry actors are consistent with the requirements of IHL.
Together with these core IHL principles, the obligation for States to operationalise IHL throughout development practices is grounded in their positive duty to ‘ensure respect’ for IHL under Common Article 1 of the Geneva Conventions.[28] This entails a due diligence obligation to ensure compliance with IHL. Due diligence obligations in international law are obligations of conduct, requiring States to make reasonable efforts to fulfil legal duties, without necessarily requiring they bring about a certain outcome.[29] IHL contains a number of due diligence obligations including the duty to ensure respect for IHL, which entails efforts to address violations that result from the conduct of third parties, such as private corporations.[30] Given the distributed development choices that ultimately produce AI systems, this due diligence obligation necessarily extends throughout the development process, not only at the approval of acquisition or during deployment.[31]
In conjunction with the core IHL principles and the duty to ensure respect for IHL under Common Article 1, the duty for States to legally review military capabilities also reinforces that the lawfulness of AI capabilities to be assessed and secured throughout development. Article 36 of the First Additional Protocol to the Geneva Conventions requires States Parties to conduct legal review of weapons, means, and methods of warfare.[32] There is debate about whether this obligation captures AI technologies that are not directly connected to the use of force, such as AI-enabled systems to support targeting decisions.[33] Nevertheless, as IHL compliance requires operationalisation prior to deployment of AI capabilities, including for decision support systems, this therefore suggests that a legal review is required throughout development, whether under Article 36 or otherwise.
The legal obligation for States to ensure that IHL is operationalised in industry development choices for AI capabilities is reinforced by the framework of international human rights law (IHRL).[34] IHRL applies to military operations during armed conflict alongside IHL and during targeting is read in light of IHL such that violations of the principles of distinction, precautions, and proportionality constitute a violation of the right to life.[35] Nevertheless, the development of AI systems driven by industry nevertheless occurs primarily (although not exclusively) during peacetime, reinforcing the relevance and applicability of IHRL. In its General Comment 36, the United Nations Human Rights Council has highlighted that human rights holds relevance for a number of situations associated with the development of military AI, including ‘the deployment, use, sale or purchase of existing weapons and in the study, development, acquisition or adoption of weapons, and means or methods of warfare’, indicating that States must assess the impact of these activities on the right to life.[36] For instance, human rights impact assessments have been proposed as one mechanism for operationalising throughout the lifecycle, including during development.[37] IHRL has particularly robust framework of positive obligations for States to protect human rights (in the military context, arguably including core IHL protections), including from business entities. This is reflected in the Business and Human Rights Framework, under which the United Nations Guiding Principles on Business and Human Rights (UNGPs) serve as authoritative guidance for preventing and addressing business-related human rights violations, including in conflict affected areas.[38] Due diligence obligations under IHRL require two key conditions: the reasonable foreseeability of the (risk of) harm and the reasonable capacity to intervene.[39] Whilst there may be some limitations to establishing jurisdiction over the actions of private companies, nevertheless responsibilities for diligent prevention and protection of human rights can require States to regulate these companies and prevent them from harming human rights of people situated outside of their jurisdiction.[40] Read together, the frameworks of IHL and IHRL reinforce positive obligations on States to ensure respect for international law throughout the development of military AI systems by industry actors.
Accountability frameworks under international law serve as a basis to hold States responsible and engage industry when the development of AI capabilities contributes to violations of IHL. State responsibility provides a basis for holding States accountable for violations of international law attributed to them,[41] including violations of IHL that result from the conduct of private developers acting as organs or agents of the State, as well as State failures to exercise due diligence in ensuring IHL compliance in private sector development.[42] Regarding developers themselves, criminal liability has also been explored as a possible means of holding private developers responsible for international crimes committed with autonomous weapons and overcoming a responsibility gap.[43]
Corporate responsibility offers a further framework to guide industry actors developing military AI with respect to IHRL. Next to State duties to protect human rights, the UNGPs also set out corporate responsibility to respect human rights, which is operationalised through human rights due diligence in business activities and relationships.[44] For industry actors developing military AI this entails identifying, preventing and mitigating human-rights related risks, which co-applied with IHL suggests addressing the risks of violations of the principles of distinction, precautions, and proportionality. Finally, regional instruments such as the EU AI Act also govern corporate conduct regarding AI and, despite excluding high-risk applications such as defence, remain relevant for companies developing dual-use AI systems for both civilian and military purposes.[45]
State Dependency on Industry and Constraints for IHL Compliance
Technical knowledge and expertise in AI systems are concentrated primarily within the private sector, creating a dependency of defence on industry with significant implications for States' ability to ensure IHL compliance throughout the development of military AI capabilities. Industry provides not only AI technologies, but also the expertise and specialised knowledge for their evaluation and assurance, use, and ongoing maintenance. This knowledge relates to frontier AI techniques, computer science, and systems engineering, with industry retaining access to training data, system architecture, and source code. This knowledge and expertise make industry integral to military operations in which these systems are deployed. This is increasingly evident with the role of the private sector offering subscription-style services to AI-powered platforms and systems, as well as the increasing role for forward-deployed engineers.[46] Incentives to embed the role of industry are reinforced by current discourse framing national security around the urgent need for AI capability development in defence.[47]
The centrality of industry for emergent military capabilities creates a distinct kind of procurement relationship. The concentration of knowledge and expertise on military AI systems within industry risks creating a dependency in which defence is reliant on the private sector.[48] This relationship of dependence can shape what kinds of AI systems are developed and procured by defence, as well as the standards, metrics, and processes used to assess these systems. This can create challenges for defence in retaining adequate and independent oversight of AI systems, including for reviewing legal compliance. In particular, it may be difficult for defence to determine how development choices impact legal compliance without industry intervention.
Though states may outsource aspects of the development of military AI systems, they cannot outsource legal compliance and must still ensure respect for IHL during the development of these systems, as outlined in Section 2.3 above. As such, throughout the development of military AI systems, defence should maintain sufficient independence from industry, demanding transparency, preserving specialised knowledge to assess AI systems (either in-house or through independent institutions), and ensuring IHL is operationalised throughout all stages of the AI lifecycle. States retain critical knowledge regarding operational and legal requirements necessary for the development of military AI systems. As procurers, they can structure development practices with IHL obligations at the forefront by facilitating collaboration and knowledge-sharing with industry rather than a public-private relationship based on dependency. Industry should likewise be concerned with IHL during development to provide operationally sound and lawful systems and avoid legal, commercial, and reputational risk. Both defence and industry must therefore seriously consider and operationalise IHL throughout the development of military AI. To this end, these actors should be aware of how development choices throughout the AI lifecycle shape legal compliance. This paper serves to trace this by outlining how key development choices made at different stages of the lifecycle can impact compliance with the key IHL principles of distinction, precautions, and proportionality.
Industry Development Practices Across the AI Lifecycle
Industry actors contribute to the development of military AI in a range of ways. Development practices are not isolated at one point pre-procurement, but occur in a distributed and iterative fashion throughout the lifecycle of these capabilities. This section traces how industry development practices across the AI lifecycle can be a source of risk for IHL violations and constrain how States are able to fulfil their due diligence obligations. Building on engineering frameworks such as those developed by the IEEE ecosystem on the military AI lifecycle,[49] which identify lifecycle stages including planning, design, development, testing, deployment, operation, and maintenance, it situates these technical stages within a broader socio-technical lifecycle.[50] This spans the involvement of industry in the development of military AI capabilities spanning (i) governance; (ii) research and development; (iii) acquisition, procurement, and review; and (iv) deployment.
Governance
Whilst it may seem counter-intuitive to suggest that the development of AI capabilities begins prior to research and development, in practice much of the ideation around AI capabilities begins already before any specific systems are developed. This occurs throughout the governance stage, where States and other stakeholders take efforts to govern, regulate, and plan for the integration of military AI.
The military-industrial complex has been pervasive for some time, with lobbyists playing a more backward-facing role by meeting with States to promote industry agenda.[51] Today, industry also plays a forward-facing role in some international governance contexts by actively participating in governance and the strategic planning of militaries on military AI, contributing to driving the conceptual space of warfare.[52] Principally, this takes place through the engagement of industry in global governance efforts and self-regulation initiatives. Industry contributes to the global agenda and norms around military AI by shaping understandings of what AI capabilities do, defining open concepts like ‘responsible AI’, and developing standards – including on legal compliance – before States do so. In doing so, industry also contributes to shaping how compliance with IHL is understood and operationalised when decisions around AI capabilities are made during planning and governance efforts.
The need to involve industry amongst a range of relevant stakeholders in the governance of military AI has been recognised by both the United Nations,[53] and other regional efforts such as the Responsible AI in the Military Domain (REAIM) process.[54] Nevertheless, industry engagement at this stage is multivarious and uneven. As highlighted in Section 2.1 above, it involves a range of companies of different sizes and with different products and expertise. Various industry actors are present at multi-stakeholder State-led initiatives, such as the REAIM Summit series, and most recently with tech executives from OpenAI, Anthropic, and Google attending the G7 Summit alongside State leaders in June 2026.[55] In contrast, there seems to be less engagement during some multilateral governance efforts, such as at the United Nations or the Group of Governmental Experts on Laws or the United Nations informal talks on military AI.[56] Industry also engages in global governance of military AI through self-regulation efforts. Industry self-regulation on AI involves the private sector developing and implementing its own rules, standards, and best practices based on its specialised knowledge, expertise, and practical experience.[57] This know-how is one of the main reasons that States engage with industry in governance debates on military AI.[58] Efforts at self-regulation may prove to fill a gap left by the governance stalemate in regulating military AI,[59] such as lack of consensus at the Group of Governmental Experts on Lethal Autonomous Weapon Systems,[60] and has been suggested to foster both innovation and ethics, legitimise developments, and build public trust.[61]
The active participation of industry at governance fora on military AI and in efforts at self-regulation is exemplified at the series of REAIM summits hosted in The Hague (the Netherlands) in 2023, Seoul (Republic of Korea) in 2024, and A Coruña (Spain) in 2026. At each of these events, industry played a role both in the promotion of their products and speaking on responsible innovation in the defence domain.[62] For instance, in Seoul in 2024, the ‘REAIM Talks’ featured in the programme to ‘invite experts from both academia and industry to discuss how the policy discussion on responsible AI can be translated into engineering tasks’. [63] Demonstrating the intersection of military AI governance and industry self-regulation, in A Coruña in 2026[64] a representative of Microsoft took part in the plenary session on ‘Framework for Responsible Industry Behaviour on AI in the Military Domain’.[65]
Industry interventions in governance and self-regulation efforts can shape discourse and planning around military AI in line with corporate interests. After all, the models of innovation of companies developing military applications of AI are incentivised by national regulatory environments that are favourable.[66] It is in the interests of business development that industry props up discourse around AI as a necessary and urgent requirement for national security, strategic acceleration, and warfighting. The notion promulgated by the private sector that AI will win wars creates and legitimises a significant role for AI within military capabilities, and consequently of industry actors in their development. Schwarz highlights that by positioning AI developers as invaluable to future warfare over traditional arms manufacturers, industry both serves promises of long-term material benefits and establishes technical experts ‘as experts in how to win wars’.[67]
The pressures for rapid innovation cultivated by both States and industry risks sidelining efforts to understand the legal and ethical implications of these technologies.[68] This is not to say that industry obscures or ignores international law, but rather that by shaping discourse and setting standards, industry contributes to influencing understandings of legal obligations in a more subtle way. At times, industry has explicitly engaged with legal frameworks, such as IHL.[69] At the same time, initiatives such as the Framework for Responsible Industry Behaviour on AI in the Military Domain reflect joint efforts between international organisations, States, and industry to develop a set of voluntary guidelines for the private sector that are not only grounded on existing legal frameworks but also notably ‘emerging principles’.[70] In this sense, industry contributes to both interpretations of existing frameworks and the formulation of new norms on military AI, as reflected in the aim of the Framework to ‘clarify what we mean by responsible industry conduct… grounded firmly in international law and norms’.[71]
This context highlights how the private sector is firmly embedded in the military AI agenda and is ostensibly indispensable to these efforts, giving it a platform through which corporate values such as efficiency and comparative advantage can infiltrate discourse on the development and legality of these systems.[72] The perceived need for rapid innovation mirrors and accentuates the need for efficient and expedited targeting processes. When AI capability development and targeting processes are increasingly optimised for efficacy and speed to better overcome the adversary, this embeds values that reflect ‘logics of quantification’ across the military AI lifecycle.[73] This quantification reflects the ‘routinization’ of military operations in which violence becomes procedural, emphasising efficiency, accuracy, and speed.[74] These values around efficiency can further filter into how IHL is operationalised across the lifecycle.
At the governance stage, technical values driven by industry innovation and optimisation can shape how compliance with IHL is understood. IHL is a value-laden legal framework: whilst specific principles are often referred to as value-laden, notably the principle of proportionality, this framework is also underpinned by the foundational principles of military necessity and humanitarianism.[75] These are values towards which IHL strives to balance. Yet, when those with technical expertise within industry underscore the urgent need for AI capabilities, values of efficiency become instilled within what States define as ‘militarily necessary’. Likewise with humanitarian considerations, the efficient streamlining of targeting processes through AI capabilities is posed as a means of ensuring more precise targeting, protecting civilians, and winning wars fast. Yet, there are serious questions as to whether the promise of AI lines up with its realities, given the inherent limitations of state-of-the-art AI systems in terms of accuracy, predictability, and transparency.[76]
Technical values around efficiency prioritised through corporate interventions in the military AI agenda can also contribute to compliance challenges for the IHL principles of distinction, precautions, and proportionality. Based on their knowledge and experience, weight and authority are given to industry claims about what AI capabilities do, even if untested. In some cases, AI systems are marketed as ‘battle tested’, for instance in the ongoing conflict in Ukraine,[77] implying that systems are operationally effective and can be used lawfully. However, whether AI systems can be used lawfully and have operational efficacy is highly context-dependent and is not generalisable across different operational contexts.
Assumptions around AI integration from developers’ perspectives can misalign with legal requirements where developers view legal challenges as requiring technical fixes. For instance, while the development of AI capabilities can be based on assumptions that battlespaces can be adequately represented through datapoints and decisions can be based on computational insights from that data, IHL requires contextual and value-laden decision-making that resists technical ‘fixes’ and requires consideration of the prevailing circumstances.[78] Another illustration is the development of AI systems using probabilistic risk scores to identify militants, as was reportedly done by Israel in Gaza.[79] Risk-based assessments of militancy do not account for the context-dependent assessment of whether a civilian is directly participating in hostilities or a member of a militant group is serving a continuous combat function. [80]
Industry instilling the value of efficiency also influences ideas around how AI can alter the organisation processes of militaries and the professional roles of military practitioners within these, including for conducting legal assessments. Technical ‘fixes’ to make targeting more efficient can diminish the critical friction points when human deliberation over IHL, whether it be the status of a target, determination of feasible precautionary measures, or assessment of proportionality.[81] These challenges present critical concerns around the ability of militaries to operationalise IHL when pursuing agendas of technological innovation and efficiency that are driven by industry. These shifts towards efficiency begin already during governance and planning efforts and are later reinforced by concrete design choices made during research and development, procurement, and deployment.
Research and Development
Research and development are critical processes in which industry actors shape AI technologies, with consequences for compliance with IHL. Subjectivity and bias are entrenched in AI systems through their development, as discussed above at Section X. Despite attempts to debias datasets, the subjective choices embedded in the development of ML models means that it is almost impossible to address and mitigate bias within these systems.[82] For instance, Cummings and Li identify at least twelve choices during the construction of ML models that involve subjective choices by developers.[83] How AI systems shape behaviour and practices inevitably reflect the assumptions of developers about ‘who will use the systems and what it should, will, and ought to be used for’.[84]
Numerous development choices are made at the research and development stage that have the potential to shape how militaries comply with legal obligations and influence risks around violations. Sufficient quality and quantity of representative and context-appropriate datasets are critical for accurate performance of military applications of AI.[85] Choices of developers around what datasets to use and how they are labelled and prepared can influence how AI systems identify objects of interest and whether practitioners conduct targeting in compliance with the duty to distinguish lawful targets from protected persons and objects. Systems adapted from foundational models, such as adapting LLMs for targeting operations, can also raise challenges for IHL compliance, as the development choices made for a base model in the civilian sector may not be appropriate for the distinct environments, tasks, and legal requirements in warfare. Furthermore, choices around specific mechanisms for explainability (such as more technical methods of xAI that provide computational explanations through probabilistically reverse engineering the system’s operation – or ‘model-of-a-model explainability’)[86] may also provide technical explanations that are not useful for the kind of explanations around the basis for identification of a certain target, in order for commanders to substantiate their decisions and contest AI outputs.[87] Each of these examples highlights how choices made by developers can have cascading effects for IHL compliance once these systems are used. By way of illustration, the rest of this section examines development choices around training data to highlight why development choices are a source of risk for IHL violations, particularly around the duty of militaries to distinguish lawful targets from protected persons, principally including civilians.
The curation and preparation of training datasets are key stages where human choices inform how AI systems behave with implications for compliance with IHL. For AI models trained through supervised learning approaches, the training data is determinative for the output of the system. Subjectivities and bias in the training data result not only from the data itself (a common concern surrounding the use of biased datasets) but also from the choices of those involved in preparing these datasets. Decisions around what data to include, how to label, clean and encode it, and which relevant features to select reflect the ‘invisible labour’ that goes into the development of these systems.[88] The output of a ML model is therefore contingent on its training data, which is itself shaped by the processes in which it is labelled, selected, and prepared to train and test the model.
Data labelling is a key part of this process, which occurs separately from the development of AI systems through a process in which data points are annotated with labels, often by companies operating under severe and exploitative working conditions.[89] There is a high degree of subjectivity in data annotation processes,[90] with data labelling taking place on the basis of natural language meaning. Much research points to the subjectivity during data annotation processes that lead to biased labels.[91] The preparation of datasets by AI developers is a further process embedding human subjectivity in the development of these systems. There are various processes that developers go through to prepare datasets to train AI models. This includes ‘data cleaning’, which involves subjective decisions around what datapoints should be included and excluded.[92] It also includes the process of feature selection, where data is selected based on what the developer deems to be relevant features. As the features selected are crucial for determining the eventual output the system, feature selection is a significant site of human subjectivity in the development of AI systems.[93]
Training data is critical to how AI systems behave. AI systems generate output based solely on the patterns identified from the training data, meaning that anything outside of this (referred to as ‘out of distribution data’) will not be accounted for.[94] The significance of training data for system output is demonstrated by the tank classifier parable, a story that explains how an artificial neural network classified pictures of tanks based not on the presence of a tank in an image, but on whether the image showed a sunny or overcast sky in the background.[95] There are several examples of this kind of ML error in reality, such as a classifier that distinguished huskies and wolves based on whether the image also showed snow.[96] These unintuitive errors highlight why the choices of developers around training data can significantly contribute to the outcomes of AI systems and how they perform in use compared to testing during development.
The significance of choices around training data has implications not only for system performance, but also risks around violations of IHL as a result. In particular, choices around the selection and preparation of training data can have implications for compliance with the IHL principle of distinction, requiring that militaries distinguish between lawful targets and protected persons and objects. Choices of developers on training data can contribute to unintended engagements and harm to protected persons when AI systems are relied upon for targeting. These choices on data used to train AI systems contribute to determining the output the system will eventually produce and creates the risks of introducing hidden assumptions, subjectivities, and bias that skew AI output into the targeting process, leading to unintended engagements.
Decisions made according to the principle of distinction entail legal assessments that are distinct from development practices around data preparation.[97] For instance, whether individuals are combatants who are surrendering (hors de combat), civilians who are directly participating in hostilities, or militants engaging in a continuous combat function require contextual and qualitative legal assessments.[98] In contrast, data labelling is conducted not based on legal categories, but natural language understandings based on fragments of imagery interpreted outside of the context of armed conflict.[99] Likewise, data preparation is conducted by technical experts without domain expertise on military operations or IHL who are situated in a private civilian context rather than in armed conflict. To the extent that developers have preconceived ideas about what constitutes a combatant, militant, or civilian or which factors might be relevant for an AI system given its targeting function, choices around the labelling and preparation of training datasets can significantly contribute to the outcomes of AI systems. These hidden assumptions, subjectivities, and biases can risk errors and unintended engagements resulting in IHL violations. Seemingly benign decisions about whether someone wearing a military uniform and bearing arms is a combatant, as well as biased judgments about what a militant ‘looks like’ based on race, gender, and other discriminatory factors reinforce biased ideas underpinning how targets are identified in armed conflict.[100]
Whatever factors are deemed relevant by developers will inevitably shape the kinds of predictions a ML model will make. In data preparation practices like feature selection, the removal of factors that seem irrelevant to the developer may obscure the nuance and novelty that is characteristic of armed conflict, and which must necessarily play into distinction assessments under IHL in targeting.[101] Developers – who most likely will not have domain expertise on IHL – may, for instance, be unaware of the relevance of accounting for cultural idiosyncrasies, such as that in some contexts carrying weapons may be a typical cultural practice rather than indicating hostile intent and participation in hostilities. As a result, developers may discount the relevance of cultural factors reflected in certain data features; instead, they merely appear as features to be ‘cleaned up’.
The contextual nature of IHL distinction assessments highlights why the choice of appropriate training datasets for the context of use is crucial not just for operational accuracy, but also compliance with IHL. A challenge for developers is that in most cases it will be impossible for developers to know what training data will be appropriate for the context of use prior to deployment, yet this is critical for how the system will perform and whether it will result in false positives or false negatives.
Ultimately, whether unintended engagements connected to choices around training data for AI systems will constitute a violation of the IHL principle of distinction will turn on the reasonableness of the commander’s targeting decision. Given the tendency of AI systems to produced biased outcomes in part due to the invisible choices of corporate actors involved in data labelling and preparation, it is unlikely these decisions relying on AI output will be reasonable unless additional measures are taken to verify targets. However, challenges to verification exist when AI systems are used, notably due to the speed and scale of these systems and tendencies towards automation bias, as well as the difficulty of establishing the reliability of AI systems and predicting errors, particularly when these result from hidden development choices. The choices of developers around training data for AI systems therefore contributes to shaping outcomes in the battlespace and can be a source of risk of non-compliance with IHL.
Acquisition, procurement, and review
During acquisition, procurement, and review, industry plays a critical role in shaping how compliance with IHL is understood and assessed. Acquisition is a broad process in which States obtain military capabilities, generally encompassing the formulation of military requirements (identifying what capabilities are needed and prioritised), various models of development (by both scientific institutions, state militaries, and industry, increasingly in collaboration), procurement processes (e.g. sourcing, tendering, and contracting), as well as cross-cutting regimes for assurance and legal review. In light of accelerating innovation for military applications of AI, industry advocacy has become particularly influential in shaping and accelerating procurement processes.[102] Along with the pressures around the urgent need for efficient integration of AI capabilities, in some instances this has resulted in the streamlining of acquisition processes.[103] Whilst typically viewed as a separate lifecycle stage to the research and development stage addressed in the preceding section, in practice, new models of co-development between militaries and industry and streamlined procurement processes mean the boundary between these lifecycle stages is blurred. The development of AI systems by industry is linked to how militaries formulate their capability needs, what requirements they implement for tendering and contracting, how AI systems are tested and evaluated, and how legal reviews are conducted. Setting aside the highly relevant question of whether industry owes obligations during acquisition processes (e.g. through the business and human rights framework) for the purposes of this article, how States respect and implement IHL obligations is directly shaped by the role of industry. Nevertheless, States retain obligations to respect and ensure respect for international law, which cannot be outsourced to industry. This raises key questions about how IHL can be operationalised within acquisition processes that are structurally shaped by industry practices, expertise, and interests.
The technical expertise of industry allows it a key role in structuring acquisition processes for military AI, with direct implications for how States fulfil obligations under IHL. Industry plays a determining role in shaping the specifications, metrics, and methods of evaluation used to determine whether AI systems are fit for purpose. This also has flow-on effects for legal reviews determining whether AI systems can be used in compliance with IHL. This section examines how industry's role in shaping acquisition processes for military AI, including technical assurance and legal review, creates structural risks for State compliance with IHL.
The development of frameworks for assessing AI systems is principally concentrated within industry and structures acquisition processes, including legal reviews. The assessment of AI systems is typically carried out through technical assurance processes, such as frameworks for Testing, Evaluation, Verification, and Validation (TEVV).[104] From an engineering perspective, reliability testing involves determining the probability that a system will function correctly.[105] This requires a particular number of tests to gain statistical assurance of a required level of confidence in reliability.[106] Some have suggested that in order for the testing of military technologies to be meaningful, ‘critical issues of performance must be translated into testable elements that can be objectively measured’.[107] However, the measurement of AI systems is never fully objective, involving assumptions and subjective choices of developers about how a system should function for specific tasks and how to measure this. Industry’s technical expertise and access to proprietary systems (including source code), development processes, and training data allow it to determine what specific metrics, methods, and evaluation frameworks are used to assess AI systems.
AI raises a number of specific issues for testing, which are largely addressed through choices by industry actors. First, testing for accuracy and predictability is particularly difficult for contemporary AI systems as performance significantly degrades when the context of use deviates from the testing conditions or assumptions of developers.[108] This difficulty stems from the tendency of AI systems to be ‘brittle’ (where small chances in input data compared to training and testing can significantly impact outputs)[109] and exhibit ‘concept drift’ (where real-world performance degrades as input data evolves over time),[110] which are particularly challenging when AI systems are used in complex, dynamic, or novel environments, all of which characterise conflict situations. Second, challenges for AI testing also relates to the limits of transparency in the design of these systems. The complex and opaque characteristics of AI systems that are unable to provide explanations presents challenges for understanding of how the system functions and testing as a result.[111] Third, difficulties also arise related to the large amount of data AI systems rely upon, given that in the military domain pre-existing datasets and the opportunity for real life testing is limited.[112] It may therefore require significant investment in methods for testing and evaluation, such as ‘modeling, simulation, and experimentation’, to ensure the system meets its specifications (verification) and functions as expected (validation), alongside rigorous testing and evaluation procedures.[113] Testing methods that use synthetic data and simulations are choices made by developers that can contribute to compounding challenges around the gap between performance in testing and real-world use. Finally, these challenges for testing AI capabilities are exacerbated by the disaggregated nature of AI systems and the broader networks and processes into which they are integrated. More specifically, the performance of different hardware and software components and the overall system must be assessed, along with the performance of the system in light of integration into broader ‘systems-of-systems’[114] and organisational processes, including human-machine interactions. For instance, accuracy rates in testing will largely hinge on the source of input information, which may also be processed through other AI systems, and explanations will only offer traceability if meaningfully integrated into decision-making procedures and trusted by users.[115]
The selection of metrics and testing procedures for accuracy, methods for explanations to facilitate traceability in testing, and conditions for testing using synthetic data and simulations are all choices made by industry to be able to test the systems they develop, and towards which developers aim optimisation. These technical specifications are then relied upon within acquisition processes.
Though technical assurance and legal review are distinct processes that serve different purposes, the technical assessment of AI systems nevertheless informs the review of their legality. As outlined above, technical assessment of AI systems during acquisition processes involves the technical expertise and choices of developers situated within industry. These technical assurance frameworks are developed to ensure systems are fit for purpose and operate as specified and expected. In contrast, legal review involves a determination on whether under some or all conditions military systems can be used in compliance with IHL, as well as other relevant frameworks such as IHRL. The review of weapons, means and methods of warfare under Art 36 API must be undertaken at the stages of ‘study, development, acquisition or adoption of a new weapon, means or methods of warfare’, based on the anticipated ordinary use of the weapon at the moment of evaluation. This obligation applies equally to the States that research, develop, produce and sell weapons, as well as those that acquire them,[116] with the latter having to conduct evaluations independently of any characterisation of legality purported to be held out by the exporting State or company.[117]
In order to review the legality of military AI capabilities, States must be able to assess how they function. Typically, one key aspect to ensure the review process is effective, amongst others, is to ensure ‘the availability of documentation on all aspects of the equipment to be procured which covers its military utility, its concept of use, capabilities and medical impact on the victim’.[118] Such documentation includes the assessment frameworks developed by industry for assurance. The challenge is that technical assurance and legal review as distinct questions, which require different methodologies to address.
Crucially, legal obligations are not reducible to technical specifications. IHL obligations under the principles of distinction, precautions, and proportionality that are held to the reasonable commander standard entail contextual and qualitative assessments.[119] This includes whether a military objective serves a military advantage, if civilians are directly participating in hostilities, whether combatants are surrendering or incapacitated (‘hors de combat’), as well as how to take constant care to avoid civilian harm, feasible precautionary measures in attacks, and avoiding disproportionate attacks where expected incidental harm to civilians is excessive with respect to the anticipated military advantage. These discretionary assessments are held to a standard of reasonableness, and must be determined in the prevailing circumstances based on the information reasonably available. Legal review of AI systems therefore entails an assessment of whether in all or some conditions the use of these systems can be consistent with these obligations. This requires analysis beyond technical assurance.
Questions around what level of accuracy would be sufficient in order to be comply with the IHL principle of distinction, requiring that militaries distinguish legitimate military targets from protected persons and objects, are inherently limited. Whilst a useful technical metric under specific conditions, accuracy rates that are determined through the limited testing of AI systems in a particular context (often outside of conflict situations) do not map well onto the highly contextual legal assessments needed to identify a military objective in armed conflict. Indeed, targeting according to IHL is not just a recognition task, and framing it as such portrays a reductive understanding of the contextual and qualitative assessments IHL requires.[120]
The ability of a military commander to take feasible precautionary measures has the potential to be undermined by the conditions of speed, scale, complexity, and opacity in targeting processes introduced by AI that result from concrete development choices. Respect for the principle of precautions arguably requires built in friction points for deliberation over mitigating civilian harm in a manner that is sufficiently integrated into the organisation of command-and-control decision making structures and addresses the challenges of uncritical reliance on AI through human-machine interaction.[121] Moreover, the proportionality principle also entails a contextual and value-laden analysis that cannot be delegated to a computational system or traded as a proxy for AI-based collateral damage assessments.[122] Legal review of AI systems must therefore account for how the use of AI systems shapes the agency of military practitioners and influences how they conduct the legal assessments they grapple with on a daily basis.
Ensuring respecting for legal requirements requires States to operationalise IHL throughout acquisition processes. This includes considering the wide range of risks to IHL compliance, including bias, errors and hallucinations, probabilistic outputs, transparency, and impacts on speed and scale of targeting. Yet, industry shapes the assurance of AI systems in line with its own expertise and interests, prioritising technical requirements over legal obligations. If States do not preserve the distinct aspects of legal review, this risks integrating an industry-driven, technically-inflected approach to military capabilities at the expense of legal protections. Whilst there may be challenges linked to lack of technical expertise within States conducting acquisition and lack of transparency within industry, States nevertheless have the obligation to ‘diligently seek information’.[123] States must therefore cultivate the capacity to understand the distinctions between technical assurance and legal review and preserve independence in assessing the legality of AI capabilities.
Because development and acquisition are deeply interconnected, this capacity for independent legal review cannot be reserved for the formal review stage alone.[124] As highlighted above, review of the legality of AI applications should occur throughout development processes, including at multiple stages during acquisition. Reviewing the legality of AI capabilities, including their integration into organisational targeting processes and resulting human-machine interactions, requires States to ensure that IHL considerations inform technical specifications and evaluation methods from the outset, rather than being applied only after industry has determined them. This requires not only the technical literacy to independently test industry claims about AI capabilities,[125] but also to scrutinise, contest, and redirect the methodological basis for such claims. Legal requirements should be a key component of understanding fitness for purpose: since the intended use of military capabilities should be lawful as stipulated in review requirements (e.g. under art 36 API), the specifications, methods, and metrics used to assess AI systems cannot be determined using technical considerations alone. This also suggests that legal obligations can be a basis for the decision not to acquire a certain application or functionality. As highlighted by Goussac and Boulanin, interrogation of whether and why a certain military AI capability is needed is a critical aspect of responsible procurement.[126]
Ensuring respect for IHL during acquisition of military AI necessitates a two-way knowledge exchange between States and industry on legal and technical expertise, and any review should resist being ‘compartmentalized’ on the basis of the different areas of expertise between lawyers, engineers, computer scientists and military stakeholders.[127] Flexible, collaborative, and iterative acquisition and procurement processes have been suggested to address the complexities of military AI and address IHL compliance.[128] States must nevertheless lead this process without ceding authority to industry based on technical expertise and resources, what one commentator calls ‘building an intelligence customer capability’.[129] This requires States to structure the acquisition of military AI in a manner that centres respect for IHL. Given the complexity of negotiating assessments of technical performance and legal review, the question of precisely how States must operationalise these IHL obligations already during acquisition of AI systems is a critical area requiring further research.
Deployment
Development choices by industry post-deployment can shape how militaries operationalise IHL. A key characteristic of the evolution of military AI, and an area warranting much closer attention and research, is that the development of these capabilities is no longer fixed to a temporal stage prior to deployment but continues throughout the use in the battlespace. As with the development choices made through earlier stages of the lifecycle already discussed, when AI systems are being developed and used in the context of ongoing military operations, this can have critical implications for compliance with IHL.
In large part, development practices continue post-deployment due to the mutable character of these technologies. Software in general is subject to updates and iterative changes. The nature of contemporary AI systems are that they continue ‘learning’, in the sense of improving ‘performance after making observations about the world’.[130] For machine learning, ‘a computer observes some data, builds a model based on the data, and uses the model as both a hypothesis about the world and a piece of software that can solve problems’.[131] To avoid performance degradation resulting from the brittleness and concept drift that AI systems are prone to (as mentioned above), models need to continue learning. Given the inherent risks of ‘online’ learning during the use of AI systems in warfare, systems are more likely to be maintained through ‘offline’ approaches, involving updates, retraining, and fine-tuning. This necessitates development practices to continue post-deployment and builds in a role for industry actors. As highlighted above, emergent models of military-industry collaboration include the prominence of forward-deployed engineers and subscription-style services for AI systems.[132] Throughout these processes, industry actors play an active role in redeveloping AI systems by engaging in maintenance activities and providing updates. This illustrates what Klonowska refers to as ‘tinkering’, which is a ‘highly experimental process of AI integration’… ‘manifest[ing] through iterative re-training and fine-tuning of AI models, repeated changes to interface design, development, and adjustments of explainability features’.[133] This section examines two distinct illustrations of how industry development choices post-deployment can shape legal compliance, focusing on design decisions on how to address distribution shift in AI systems and corporate governance choices on what kinds of AI systems companies are willing to develop.
Distribution shift is a phenomenon that occurs when the scope of an AI system’s training data (the distribution) is no longer generalisable to the conditions of use, which can become significantly apparent after the system has already been deployed. As put by Wu, “the generalization performance of a learning algorithm is largely affected by distribution shifts between the training and test environments”.[134] This also impacts the real-world performance of AI systems where there is misalignment between training/testing data and the context of use. An example is if an AI system trained in one conflict environment is then deployed in a different conflict, for instance in the snowy conditions of Ukraine compared to the differing weather conditions in Iran.
in the conflict in Ukraine (including under snowy conditions) is used in the current conflict in Iran, which has a significantly different climate. AI systems will only reliably analyse situations based on patterns found within its training data, often generating unreliable or invalid output for situations that fall outside of that dataset. Distribution shift can raise particular challenges when general purpose LLMs are used in the military domain but have been trained on large swathes of language-based data from the civilian domain,[135] requiring fine-tuning or other adaptation methods. A further difficulty is that distribution shift may be more difficult to detect due to the proneness of LLMs for confidently presenting invalid information (hallucinations) and the way these systems tend to appeal to the user (sycophancy).[136]
If after deployment a model exhibits distribution shift, developers make choices about ways to (re)train, update, or adapt systems for new data conditions, selecting from a number of available methods, such as transfer learning/domain adaptation, out-of-distribution generalization, multitask learning, continual learning or meta-learning.[137] These are critical choices for the performance of the system and therefore how it impacts legal compliance.
Since the data distribution is a key determinant of how a system will perform in a given context, this has implications for assessments under IHL that are necessarily context-dependent. When an AI system is updated to address distribution shift, the context the system is adapted to (e.g. through retraining or finetuning on new data) directly informs legal assessments subsequently supported by the system. When distinction assessments on who counts as a combatant, militant, or civilian directly participating in hostilities, and therefore a lawful target, or assessments on what feasible precautions are taken or if a proposed attack will be proportionate, all involve assessments based on the context at hand. How AI systems account for context can be shaped through the choices made by developers post-deployment, including how to address distribution shift. These approaches to the redevelopment of AI systems to address distribution shift shape how the information space is understood by commanders, highlighting (and obscuring) certain features of the environment and recommending particular courses of action over others, all of which shapes how IHL is operationalised during military operations. If military AI systems serve as a way to frame and filter information in a way that guides operational decision making, then the choices made by both initial developers and those developing systems post-deployment determine the boundaries of that frame. When systems are the first point of inquiry – rather than legal advisors, for example – this positions those developing and retraining these systems as the experts and highlights the significance of the decisions they make around how AI systems are updated. These decisions, such as the choice of data for retraining, can mean the difference between the identification of an individual as a militant or civilian, for example, shaping distinctions under IHL in targeting. It also raises the question of when unintended engagements result from inaccurate information by AI systems, what role the developer has who has been updating and adapting the system throughout deployment to address concerns about accuracy and how accountability is distributed and attributed to those responsible for any resulting violations of IHL, particularly for incremental development choices regarding learning paradigms that are hard to trace and understand.[138] Commanders and legal advisors must be able to have the capacity to understand how updates to AI capabilities during deployment can alter their performance, the conditions under which they can be lawfully used, and the way that they impact interactions with military practitioners. This also raises crucial questions about the need to continue conducting legal reviews of AI capabilities post-deployment.[139]
A different kind of industry choice during deployment that impact legal compliance is demonstrated by the 2026 controversy between the tech company Anthropic and the United States Department of Defense.[140] During the deployment of Anthropic systems, such as Claude, disagreement arose as to whether these systems could be used for certain purposes. Anthropic indicated that its systems could not be used for mass surveillance of the US population or as part of autonomous weapons systems, causing the DoD to cancel its contract with them. This illustrates one way for industry to exercise agency by setting standards and self-regulating, even post-deployment. It demonstrates the fluidity between lifecycle stages for development choices made by industry that link to AI governance and self-regulation, procurement (through contract administration), and broader development choices about the kinds of systems they are not willing to produce. When Anthropic set their red lines, this may serve to legitimise other AI capabilities that are nevertheless problematic and raise legal challenges, such as AI-enabled decision-support systems. This article has highlighted many of the concerns around the development and use of AI capabilities, including decision-support systems, contributing to violations of IHL. Yet, when industry sets its red lines only on domestic surveillance or autonomous weapons, these concerns are minimised. Moreover, other industry actors rapidly step in to fill the gap, as was the case in this situation with Open AI, suggesting that understandings about the ethical desirability and legal compliance of certain applications will be interpreted differently by different industry actors, based on their commercial interests (whether reputational, ethical, or commercial). This similarly occurred when Google employees protested project Maven being used for military purposes leading to company pulling out,[141] and eventually being replaced by Palantir who developed Maven Smart Systems.[142]
Whilst significantly distinct processes, the illustrations of system updates for distribution shift and self-regulation through choices not to develop certain applications show how industry development choices at different levels (one technical, the other corporate policy) and cross-cutting what are often treated as separate lifecycle stages (e.g. governance, procurement, and development) shape how militaries comply with IHL. Development processes continue post-deployment, meaning that the choices industry actors make even after a system has been deployed can critically impact operational decision making and the operationalisation of IHL therein. The values and interests that are embedded at the up-stream stages of governance and planning, research and development, and acquisition are reinforced by these downstream development choices.
Conclusion
The development of military AI capabilities is not a neutral technical process. Industry actors make choices throughout the lifecycle of these systems that embed subjectivities, shape military operations, and have direct implications for respect for international law. Considering IHL compliance only in a final system review or during deployment is, therefore, already too late.
This article argues that States have a legal obligation to diligently operationalise IHL throughout the development of military AI capabilities, which cannot be outsourced to industry. Whilst States may and increasingly do rely on industry for the development, evaluation, and maintenance of military AI systems, the concentration of knowledge and technical expertise within the private sector creates a structural challenge for discharging IHL obligations. The article demonstrated this by tracing how concrete development choices by industry across the lifecycle, from data collection and model design through to testing, integration, and ongoing maintenance, can be a source of risk for IHL violations and can constrain how States are able to fulfil their due diligence obligations to ensure respect for IHL.
Whilst existing debates on military AI and international law gesture to the need to consider IHL throughout the development of military AI, this article formulates this as a legal obligation and demonstrates concretely how development choices across different lifecycle stages shape the capacity of States to discharge it. This analysis is not intended to be comprehensive, but to demonstrate why industry development practices have direct implications for IHL compliance and that States are legally required to address these risks throughout the development of AI capabilities. There are myriad development choices across the lifecycle, as well as broader normative and epistemic questions about the agency of industry in the military domain, that warrant further scrutiny. Open questions remain, for instance, on whether industry actors developing military capabilities are directly participating in hostilities and owe legal obligations on that basis, or whether the reasonable commander standard may eventually need to account for a reasonable engineering standard.
For military organisations, including the Army, the immediate implication is practical. Fulfilling legal obligations in relation to military AI requires cultivating the institutional capacity to scrutinise not only technical systems, but the development practices that shape system behaviour, how practitioners can use them, and broader operational processes. This means engaging with industry not from a position of dependency, but with sufficient independence and specialised knowledge to direct development in accordance with legal obligations and operational requirements. As the development of military AI capabilities continues at an accelerated pace, this is a challenge that military organisations cannot afford to defer.
Endnotes
[1] Rob Kitchin, “Thinking critically about and researching algorithms,” Information, Communication & Society 20 (2017): 14-29, 18.
[2] Alexander Blanchard, Vincent Boulanin and Laura Brunn, “Mapping the military AI industry,” SIPRI, April 23, 2026, https://www.sipri.org/commentary/topical-backgrounder/2026/mapping-military-ai-industry.
[3] Elke Schwarz, ‘From blitzkrieg to blitzscaling: Assessing the impact of venture capital dynamics on military norms’ Finance and Society 1 (2025): 1-24, 1.
[4] Hoijtink and van der Kist, “Platforms on the Frontline”.
[5] Kevin C Desouza, “The evolution of artificial intelligence (AI) spending by the U.S. government,” Brookings Institute, March 26, 2024, www.brookings.edu/articles/the-evolution-of-artificial-intelligence-ai-….
[6] US Army Public Affairs, “U.S. Army Awards Enterprise Service Agreement to Enhance Military Readiness and Drive Operational Efficiency,” July 31, 2025, https://www.army.mil/article/287506/u_s_army_awards_enterprise_service_….
[7] Anduril Industries, “Anduril Awarded $99.6M for U.S. Army Next Generation Command and Control Prototype,” June 18, 2025, www.anduril.com/article/anduril-awarded-usd99-6m-for-u-s-army-next-gene….
[8] NATO, “NATO Acquires AI-Enabled Warfighting System,” April 14, 2025, https://www.ncia.nato.int/newsroom/news/nato-acquires-aienabled-warfighting-system.
[9] US Chief Digital and Artificial Intelligence Officer, “Task Force Lima Executive Summary,” www.ai.mil/Portals/137/Documents/Resources%20Page/2024-12-TF%20Lima-ExecSum-TAB-A.pdf.
[10] AFP, “OpenAI wins $200m contract with US military for ‘warfighting’,” The Guardian, June 17, 2025, www.theguardian.com/technology/2025/jun/17/openai-military-contract-warfighting.
[11] Christine Casimiro, “Australia Plugs $26M Into AI to Supercharge Military Decisions,” Military AI, January 7 2026, https://militaryai.ai/australia-defense-ai/.
[12] Australian Government Department of Defence, “Harnessing Tech to Drive Faster Decisions across Defence,” media release, January 5, 2026, https://www.defence.gov.au/news-events/releases/2026-01-05/harnessing-t….
[13] Robert Dougherty, “Palantir secures $7.6m Defence contract to supply ICT system platform,” Defence Connect, February 17, 2026, https://www.defenceconnect.com.au/industry/17697-palantir-secures-7-6-million-defence-contract-to-supply-ict-system-platform.
[14] Josh Taylor, “Calls grow to ban Palantir in Australia after manifesto described by UK MP as ‘ramblings of a supervillain’,” The Guardian, April 30, 2026 https://www.theguardian.com/technology/2026/apr/30/palantir-manifesto-australia-government-contracts.
[15] Neil Renic and Elke Schwarz, “Crimes of Dispassion: Autonomous Weapons and the Moral Challenge of Systematic Killing,” Ethics & International Affairs 37, no. 3 (2023): 321-343, 337.
[16] Tarleton Gillespie, “The Relevance of Algorithms,” in Media Technologies: Essays on Communication, Materiality, and Society, ed. Tarleton Gillespie, Pablo J. Boczkowski, and Kirsten A. Foot (Cambridge, MA: MIT Press, 2014), 180–81.
[17] Mary L Cummings and Songpo Li, “Subjectivity in the Creation of Machine Learning Models,” Journal of Data and Information Quality 13, no. 2 (2021): 1-19; Gillespie, “The Relevance of Algorithms,” 178; Kitchin, “Thinking critically,” 18.
[18] Theo Araujo, Natali Helberger, Sanne Kruikemeier, and Claes H. de Vreese, “In AI We Trust? Perceptions about Automated Decision-Making by Artificial Intelligence,” AI & Society 35, no. 3 (2020): 611–23, 613.
[19] R Stuart Greiger, “Bots, bespoke, code and the materiality of software platforms,” Information, Communication & Society 17, no. 3 (2014): 342-356, 346.
[20] See e.g. European Commission, “Artificial Intelligence (AI) in Defence,” https://op.europa.eu/en/publication-detail/-/publication/a856c715-e075-11f0-8439-01aa75ed71a1/language-en.
[21] Nick Seaver, ‘Knowing Algorithms’ in Janet Vertesi and David Ribes (eds) digitalSTS: A Field Guide for Science & Technology Studies (De Gruyter Brill 2019) 412-422, 419.
[22] Increasing attention has been paid to non-state actors as bearers of international law obligations: see e.g. James Summers and Alex Gough, eds., Non-State Actors and International Obligations (Leiden: Brill | Nijhoff, 2018), https://doi.org/10.1163/9789004340251; as well as more specifically non-State armed groups and private contractors: see e.g. Murray, Daragh. “How International Humanitarian Law Treaties Bind Non-State Armed Groups” Journal of Conflict & Security Law 20, no. 1 (2015): 101–31; Emanuela-Chiara Gillard, “Business Goes to War: Private Military/Security Companies and International Humanitarian Law.” International Review of the Red Cross 88, no. 863 (2006): 525–72.
[23] See, Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts (Protocol I), June 8, 1977, 1125 UNTS 3 (API), arts. 48, 51, 57.
[24] See e.g., Abhimanyu George Jain, “Autonomous Weapon Systems, Errors and Breaches of International Humanitarian Law,” Journal of International Criminal Justice 21, no. 5 (November 2023): 1005–32, https://doi.org/10.1093/jicj/mqad043; Jessica Dorsey and Marta Bo, “AI-Enabled Decision-Support Systems in the Joint Targeting Cycle: Legal Challenges, Risks and the Human(e) Dimension,” International Law Studies 107 (2026): 137; Jessica Dorsey, “The Erosion of Human(e) Judgement in Targeting? Quantification Logics, AI-Enabled Decision Support Systems and Proportionality Assessments in IHL.” International Review of the Red Cross 107, no. 930 (2025): 1041–71; Rainer Rehak and Taylor Kate Woodcock, “Automating Civilian Harm: On Israel’s Use of the AI-Enabled Targeting System Lavender in Gaza and International Humanitarian Law,” in FAccT ’26: The 2026 ACM Conference on Fairness, Accountability, and Transparency, 4885–98, https://doi.org/10.1145/3805689.3812357; Taylor Kate Woodcock, “Human-Machine (Learning) Interactions: War and Law in the AI Era” (PhD diss., University of Amsterdam, March 2026); Klaudia Klonowska and Taylor Kate Woodcock, “Rhetoric and Regulation: The (Limits of) Human/AI Comparison in Legal Debates on Military AI” in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, eds Bérénice Boutin, Taylor Kate Woodcock and Sadjad Soltanzadeh (Asser Press 2026).
[25] Woodcock, “Human-Machine (Learning) Interactions”; Taylor Kate Woodcock, “Machine Learning: AI, Algorithmic Arbitrariness and Legal Decisions,” in AI, Law and Politics by Matilda Arvidsson and Kieran Tranter (Routledge, forthcoming 2026).
[26] Klaudia Klonowska and Taylor Kate Woodcock “Rhetoric and Regulation”; Rehak and Woodcock, “Automating Civilian Harm”; Henning Lahmann, "Bad Algorithms and the Epistemic and Discursive Powers of Military AI," SSRN Working Paper, 2026, https://papers.ssrn.com/sol3/Delivery.cfm/6392939.pdf?abstractid=6392939&mirid=1&type=2.
[27] These are the core targeting obligations under the principles of distinction (API arts 48, 51(2)), precautions (API art 57), and proportionality under IHL (API arts 51(5)(b), 57(2)(a)(ii)), which are held to the ‘reasonable commander standard’.
[28] Geneva Convention (I) for the Amelioration of the Condition of the Wounded and Sick in Armed Forces in the Field, 12 August 1949, 75 UNTS 31, common art. 1.
[29] Samantha Besson, Due Diligence in International Law (Brill Nijhoff 2023), 23; Bérénice Boutin, “Between Negligence and Malfunction: How to Address Responsibility for AI Failures,” in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, ed. Bérénice Boutin, Taylor Kate Woodcock, and S. Soltanzadeh (The Hague: T.M.C. Asser Press, 2026), 319.
[30] Marco Longobardo, “The Relevance of the Concept of Due Diligence for International Humanitarian Law,” Wisconsin International Law Journal 37, no. 1 (2020): 44-87, 59-60.
[31] Bérénice Boutin, "State Responsibility in Relation to Military Applications of Artificial Intelligence," Leiden Journal of International Law 36, no. 1 (2023): 133–150, 143-147; Bérénice Boutin and Taylor Kate Woodcock, "Aspects of Realizing (Meaningful) Human Control," in Research Handbook on Warfare and Artificial Intelligence, ed. Robert Geiß and Henning Lahmann (Cheltenham: Edward Elgar Publishing, 2024), 179–96; Bérénice Boutin, “Between Negligence and Malfunction: How to Address Responsibility for AI Failures,” in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, eds. Bérénice Boutin, Taylor Kate Woodcock, and Sadjad Soltanzadeh (The Hague: T.M.C. Asser Press, 2026), 320.
[32] As Copeland highlights in this special issue, article 36 specifically refers to the ‘study, development, acquisition or adoption of a new weapon’ (emphasis added): [insert cross-reference].
[33] There are compelling cases for why AI systems used for target production should be subject to article 36 review: Justin McClelland, “The review of weapons in accordance with Article 36 of Additional Protocol I” International Review of the Red Cross 85, no 850 (2003): 397-415; Klaudia Klonowska, “Article 36: Review of AI Decision-Support Systems and Other Emerging Technologies of Warfare,” in Terry D Gill, Robin Geiß, Heike Krieger, Rebecca Mignot-Mahdavi (eds), Yearbook of International Humanitarian Law, Volume 23 (2020) (Asser Press 2022) 123–153; Russell Buach, “Artificial Intelligence and the Article 36 Legal Review,” Saint Louis University Law Journal (2026) 70, no. 2: 285-308, 297-298.
[34] Grounded for instance in the International Covenant on Civil and Political Rights, Dec. 16, 1966, 999 UNTS 171 (entered into force Mar. 23, 1976).
[35] As such, violations of the core IHL principles will also violate the right to life. See generally, Legality of the Threat or Use of Nuclear Weapons (Advisory Opinion) [1996] ICJ Rep 226, para 25; Human Rights Committee (HRC), “General Comment No. 36, Article 6: Right to Life,” 30 October 2018, UN Doc. CCPR/C/GC/36, paras. 2, 64; Orna Ben-Naftali, “Introduction: International Humanitarian Law and International Human Rights Law—Pas de Deux,” in International Humanitarian Law and International Human Rights Law, ed. Orna Ben-Naftali (Oxford: Oxford University Press, 2011), 3–10, 5; Cordula Droege, “The Interplay between International Humanitarian Law and International Human Rights Law in Situations of Armed Conflict' (2007) 40 Israel Law Review 310, 310. See also Dieter Fleck, ‘Human Rights in Armed Conflict’ in Dieter Fleck (ed) The Handbook of International Humanitarian Law (4th edn OUP, 2021) 449-457, 450.
[36] HRC, ‘General Comment No 36’ (30 October 2018) UN Doc CCPR/C/GC/36 para 65.
[37] Daragh Murray, “Adapting a Human Rights-Based Framework to Inform Militaries’ Artificial Intelligence Decision-Making Processes,” St Louis University Law Journal 68, no. 2 (2024): 293-326, 304.
[38] Rachel Davis, “The UN Guiding Principles on Business and Human Rights and conflict-affected areas: state obligations and business responsibilities,” International Review of the Red Cross 94, no. 887 (2012): 961-979.
[39] Besson, “Due Diligence,” 101-102.
[40] Besson highlights that this regulation may even be an obligation in order for States to fulfil their own human rights due diligence obligations: “Due Diligence,” 201. Regardless of the potential jurisdictional limitations of IHRL, reading it together with IHL arguably applies to corporate activities that are central to developing a State’s military capabilities in line with their duty to ensure respect for IHL, fulfil core IHL conduct of hostilities duties, and conduct legal reviews.
[41] ILC Draft Articles on the Responsibility of States for Internationally Wrongful Acts, 2001 YILC, Vol. II (Part Two), 26–30.
[42] Bérénice Boutin, “Between Negligence and Malfunction: How to Address Responsibility for AI Failures,” in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, eds. Bérénice Boutin, Taylor Kate Woodcock, and Sadjad Soltanzadeh (The Hague: T.M.C. Asser Press, 2026), 329; Bérénice Boutin, “State Responsibility in Relation to Military Applications of Artificial Intelligence,” Leiden Journal of International Law 36, no. 1 (March 2023): 133–50.
[43] Marta Bo, “Are Programmers In or ‘Out of’ Control? The Individual Criminal Responsibility of Programmers of Autonomous Weapons and Self-Driving Cars,” in Human-Robot Interaction in Law and Its Narratives, ed. Sabine Gless and Helena Whalen-Bridge (Cambridge University Press, 2024).
[44] United Nations Human Rights Office of the High Commissioner, Guiding Principles on Business and Human Rights: Implementing the United Nations “Protect, Respect and Remedy” Framework, Principles 11–24, https://www.ohchr.org/sites/default/files/documents/publications/guidingprinciplesbusinesshr_en.pdf.
[45] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act), Official Journal of the European Union L 1689, 12 July 2024, 1–144, https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng.
[46] Rupert Barrett-Taylor and Matthew Ford, "Eroding Sovereignty in the Age of ‘War as a Service,’" Opinio Juris, November 3, 2025, https://opiniojuris.org/2025/11/03/eroding-sovereignty-in-the-age-of-war-as-a-service/; “The Rise of the Forward-Deployed Engineer: Bridging the High-Stakes Chasm Between AI Theory and Execution,” AI Critique, May 20, 2026, https://www.aicritique.org/us/2026/05/20/the-rise-of-the-forward-deployed-engineer-bridging-the-high-stakes-chasm-between-ai-theory-and-execution/; Alvarez & Marsal, “The Rise and Role of the Forward Deployed Engineer,” April 2026, https://www.alvarezandmarsal.com/sites/default/files/2026-04/The%20Rise%20and%20Role%20of%20the%20Forward%20Deployed%20Engineer.pdf; Defence Agenda, “Defence-as-a-Service: Europe’s New Military Procurement Model,” Defence Agenda, accessed July 3, 2026, https://defenceagenda.com/defence-as-a-service-europe-military-procurem….
[47] Katja Bego, “How a surge in defence and dual-use technology investment could reconfigure the global AI race,” Chatham House, April 2026) https://www.chathamhouse.org/sites/default/files/2026-05/2026-04-28-how-surge-defence-dual-use-investment-could-reconfigure-global-AI-race-bego.pdf.
[48] Rupert Barrett-Taylor and Natasha Karner, “AI Could Support the Military but Won’t Replace Generals,” Alan Turing Institute, https://www.turing.ac.uk/news/ai-could-support-military-wont-replace-general, 7; Emily Bienvenue et al. “Private Tech Companies, the State, and the New Character of War,” Carnegie Endowment for International Peace 2025, https://carnegieendowment.org/research/2025/12/ukraine-war-tech-companies.
[49] IEEE, A Framework for Human Decision-Making through the Lifecycle of Autonomous and Intelligent Systems in Defense Applications (IEEE, 2024), https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?arnumber=10707139.
[50] On the dual technical and socio-technical lifecycles for military AI capabilities, see: Jessica Dorsey et al., "Bridging Understandings of the Military AI Lifecycle: A Transdisciplinary Socio-Technical Approach to Governance," Opinio Juris, June 12, 2026, https://opiniojuris.org/2026/06/12/bridging-understandings-of-the-lifec….
[51] See e.g. David N Gibbs, “The Military-Industrial Complex in a Globalized Context,” in Corporate Power and Globalization in US Foreign Policy, ed. Ronald W. Cox (New York: Taylor & Francis, 2012), 95–113; Frank Slijper, “The Emerging EU Military-Industrial Complex,” TNI Briefing Series, no. 2005/1 (Amsterdam: Transnational Institute, 2005).
[52] Rupert Barrett-Taylor and Natasha Karner, “AI Could Support the Military but Won’t Replace Generals,” Alan Turing Institute, https://www.turing.ac.uk/news/ai-could-support-military-wont-replace-general, 37.
[53] United Nations, UN Doc. N25/107/66, https://documents.un.org/doc/undoc/gen/n25/107/66/pdf/n2510766.pdf.
[54] Global Commission on Responsible Artificial Intelligence in the Military Domain (GC REAIM), Responsible by Design: Strategic Guidance Report on the Risks, Opportunities, and Governance of Artificial Intelligence in the Military Domain (The Hague Centre for Strategic Studies, September 2025), https://hcss.nl/wp-content/uploads/2025/09/GC-REAIM-Strategic-Guidance-Report-Final-WEB.pdf.
[55] Le Monde, “Regulating Military AI, a Challenge Debated in Geneva Alongside the G7,” June 15, 2026, https://www.lemonde.fr/en/international/article/2026/06/15/regulating-m…; Reuters, “Tech Executives Attend G7 Summit as Leaders Address AI, Online Safety,” June 12, 2026, https://www.reuters.com/world/tech-executives-attend-g7-summit-leaders-….
[56] At the latter, Microsoft was notably the only company present and participating in the discussions: UK Stop Killer Robots, “Informal Exchanges on Artificial Intelligence in the Military Domain and Its Implications for International Peace and Security – Day 1,” June 16, 2026, https://ukstopkillerrobots.org.uk/2026/06/16/informal-exchanges-on-arti….
[57] Tshilidzi Marwala, The Balancing Problem in the Governance of Artificial Intelligence (Springer 2024), 208.
[58] Alexander Blanchard, Vinvent Boulanin and Laura Brunn, ‘ Mapping the military AI industry’ (SPIRI 2026) <https://www.sipri.org/commentary/topical-backgrounder/2026/mapping-military-ai-industry> accessed 1 June 2026.
[59] Vanessa Vohs and Marcel Schliebs,’ ‘This is my Last Resort’: Approaches to Overcome the Stalemate in Autonomous Weapons Regulation Through National Legislation and Industry Self-Regulation’ in Bérénice Boutin, Taylor Kate Woodcock and Sadjad Soltanzadeh (eds), Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence (Asser Press 2026).
[60] Alexander Blanchard et al., “Dilemmas in the Policy Debate on Autonomous Weapon Systems,” Stockholm International Peace Research Institute (SIPRI), February 6, 2025, https://www.sipri.org/commentary/topical-backgrounder/2025/dilemmas-policy-debate-autonomous-weapon-systems.
[61] Vanessa Vohs and Marcel Schliebs, “‘This Is My Last Resort’: Approaches to Overcome the Stalemate in Autonomous Weapons Regulation Through National Legislation and Industry Self-Regulation," in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, ed. Bérénice Boutin, Taylor Kate Woodcock, and Sadjad Soltanzadeh (The Hague: T.M.C. Asser Press, 2026); Tshilidzi Marwala, The Balancing Problem in the Governance of Artificial Intelligence (Springer 2024), 208.
[62] In the Hague in 2023, Palantir CEO gave an interview on ‘his vision on AI and how to use this in the military domain’, whilst the CEO of Avalor AI spoke about ‘the role of unmanned vehicles in the military domain, and how Avalor AI is enhancing the tactical value of military unmanned systems’ whilst displaying a demo of such a system: “Alex Karp's (CEO Palantir Technologies) vision on AI in the military domain | REAIM 2023,” YouTube video, 33:18, posted by REAIMSummit, 15 February 2023 <https://www.youtube.com/watch?v=L1PnaB15XBI>; “Responsible Innovation Talk: Avalor AI | REAIM”, YouTube video, 21:53, posted by REAIMSummit, 16 February 2023.
[63] “[REAIM Summit 2024] Day 1 _ REAIM Talks”, YouTube video, 01:02, posted by REAIMSummit, 9 September 2024 https://www.youtube.com/watch?v=zUY_Wh9yZsk. Also in Seoul, the CEO of Two Platforms, a startup focused on ‘artificial reality’ and generative AI, sat on a panel discussion alongside policymakers, academics, and other members of other international institutions for a plenary session on ‘Envisioning the Future Governance of AI in the Military Domain’: “[REAIM Summit 2024] Day 2 _ Plenary Session 3”, YouTube video, 52:12, posted by REAIMSummit, 10 September 2024 https://www.youtube.com/watch?v=3_QCOrZhRbc.
[64] Many industry booths were present with representatives from companies such as Microsoft, Capgemini, Oracle, Proportione, to name just a few.
[65] Program REAIM 2026, PDF, accessed March 22, 2026, https://reaim2026.es/es/REAIM2026/Documents/Program%20REAIM%202026_20260129.pdf.
[66] Tim Sweijs and Sofia Romansky, “International Norms Development and AI in the Military Domain,” CIGI Papers, no. 300 (Waterloo, ON: Centre for International Governance Innovation, 2024).
[67] Schwarz, “From blitzkrieg ,” 13.
[68] Amos Toh and Emile Ayoub, "The Business of Military AI," Brennan Center for Justice, 2026, https://www.brennancenter.org/media/15340/download/bcj-167_business_of_military_ai_final.pdf.
[69] This has been done in particular by Palantir, including at the REAIM Summit in Seoul in 2024, as well as when addressing the United Kingdom House of Lords on autonomous weapons: Palantir Technologies UK, Ltd., Submission to the House of Lords AI in Weapons Systems Committee: Inquiry on AI in Weapons Systems, April 2023, https://www.palantir.com/assets/xrfr7uokpv1b/T6XBvRNbtgOysf4XuYqpH/1ba005ae5b469eb47ea484ad34dadcea/Palantir_Submission_to_the_HL_AI_in_Weapons_Systems_Committee.pdf.
[70] Though collaborators on this programme led by UNIDIR with the support of the OHCHR are not publicly listed, known industry actors involved include Microsoft, Hitachi America, Ltd and the Japan Defense Technology Foundation: UNIDIR, “Framework of Responsible Industry Behaviour for AI in the Military Domain,”
https://unidir.org/framework-of-responsible-industry-behaviour-for-ai-in-the-military-domain.
[71] UNIDIR, “Framework of Responsible Industry Behaviour”.
[72] Rupert Barrett-Taylor and Natasha Karner, “AI Could Support the Military but Won’t Replace Generals,” Alan Turing Institute, https://www.turing.ac.uk/news/ai-could-support-military-wont-replace-general, 7, 9.
[73] Woodcock, “Human-Machine (Learning) Interactions”; Jessica Dorsey, “The Erosion of Human(e) Judgement in Targeting? Quantification Logics, AI-Enabled Decision Support Systems and Proportionality Assessments in IHL” International Review of the Red Cross 107, no. 930 (2025): 1041–1071. Barrett-Taylor situates contemporary developments for AI in warfare in the historical trend of militaries using quantification and data processing to manage complexity since at least the 1950s: Rupert Barrett-Taylor, "A Digitized, Efficient Model of War," Carnegie Endowment for International Peace (June 2025), https://carnegieendowment.org/research/2025/06/a-digitized-efficient-model-of-war.
[74] Renic and Schwarz “Crimes of Dispassion,” 337-338. Spaulding highlights that ‘the displacement of human judgment and the deterioration of conditions for its exercise… are signal attributes of bureaucratic systems’: Norman W. Spaulding, "Is Human Judgment Necessary?: Artificial Intelligence, Algorithmic Governance, and the Law," in The Oxford Handbook of Ethics of AI, ed. Markus D. Dubber et al. (Oxford: Oxford University Press, 2020), 392.
[75] See e.g., Marco Sassòli, ‘Targeting: The Scope and Utility of the Concept of “Military Objectives” for the Protection of Civilians in Contemporary Armed Conflicts’ in David Wippman and Matthew Evangelista (eds) New Wars, New Laws? Applying the Laws of War in 21st Century Conflicts (Transnational Publishers 2005) 181-210, 204; Amanda Alexander, “A Short History of International Humanitarian Law,” European Journal of International Law 26, no. 1 (2015): 109-138.
[76] Klaudia Klonowska and Taylor Kate Woodcock “Rhetoric and Regulation”; Rehak and Woodcock, “Automating Civilian Harm”; Henning Lahmann, "Bad Algorithms”.
[77] “How Tech Giants Turned Ukraine Into an AI War Lab,” TIME, https://time.com/6691662/ai-ukraine-war-palantir/.
[78] Taylor Kate Woodcock, “Human/Machine(-Learning) Interactions, Human Agency and the International Humanitarian Law Proportionality Standard,” (2024) Global Society, 38, no 1: 100–121; Woodcock, “Human-Machine (Learning) Interactions”.
[79] Yuval Abraham, “‘Lavender’: The AI Machine Directing Israel’s Bombing Spree in Gaza,” +972 Magazine, 2024, https://www.972mag.com/lavender-ai-israeliarmy-gaza.
[80] Rainer Rehak and Taylor Kate Woodcock, “Automating Civilian Harm: On Israel’s Use of the AI-Enabled Targeting System Lavender in Gaza and International Humanitarian Law,” in FAccT ’26: The 2026 ACM Conference on Fairness, Accountability, and Transparency, 4885–98, https://doi.org/10.1145/3805689.3812357.
[81] Woodcock, “Human-Machine (Learning) Interactions”.
[82] Zeerak Waseem et al., “Disembodied Machine Learning: On the Illusion of objectivity in NLP” arXiv:2101.11974, 2021 https://arxiv.org/abs/2101.11974v1.
[83] Cummings and Li, “Subjectivity in the Creation of Machine Learning,” 2-3. These include: ‘Picking the modeling approach to be used, Picking which features should be included out of large datasets, Determining whether to drop cases with missing data or to generate missing data estimates, Picking a p value for statistical significance, Deciding numbers of neurons for hidden NN layers, Picking the maximal training iteration for the NN training process, Picking stopping rules for training performance factors (software-dependent), Selecting data training and testing ratios, Picking threshold values between binary unbalanced outcomes, Choosing thresholds between important/unimportant features, Determining whether model accuracy is good enough, Deciding what the actual important features for a model’: 14.
[84] Laurence Diver, “Law as a User: Design, Affordance, and the Technological Mediation of Norms,” SCRIPTed 15, no 1. (2018): 4.
[85] Antoine Smallegange, Jurriaan van Diggelen, and Jonathan Kwik, “Characteristics of the Military Domain and Their Impact on Military AI Applications,” in 2025 International Conference on Military Communication and Information Systems (ICMCIS 2025): Oeiras, Portugal, 13–14 May 2025 (Piscataway, NJ: IEEE, 2025), 232–41, https://doi.org/10.1109/ICMCIS64378.2025.11047950.
[86] Zachary C. Lipton, "The Mythos of Model Interpretability," arXiv, 1606.03490 (2016), https://arxiv.org/abs/1606.03490, 5; Riccardo Guidotti et al., "A Survey of Methods for Explaining Black Box Models," arXiv, 1802.01933, https://arxiv.org/abs/1802.01933, 1, 2, 5, 11; Brent Mittelstadt et al., "Explaining Explanations in AI," arXiv, 1811.01439 (2019), https://arxiv.org/abs/1811.01439, 2.
[87] Taylor Kate Woodcock, “Machine Learning: AI, Algorithmic Arbitrariness and Legal Decisions,” in AI, Law and Politics, eds Matilda Arvidsson and Kieran Tranter (Routledge, forthcoming 2026).
[88] Matteo Pasquinelli and Vladan Joler, "The Nooscope Manifested: AI as Instrument of Knowledge Extractivism," AI & Society 36 (2021): 1263–1280, 1278; Waseem et al, “Disembodied Machine Learning,” 1.
[89] Meltem Bostanci and Özgür Yilmaz, “Labour and Exploitation Processes in Artificial Intelligence: Example of Digital Taylorism in Data Labelling,” Journal of Social & Cultural Studies 15 (2025): 24-48; Julian Posada, "Embedded Reproduction in Platform Data Work," Information, Communication & Society 25, no. 6 (2022): 816–834.
[90] Pasquinelli and Joler, “The Nooscope Manifested”; Milagros Miceli, Martin Schuessler, and Tianling Yang, "Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision," arXiv, 2007.14886 (2020), https://arxiv.org/abs/2007.14886, 2.
[91] Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, and Klaus Mueller, "Measuring Social Biases of Crowd Workers using Counterfactual Queries," arXiv, 2004.02028 (2020), https://arxiv.org/abs/2004.02028; Christoph Hube, Besnik Fetahu, and Ujwal Gadiraj, "Understanding and Mitigating Worker Biases in the Crowdsourced Collection of Subjective Judgments," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York: ACM, 2019), 407:1–12, https://doi.org/10.1145/3290605.3300637; Fabian L. Wauthier and Michael I. Jordan, “Bayesian Bias Mitigation for Crowdsourcing,” in Advances in Neural Information Processing Systems 24 (NIPS 2011), https://people.eecs.berkeley.edu/~jordan/papers/wauthier-jordan-nips12.pdf.
[92] danah boyd and Kate Crawford, “Six Provocations for Big Data,” (2011) SSRN <https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431> accessed 1 June 2025, 5.
[93] Cummings and Li, “Subjectivity in the Creation of Machine Learning,” 3.
[94] Gary Marcus, ‘The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence’ (2020) arXiv
2002.06177, 5-6, 16.
[95] Neil Fraser, “Neural Network Follies,” accessed July 5, 2026, https://neil.fraser.name/writing/tank/.
[96] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, "Why Should I Trust You?: Explaining the Predictions of Any Classifier," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016). Alex A. Freitas, "Comprehensible Classification Models – A Position Paper," ACM SIGKDD Explorations Newsletter 15, no. 1 (2014): 2.
[97] Woodcock, “Human-Machine (Learning) Interactions”.
[98] Ibid.
[99] Mireille Hildebrandt, “Data-Driven Prediction of Judgment: Law’s New Mode of Existence?” in Collected Courses of the Academy of European Law (Oxford: Oxford University Press, 2019), EUI Summer School, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3548504, 6.
[100] Concerns around bias in targeting, such as the targeting of military-aged men from various regions as militia members, predates the use of AI systems, see Kristina Benson, "Kill 'em and Sort it Out Later: Signature Drone Strikes and International Humanitarian Law," Pacific McGeorge Global Business & Development Law Journal 27, no 1 (2014): 17-51, 34–36.
[101] Woodcock, “Human-Machine (Learning) Interactions”.
[102] Netta Goussac and Vincent Boulanin, "Responsible Procurement of Military Artificial Intelligence" (2026), SIPRI, https://www.sipri.org/sites/default/files/2026-02/0226_milai_procuremen…, 6–7.
[103] An example is in the United States, which implemented the Better Buying Power Directive and Tradewind initiative to foster more efficient military acquisition: US Defense Logistics Agency, "Better Buying Power 3.0 Stresses Innovation, Affordability," https://www.dla.mil/About-DLA/News/News-Article-View/Article/633556/better-buying-power-30-stresses-innovation-affordability/; Tradewind AI, https://www.tradewindai.com/.
[104] [Insert cross-reference to Manoj Harjani’s special issue paper on TEVV].
[105] Alan Backstrom and Ian Henderson, “New Capabilities in Warfare: An Overview of Contemporary Technological Developments and the Associated Legal and Engineering Issues in Article 36 Weapons Reviews,” International Review of the Red Cross 94, no 886 (2012): 483-514, 508.
[106] Ibid.
[107] Ibid, 509, suggesting that the transposition of legal requirements for the purposes of review should be ‘testable, quantifiable, measurable, and reasonable’.
[108] Gary Marcus, ‘The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence’ (2020) arXiv 2002.06177, 39; François Chollet, ‘On the Measure of Intelligence’ (2019) arXiv:1911.01547, 3.
[109] Missy L Cummings, ‘Artificial Intelligence and the Future of War’ in Chatham House (ed) Artificial Intelligence and International Affairs: Disruption Anticipated (2018) http://chathamhouse.org/sites/default/files/publications/research/2018-06-14-artificial-intelligence-international-affairs-cummings-roff-cukier-parakilas-bryce.pdf, 12.
[110] Peter Layton, “Fighting Artificial Intelligence Battles: Operational Concepts for Future AI-Enabled War,” (Working Paper, Griffith University, 2021), doi:10.51174/JPS.004, 14.
[111] Martin Hagström, “Military Applications of Machine Learning and Autonomous Systems,” in The Impact of Artificial Intelligence on Strategic Stability and Nuclear Risk: Volume I Euro-Atlantic Perspectives, ed. Vincent Boulanin (SIPRI, 2019), 32–38, 37; ICRC, "Autonomy, Artificial Intelligence and Robotics: Technical Aspects of Human Control" (2019), https://www.icrc.org/sites/default/themes/icrc_theme/images/document-fi…, 3. Boulanin points to the lack of reliable methods for testing and evaluating the performance of systems that are “complex, adaptive and nonlinear”: Vincent Boulanin, “Implementing Article 36 Weapon Reviews in the Light of Increasing Autonomy in Weapon Systems,” SIPRI Insights on Peace and Security, no. 2015/1 (2015): 15.
[112] Hagström, “Military applications of machine learning,” 37-38.
[113] Brian K. Hall, "Autonomous Weapons Systems Safety," Joint Force Quarterly 86 (2017): 86–93, 89.
[114] Boulanin, “Implementing Article 36,” 15.
[115] This can include what has been referred to as ‘user-specific’ explainable AI failures: Clara Bove et al., “Why do explanations fail? A typology and discussion on failures in XAI,” arXiv, https://arxiv.org/html/2405.13474v1, 7-9.
[116] ICRC,” A Guide to the Legal Review of New Weapons, Means and Methods of Warfare: Measures to Implement Article 36 of Additional Protocol I of 1977,” International Review of the Red Cross 88, no 864 (2006), 937.
[117] Sandoz et al., Commentary on the Additional Protocols, para 1473.
[118] Justin McClelland, “The review of weapons in accordance with Article 36 of Additional Protocol I” International Review of the Red Cross 85, no 850 (2003): 397-415, 413.
[119] Woodcock, “Human-Machine (Learning) Interactions”.
[120] Klaudia Klonowska and Taylor Kate Woodcock ‘Rhetoric and Regulation: The (Limits of) Human/AI Comparison in Legal Debates on Military AI’ in Bérénice Boutin, Taylor Kate Woodcock and Sadjad Soltanzadeh (eds), Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence (Asser Press 2026).
[121] Woodcock, “Human-Machine (Learning) Interactions”.
[122] Taylor Kate Woodcock, “Human/Machine(-Learning) Interactions, Human Agency and the International Humanitarian Law Proportionality Standard,” (2024) Global Society, 38, no 1: 100–121.
[123] Boutin, “State Responsibility,” 146.
[124] This is acknowledged by the range of ‘lifecycle’ approaches to military AI and the important work being done on compliance-, lawfulness-, and legal protection-by-design for AI systems. See e.g., GC REAIM, “Responsible by design”; Boutin and Woodcock, “Legal Aspects”; Article 36 Legal, "Lawful by Design Initiative," Article 36 Legal, 2025, https://www.article36legal.com/lawful-by-design; Mireille Hildebrandt, "Saved by Design? The Case of Legal Protection by Design," Nanoethics 11 (2017): 307–311, https://doi.org/10.1007/s11569-017-0299-0.
[125] Netta Goussac and Vincent Boulanin, “Responsible Procurement of Military Artificial Intelligence,” SIPRI, 2026, https://www.sipri.org/sites/default/files/2026-02/0226_milai_procurement_260216.pdf, 19.
[126] Ibid, 18.
[127] Backstrom and In Henderson, “New capabilities in warfare,” 513, suggesting that each discipline should ‘have enough understanding of the other fields to appreciate potential interactions, facilitate meaningful discussion, and understand their own decisions in the context of impacts on other areas of development’.
[128] Lena Trabucco, "International Humanitarian Law and Lethal Autonomous Weapon Systems" (Centre for Military Studies, University of Copenhagen, 2024), https://cms.polsci.ku.dk/publikationer/folkeretten-og-doedbringende-autonome-vaabensystemer/International_Humanitarian_Law_and_Lethal_Autonomous_Weapon_Systems.pdf.
[129] Trevor Taylor, “Artificial Intelligence in Defence: When AI Meets Defence Acquisition Processes and Behaviours,” RUSI Journal 164, nos. 5–6 (2019): 72-81, 76.
[130] Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Pearson, 2022) 669-670.
[131] Ibid.
[132] See e.g., Rupert Barrett-Taylor and Matthew Ford, "Eroding Sovereignty in the Age of ‘War as a Service,’" Opinio Juris, November 3, 2025, https://opiniojuris.org/2025/11/03/eroding-sovereignty-in-the-age-of-war-as-a-service/.
[133] Klaudia M Klonowska, Techno‑Legal Tinkering in War: AI Decision‑Support Systems and International Humanitarian Law, PhD dissertation, Universiteit van Amsterdam, 2026 (on file with the author), 26-27. In another publication, Klonowska highlights some of the subjectivities that become embedded into AI systems through ‘tinkering’ processes resulting in feedback loops, including performance monitoring and model retraining that use methods involving user feedback for offline learning systems. She highlights that maintaining performance and reliability of such systems involves consistent redevelopment, not only creating challenges for ensuring the lawfulness of military operations based on the reliability of these systems, but also for conducting meaningful review of the lawfulness of these systems before they result in harm: Klaudia Klonowska, “Elusive reliability: Techno-legal considerations on the challenges of maintaining and reviewing performance of AI in military applications,” Computer Law & Security Review 6, no. 1063181 (2026):1-22, 6.
[134] Jun Wu, “Distribution Shifts in Trustworthy Machine Learning,” AI Magazine 47 (2026): 1–15, https://onlinelibrary.wiley.com/doi/pdf/10.1002/aaai.70057, 1.
[135] See e.g., Sophia Sheera, "US Military Used Musk’s Grok AI to Fire 2,000 Missiles at Iran: Court Documents Reveal the Trillionaire’s Role in Operation Epic Fury," Novara Media, June 18, 2026, https://novaramedia.com/2026/06/18/us-military-used-musks-grok-ai-to-fire-2000-missiles-at-iran/; Josh Taylor, "Pokémon Go Data Trained AI That Could Assist Military Drones in War Zones," The Guardian, June 12, 2026, https://www.theguardian.com/technology/2026/jun/12/pokemon-go-data-trai…
[136] Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky. 2025. Social Sycophancy: A Broader Understanding of LLM Sycophancy. arXiv preprint arXiv:2505.13995. onathan Kwik, "Digital Yes-Men: How to Deal with Sycophantic Military AI?," Global Policy 16, no. 3 (July 9, 2025): 467–473.
[137] Wu, “Distribution Shifts,” 1.
[138] Keeping in mind that violations for unintended engagements will turn on the reasonableness of commanders’ decisions, this raises the question of whether reasonable system engineering choices are also part of this calculation, or the ‘reasonable military software engineer’.
[139] See e.g., Damian Copeland and Luke Reynoldson, ‘How to Avoid “Summoning the Demon”: The Weapons Review of Weapons with Artificial Intelligence’ Pandora’s Box 97(2017): 106–8; Damian Copeland, A Functional Approach to the Legal Review of Autonomous Weapon Systems, International Humanitarian Law Series, vol 72 Brill 2025), 182; Netta Goussac et al., Enhancing the Legal Review of Autonomous Weapon Systems: Report of an Expert Meeting (Sydney, 28–30 March 2023) (Brisbane: Law and the Future of War Research Group, TC Beirne School of Law, The University of Queensland, 2023), doi:10.14264/2bbfd31; Jonathan Kwik, "Iterative Assessment for Military Artificial Intelligence (AI) Systems," in Legal, Ethical, and Technical Dilemmas in Military Artificial Intelligence, ed. Bérénice Boutin, T. K. Woodcock, and S. Soltanzadeh (The Hague: T.M.C. Asser Press, 2026); Klaudia Klonowska and Jonathan Kwik, “Artificial intelligence in contemporary conflicts and the future of military law,” in A Research Agenda for Military Law by (eds.) Paul A.L Ducheine, Terry D. Gill, Peter B.M.J. Pijpers, and Marten C. Zwanenburg (Edward Elgar 2026) 107-108.
[140] Alex Krasodomski, “Anthropic’s feud with the Pentagon reveals the limits of AI governance,” Chatham House, March 4, 2026, https://www.chathamhouse.org/2026/03/anthropics-feud-pentagon-reveals-limits-ai-governance.
[141] Erin Griffith, “Google Won’t Renew Controversial Pentagon AI Project,” Wired, June 1, 2018, https://www.wired.com/story/google-wont-renew-controversial-pentagon-ai-project/.
[142] Palantir Technologies Inc., “Palantir Expands Maven Smart System AI/ML Capabilities to Military Services,” March 2024, https://investors.palantir.com/news-details/2024/Palantir-Expands-Maven-Smart-System-AIML-Capabilities-to-Military-Services/.