Contribute

Military AI Systems and the Design of Lifecycle-Sensitive Governance Frameworks

Authors: Rob McLaughlin[1] and Conor McLaughlin[2]

Introduction

As Natalie Nunn recently observed in the context of using the CCW process and experience as guide for AWS governance development,

If one reads the CCW closely, a pattern emerges. Where States drew clear, technically legible regulatory lines early in weapons development, as with blinding laser weapons, the law shaped development pathways. Where States relied on incremental restriction for a weapon already entrenched in their arsenals, as was the case with landmines, regulation became longer, harder, and ultimately vulnerable to geopolitical reversal. And most importantly, where definitional choices left loopholes, such as the case with incendiary weapons, the law struggled to keep up with how weapons were actually used.[3]

Nevertheless - as a general rule – on those few occasions where the law has tried to get ahead of technology – to pre-empt through prohibition - this has often led to unintended consequences, or is subsequently considered to be ‘off with the fairies’. An example of the former is flattening and expanding rounds in so far as these rounds have become more evolved since their early prohibition.[4] Such rounds are today regularly used by policing and counter-terrorism forces (including military forces) because of their ‘stopping power’, whilst remaining prohibited to military forces in operations governed by LOAC.[5] An example of the latter is ENMOD[6], which was negotiated in the 1970s in relation to suspicions and fears around use of technology to weaponize the environment that have not eventuated to date, although it is possible that ENMOD will become more acutely relevant in the near term if current reports about the Russian Poseidon nuclear torpedo (which could create a radioactive tsunami) are accurate.[7] It is also instructive that the ENMOD idea was in part a ‘look over there’ Cold War era distraction to keep non-nuclear weapon states away from seeking to engage in treaty negotiations on nuclear weapons. As Emily Crawford has noted, the ENMOD was in some respects ‘attributable to a desire, on the part of both the US and the USSR, to be seen as acting on disarmament issues, if not actually acting on disarmament issues’.[8] Indeed, it is interesting to ask whether the current mode of engagement by some states with existing negotiations around AI in a military context is likewise an attempt to be seen to be acting, whilst in fact hampering action on this issue.

There is no reason to suppose that the current legal debate around regulation of military AI, or AI with military uses, will not run very similar risks. The preoccupation with defining and classifying, rather than ‘regulating backwards from desired effects and state practice’[9], for example, is an indicator of this challenge. Another indicator is continuing debate as to the adequacy of the current rules to regulate AWS generally. On one hand, for example, the Secretary-General’s 2025 Report in response to UNGA Resolution 79/239[10] observes that there is a clear concern as to the application of existing law, including IHL:

The affirmation by the General Assembly in its resolution 79/239 that international law, including the Charter of the United Nations, international humanitarian law and international human rights law, applies throughout the life cycle of AI is an important baseline. However, important questions on how the law applies remain to be resolved.[11]

Similarly, the 2025 Draft articles on autonomous weapon systems[12] proposes that the idea of new ‘articles’ on (in this case) LAWS IHL regulation is not about new ‘hard law’, but rather is designed to support proper and coherent interpretation of existing specific IHL rules.

On the other hand, however, it is the equally clear view of many stakeholders that the existing governance frameworks for AI in a military context are already inadequate to the technology, and the solution is to develop a fresh governance framework based on new treaty instruments and rules.[13] This option requires new ‘hard law’ – that is, legally binding rules. Hard law is generally considered to be rules contained within the sources of law set out in Article 38(1) of the ICJ Statute:

  1. international conventions, whether general or particular, establishing rules expressly recognized by the contesting states;
  2. international custom, as evidence of a general practice accepted as law;
  3. the general principles of law recognized by civilized nations;
  4. subject to the provisions of Article 59 [ICJ decisions bind only the parties to it], judicial decisions and the teachings of the most highly qualified publicists of the various nations, as subsidiary means for the determination of rules of law.[14]

However, designing a governance framework that requires new hard law at its core will face significant practical challenges. Chief amongst these is that the ambition to negotiate, agree, and then implement a new treaty on AI in a military context is an almost impossible ask in the current geo-political context.[15] This is perhaps most readily evident in the glacial pace of negotiations and debate in international fora, and the fact that it has taken at least a decade to build the political will to bring the topic even to the General Assembly:

While the General Assembly has adopted multiple resolutions related to the peaceful uses of AI, it only began considering the subject of AI in the military domain at its seventy-ninth session 2024. Previously, such discussions had taken place in several plurilateral groups outside the UN.[16]

The consequence is, in terms of hard law, that we need to use what we already have, which means that we need to regulate by leveraging and interlocking existing hard law general principles and specific prohibitions with new soft law instruments.

Soft law, in contrast to hard law, is rules that are politically / operationally, and interpretively, influential, but not legally binding. Daniel Thurer defines soft law as follows: ‘soft law is used in legal literature to describe principles, rules, and standards governing international relations which are not considered to stem from one of the sources of international law enumerated in Art. 38(1) ICJ Statute’. [17] 

However, because soft law is less formalised, not governed by the Vienna Convention on the Law of Treaties (VCLT) and treaty-making processes, and not legally binding, it can be developed and applied in myriad ways and formats, and through a much wider array of fora and institutions. States are (currently) moderately more willing to engage in soft law projects than in hard law projects[18], and so using soft law mechanisms to stitch together existing hard law rules in a life-cycle governance regime for AI in a military context is a significantly more realistic (albeit still restrained) ambition than negotiating a new hard law regulatory regime specifically targeted at AI in a military context.

Consequently, this paper argues that a new hard law enterprise is not (currently) the right approach to take with respect to designing AI lifecycle governance frameworks for military contexts. Rather, the paper proposes three guardrails that could perhaps usefully inform the development – or rather evolution – of a governance framework for AI in military contexts that is based in existing rules, mechanisms, and processes. The paper then briefly notes some of these key components of a lifecycle governance framework that are already in place, and also the need to identify the point of diminishing returns by which a new regime would need to be developed.

Three Guardrails to Inform the Design of a Useful and (for the Time Being) Enduring Governance Framework

The basic proposition underpinning the following guardrails is that it is unrealistic, in the current context, to assert that a single governance mechanism, or a single metric, might be developed to cover the life-cycle of AI in a military context. Certainly, LOAC and weapons law are essential components of any such regime, but they cannot provide the totality of that regime because they cannot cover a complete life-cycle. Nor can LOAC and weapons law cover the penumbra that surrounds technology regulation more broadly, such as export control. That is, the development of an effective life-cycle governance regime for AI in military contexts will also need to learn from, and employ elements of, a wide range of weapons- and technology-adjacent regulatory schemes such as non-proliferation law[19], the Arms Trade Treaty[20], international standards regulatory systems, incident investigatory regimes, and so on. That is, an effective life-cycle governance regime will of necessity need to be piecemeal, leveraging different regulatory mechanisms, concepts, and options for different aspects of the life-cycle – design; precursor management; trade and technology transfer; development and acquisition; test and evaluation; employment; updating; decommissioning. It is difficult to envisage, for example, what a singularly focussed and all-encompassing regulatory treaty would look like in respect of life-cycle governance of AI in a military context. It is more realistic to envisage such a regime as a lego of interlocking or touching mechanisms and sub-regimes regulating different aspects of the technology’s use in military contexts in a coherent, linear, albeit sectoral manner. An example of relative (albeit currently stressed) success in such a lego regime approach is nuclear non-proliferation, created through an interlocking set of chapeau treaties (eg, the 1968 Nuclear Non-proliferation Treaty[21], and the 1996 Comprehensive Nuclear Test-Ban Treaty[22]), implementation treaties (such as the Comprehensive Safeguards Agreements between states and the IAEA[23]), sectoral enforcement treaties (such as the International Convention for the Suppression of Acts of Nuclear Terrorism[24], and the 1988 Suppression of Unlawful Activities at Sea Convention 2005 Protocol[25]), liability treaties (1977 Vienna Convention on Civil Liability for Nuclear Damage[26]), security and transport treaties and guidelines (including the 1979 Convention on the Physical Protection of Nuclear Material and its 2005 Protocol[27] and the regularly updated IAEA Regulations for the Safe Transport of Radioactive Material[28]), and other soft law instruments designed to fill gaps and join the seams of hard law regimes (such as UNGA Resolutions[29] and Codes of Conduct[30]). With this in mind, we propose three guardrails to help inform the development of a life-cycle governance regime for AI in military contexts.

Don’t Regulate Prospective Technology; Regulate Process

As noted above, attempts to comprehensively regulate specific new technologies before their scope and utility is adequately understood (flattening and expanding rounds), or prospectively before they are developed (ENMOD), carries risk. One risk is unintended consequence, such as inconsistency or even illogicality in operational limitations. Another risk is the regulatory regime’s inability to evolve as the technology evolves because the prohibition was pre-emptively comprehensive – particularly considering the intention for AI to learn, adapt, and innovate.[31] Another risk is that precursor comprehensive regulation of a specific military technology can also roadmap for states how to avoid regulation by developing similar weapons effects in a particular way so as to avoid the application of that regulatory instrument.

An alternative approach is to regulate the process by which technologies – including anticipated technologies – are to be developed, acquired, assessed, and employed. This approach is point-in-time neutral, and can generally be achieved through one or more interlocking process-based regulatory approaches, three of which are highly relevant to AI: The ‘system of control’; principles based regulation; and / or assurance regulation. However, it is essential to note at the outset that the solution is unlikely to be found solely in one or other of these approaches. Rather, the solution will likely be found in an amalgam and synchronisation of these various approaches. That will in itself present an organisation=al and regulatory challenge, but – again – it is a challenge of sequencing and linking rather than of ab initio creation.

The System of Control Approach

In its 2019 non-paper submitted to the GGE, Australia proposed a ‘system of control’ approach to LAWS regulation.[32] In this paper, the concept of ‘system of control’ was defined as follows:

A system of control is an incremental, layered approach to applying control, covering all aspects of a weapon system from design through to engagement.[33]

That is, a system of control approach employs already existing legal processes and rule sets in an interlocking, mutually reinforcing, and life-cycle focussed manner, to provide a holistic governance regime. Components of this system of control include research and development regulations, Article 36 reviews (including as an ongoing obligation[34]), targeting processes, command responsibility assessments, and end of lifecycle decommissioning requirements. Certainly, the military uses of AI are broader than just LAWS, but there is no obvious reason why a similarly lifecycle focussed amalgam of processes would not be achievable for this broader capability. Indeed, an approach that covers all military applications of AI would, from a regulatory coherence perspective, be far preferable to a stove-piped set of parallel but separate regulatory schemes.

Principles-Based Regulation

UK policy states:

We will work closely with allies and partners to build consensus, promote a common vision for the safe, responsible and ethical use of these technologies globally, and push for compliance with International Humanitarian Law.[35]

This approach differs from the system of control approach in that it focusses upon principles rather than specific system-embedded rules. This approach also tends to be more ecumenical in terms of the sources of, and concepts for, a regulatory system, precisely because it is based in principles rather than rules. For example, the UK approach to principles based regulation of AI and LAWS, as noted above, includes reference to safety, responsibility, and ethics, as well as IHL, as sources for these principles. However, the challenge with this approach as compared to the system of control approach, is that it would be ineffective without a precursor (and likely contentious) negotiation as to the fundamental principles which will underpin the governance regime, whereas the system of control approach tends to leverage a matrix of already existing rules and rule sets. And as SIPRI have noted, the diversification and fragmentation of fora and the breadth of the debate, and the trade-offs that will be required to crystalise outcomes, will continue to challenge translation of principles into practical guidelines.[36]

Assurance-Based Regulation

The assurance approach focusses particularly on the substantive, measurable content of an ‘assurance’ outcome. That is, the assurance approach is animated by the requirements of generating adequate levels of trust, rather than by a focus on direct, compliance-indicating, specific hard law obligations. An example is the DoD/W’s ‘AI ethical principles’, which assess five factors within its assurance metric:

Responsible. DoD personnel will exercise appropriate levels of judgment and care, while remaining responsible for the development, deployment, and use of AI capabilities.

Equitable. The Department will take deliberate steps to minimize unintended bias in AI capabilities.

Traceable. The Department’s AI capabilities will be developed and deployed such that relevant personnel possess an appropriate understanding of the technology, development processes, and operational methods applicable to AI capabilities, including with transparent and auditable methodologies, data sources, and design procedure and documentation.

Reliable. The Department’s AI capabilities will have explicit, well-defined uses, and the safety, security, and effectiveness of such capabilities will be subject to testing and assurance within those defined uses across their entire life-cycles.

Governable. The Department will design and engineer AI capabilities to fulfill their intended functions while possessing the ability to detect and avoid unintended consequences, and the ability to disengage or deactivate deployed systems that demonstrate unintended behavior.’[37]

An assurance focussed governance regime, consequently, is not prima facie concerned with potentially hard to pin down granular definitions of the technology itself, or the details of how or why it is used, but rather with how contextually trustworthy it is assessed to be. It differs from the system of control approach in that it is governed by an outcome metric - trust – rather than a list of specific rule compliances in the context of a relatively linear set of metrics. Assurance also differs from principles based governance in that it is targeted at a perhaps more explicable and unitary measure – trust – rather than at a matrix assessment of potentially differently weighted and situationally variable principles. However, as with the principles based approach, the development of an agreed assurance metric (such as trust) again presupposes a pre-cursor agreement as to how that metric is defined and measured.

Conclusion

Use of process based regulatory systems for as yet incompletely evolved and understood technologies (including AI technologies) that are capable of use in a military context is one method by which some of the risks of unintended limitation, ‘rainbow unicorn’ regulation, and providing a roadmap for avoiding regulation – which can result from a focus on the specific technology rather than the process of regulation - might be avoided.

Don’t Regulate to the Nightmare (Terminator); Regulate to What Is Actually in the ‘Pipeline’

It is difficult to argue against the proposition that The Terminator is currently used as a touchpoint for debate around how to regulate AI in a military context. As one group of scholars have recently asserted of this linkage,

The notoriety of LAWS is not surprising given the predatory, apocalyptic light in which they are commonly cast. In the media and elsewhere, LAWS are frequently identified with the killer robots of The Terminator movie.[38]

Or, perhaps more viscerally, as Ford has observed,

Who, at this point, doesn’t approach a discussion of lethal autonomous weapons systems without thinking, even inadvertently, of the villainous “Skynet” . . . and the robot-assassins it dispatches against the noble but hard-pressed remnants of humanity in Hollywood’s Terminator franchise?[39]

Menadue has similarly argued that ‘the Terminator is … considered the poster boy for any debate on lethal autonomous weapons’.[40]

The problem with trying to regulate to the nightmare as the first priority rather than focusing upon what is actually and realistically anticipated – that is, basing regulatory assumptions upon possible future, but as yet impractical, technology, rather than near-term anticipated technological evolution – is precisely that it runs the risk of not dealing with what is in train. It is the ‘now’ and the ‘soon’ which ought to be the initial focus of an effective and implementable regulatory system.[41] The problem with focussing on ‘sci-fi’ technology that does not yet exist, as John Lewis argued as far back as 2015, is that ‘the time to act is now’ arguments in relation to these sci-fi techno-fears tend to ‘exaggerate the danger posed … and underappreciate regulation’s potential to respond to the threats.[42] This is not to say that the Terminator nightmare should be ignored; rather it is to say that achieving a stable and scalable regulatory scheme for the now and the soon should provide experience that can be deployed to address this more distant concern if and as it evolves.

Don’t Focus on One Mechanism or a Single Novel Metric; Regulate by Multiple Mechanisms and by Reference to Already Existing and Defined Metrics and Contexts

The proposition behind this guardrail is that it is impractical to seek to create and define a new regulatory silver bullet - such as meaningful human control, a concept that has preoccupied the CCW discussions for more than decade thus far.[43] Rather, a less fraught option may be to choose metrics or limiters that are more stable and enduring - such as domain, place in the ‘kill chain’, and / or consequence – even if this requires stitching together multiple processes rather than the development of a singular, stand-alone, all-encompassing process.

Meaningful human control has been defined as:

Elements of human control’ which are ‘types of control measures necessary to ensure legal compliance, ethical acceptability and operational utility … any kind of measure that acts as a necessary constraint on AWS and that could form the basis of internationally agreed standards or rules.’[44] 

It is the assumed lack of such meaningful, humanly defined and recognisably human, controls that is the essence of the MHC debate. But the problem with focussing so heavily on MHC as the silver bullet metric for governance is that – as a concept - it is difficult to define and operationally assess, and has little textual support in existing law. As Gerald Mako has observed of MHC

[T]he Gaza war already pointed to MHC’s serious limitations. Israeli Defense Force operators were reportedly approving AI-generated targets in an average of 20 seconds with minimal scrutiny, leading to a 10% error rate and thousands of civilian deaths. This underscores that while MHC may very well be effective in smaller operations, in large-scale and/or drawn-out conflicts where the speed and volume of AI decisions overwhelm operators, it is unlikely that rushed oversight will prevent IHL violations.[45]

Perhaps a better approach might be to start by regulating via an already established or known concept, metric, or paradigm. This approach would offer a discrete but scalable start point, and there are a variety of options that immediately present.

The first option is to iteratively regulate by domain. For example, a scheme could start with regulation of AI in naval warfare contexts where the issues of identification, distinction, and proportionality are to some extent easier to manage because of the platform-based targeting rules, than situations on land.[46] Once the ‘easier’ issues are resolved, the scheme could then step up to the next domain (air, cyber, space?) before then progressing to what is undoubtedly the most complex domain in this regard – land.

A second option is to regulate by place in the kill chain. This could involve beginning at one end of the spectrum and then iteratively build – or reduce – from that point. For example, a regulatory regime could begin with AI that does supply or logistics but has little or no role in the kill chain; then move to enablers that get successively closer to the end effect such as senor managers or decision support mangers; then get to AI that directly delivers a destructive effect. The lessons learned by first regulating farther from the lethal effect could then inform the evolution of the regulatory scheme as it leans further into complexity as it approaches the delivery of lethal effects.

A third option is to regulate by consequence. This approach would prioritise what it is that using the AI ‘causes’. This is more than just an analysis of risk simpliciter as it includes quantitative and qualitative assessments of the nature of the harm. This is in some ways the EU approach in respect of prohibited practices, as encapsulated in the 2024 EU Artificial Intelligence Act.[47] For example, in a military context, if the AI kills or destroys, then LOAC already has clearly applicable rules which are well embodied and embedded in the targeting process, and these rules already have a built-in balancing mechanism – the principle of proportionality. If the AI enables a consequence, then LOAC already has criteria for assessing what first and second order effects, or reverberating consequences, are relevant to the assessment[48], and also has rules about what acts or roles or conduct has the consequence of characterising that ‘thing’ as direct participation in hostilities and thus describing proximity to destructive consequences.[49] The point is that it may be more efficient, effective, and sensible to use a known paradigm, concept, or metric to underpin how we consider, assess, and sequence the regulation of AI operations, rather than trying to impose a single, new, ill-defined, and legally indeterminate metric as the regulatory silver bullet.

Conclusion

The above three guardrails are offered as suggestions for how we approach the development of a (currently) enduring and useful governance regime for AI in a military context. However, equally important to any regulatory development enterprise is the identification of necessary inputs. In this regard, there are three key inputs or components for such a regime which we can say are already in place (knowns), but two key inputs or components necessary for the regime which would need to be identified (unknowns). Additionally, there are a number of more variable components which are quantifiable only in point-in-time assessments, such as reaching the tipping point of political will to develop a governance regime. Given their fluidity, these uncertain variables such as political will not be further addressed; however, the three key knowns and the two key unknowns are more capable of general description and will be briefly outlined below.

Existing components

The three already existing components of a life-cycle governance regime for AI in a military context are the existing law, the existence of recognised mechanisms for evolutive interpretation of existing law, and a generally permissive approach to soft law development.

The substantive law / rules (and the expertise to interpret and apply them)

The Australian government has told the UN’s Group of Government Experts on LAWS that:

Australia advocates for building a shared understanding of how existing IHL applies to LAWS before pursuing any new legal instrument. Australia is fully committed to building this shared understanding through the ongoing discussions in the GGE. Accordingly, we do not support the creation of a parallel process on LAWS that would be detrimental to our collective efforts in the GGE.[50]

That is, amongst the myriad existing rule sets that directly (LOAC, weapons law, command responsibility), or indirectly (counter-proliferation, technology transfers, international standards) govern AI that can be used in military contexts, there is a patchwork of sub-rules and processes that can readily be identified and networked into a coherent governance ‘paradigm’ for the technology. For example, as Marko Milanovic recently noted of AI in military contexts within the International Criminal Law paradigm,

[I]in my view, and when it comes to international criminal justice in particular, there is a tendency in some of the literature to overemphasize the extent of these challenges and the game-changing nature of AI as a disruptive new technology

[W]e’re going to be fine with the law as it stands today, at least insofar as the core business of international criminal courts and tribunals is concerned. AI does pose challenges, but they are not so radical and transformative that we won’t be able to effectively address them.[51]

This does not mean that there is a single, clear, existing sub-rule for every current and anticipated issue – no rule set can achieve that. But the existing rules include criteria and mechanisms that also allow for any gaps or uncertainties in this patchwork paradigm to be resolved, and this is the second already existing component of a lifecycle governance regime for AI in military contexts without the need for new hard law.

Additionally, it is also essential that what might be considered peripheral rule sets should also be mined for their utility in establishing governance frameworks. A case in point is contract law, which can operate to both constrain users (governments) and creators (enterprises), and give effect to the ‘red lines’ of both the regulator and the regulated. In the first instance, contract is a sound method by which governments can build into development programs the limitations that international law imposes upon them. In the second instance, as the Pentagon-Anthropic dispute illustrates, enterprise can also seek to leverage contract to place limitations on governments and militaries in terms of the uses that AI is put to.[52]

One further benefit available from the existing rule sets is the extent to which parallel existing capabilities in interpreting and applying these rule sets – for example military and government legal advisers and legally aware operators – are essential to this endeavour. The application of human experience and expertise in the initial acceptance (eg, Article 36 weapons review), and lifecycle monitoring as against the relevant governance frameworks, will be essential to building trust in respect of AI in military contexts.

The Existence of Interpretive Mechanisms to Evolve Existing Law in the Face of New Technologies

International law already encompasses quite liberal systemic rules on interpreting extant law to fill gaps or clarify uncertainties. These range from the textual rules found in the VCLT[53], to the subsequent state practice rules[54], through to the rules for identifying CIL[55] (noting that there is scope for the rapid formation of CIL where critical advances outrun the usual pace of CIL formation – as with Nuremberg[56], space law[57], and thus potentially with AI law). That is, the existing systemic rules around international law interpretation are fundamentally evolutive rather than treating rules as frozen in time in an originalist manner. This allows for rules to be applied by analogy, extension, or by reduction from general principles, in order to ensure coherent responses to new challenges, including the advent of new technologies requiring regulation in a manner that is coherent with existing paradigms.[58] The ‘how’ of this process is often a function of non-treaty negotiation fora such as soft law development, which is the third already existing component that supports the development of a lifecycle governance regime for AI in military contexts without the new for new hard law.

The General and Highly Useful Permissiveness Around Processes for Developing Soft Law

As noted above, soft law can be found in a wide variety of sources that go well beyond the more narrowly defined sources from which hard law can be distilled. Guidelines, declarations, resolutions, expert commentaries, statements of principle, codes of conduct, state non-papers, and so on, can all be used to develop the necessary interstitial threads required to bind together existing hard law and existing interpretive methodologies into a coherent regulatory scheme. And because the fora in which soft law instruments or processes can be developed are highly varied and permissive –which is not the case for hard law – this means that using soft law as the regulatory glue for a lifecycle governance regime for AI in military contexts is achievable, acceptable, and much simpler than seeking hard law responses to identified gaps or uncertainties. This is not to ignore the limitations that can attend soft law development – non-bindingness, like-mindedness, the risk of advocacy obscuring pragmatism – but it is inescapably the case that soft law (currently) offers a more realistic path to creating a lifecycle governance framework out of existing components, than the negotiation of a new hard law instrument to that effect.

New Components Required

The previous section briefly described three existing components of the international legal system that can and are currently being leveraged to develop a lifecycle regulatory scheme for AI in military contexts. However, this should not obscure the fact that there are a number of unknowns which will still need to be grappled to ground so that the longevity of the lifecycle governance regime can be monitored and the next regime planned. That is, there is a point at which a lifecycle governance regime built upon components we already have will fail because the ‘thing’ being governed will itself change so radically and qualitatively that it is no longer recognisable to (and thus capable of being regulated by) the applicable governance paradigm. This could perhaps be called, as The Economist noted of the advent of the Claude Mythos AI, a ‘Mythos moment’. In this specific case, for example, it has been reported that the AI has not been publicly released precisely because its capabilities (including in respect of hacking and potential misuse in relation to biosecurity) are ‘unnerving’.[59] In this respect, the two key unknowns are the ‘what’ and the ‘when’.

Identifying the ‘What’ That Will Make an AI Regulatory Paradigm Shift Necessary

The first unknown is to identify ‘the thing’ – the ‘Mythos’ of the Mythos moment - that will change the paradigm as regards AI and military operations and capabilities – the advent or event that will render any existing or evolved governance approach qualitatively and systemically obsolete and ineffective. That is, we need to identify what is ‘the thing’ – the incident, the initiative, the discovery, the accomplishment, the threshold - that will fundamentally change the nature, capabilities, and / or latent malignity of AI from what we now see and what we realistically anticipate.

But identifying that thing is itself a problem. Is ‘the thing’ about the measure of, or capacity to, ‘trust’ an AI? Or is ‘the thing’ what happens when what goes on inside the black box is no longer explicable by or to humans? Or is ‘the thing’ – as Sam White and others have argued – the point at which ‘apply[ing] anthropocentric law to a situation where damage is caused by this autonomous system’ becomes intolerably difficult or even impossible?[60] Or is ‘the thing’ something else entirely unforeseen at the moment? An example might be a globally condemned AI-executed strike that has disastrous consequences – a radically scaled up version of the Minab school strike on 28 February 2026 in Iran, which was linked to Claude and Maven.[61] A more tidal example might be the point at which an accumulation of AI reliance by a military force manifests in a loss of receptiveness to political control, prompting governments to regulate more closely and intensely. Regardless of the nature of the ‘Mythos’ event, however, it is essential that someone, somewhere, is planning for the ‘what next’ once the Mythos – the paradigm shifting ‘what’ – occurs.

Identifying the ‘When’ of That ‘What’

Once the ‘what’ of the thing that will herald the necessity of a paradigm change is identified, we then need to start thinking about the ‘when’ - an identifiable point in time, technology, and / or law where AI is approaching or achieves ‘the thing’ that is, the ‘moment’ aspect of the ‘Mythos moment’. This is because, at the culminating point of the anticipated what, any governance paradigm we evolve from the tools we currently have will likely cease to be effective, or sufficiently effective, and will therefore need to be replaced. That is, the ‘when’ of the ‘what’ is the point-in-time of failure for the current regulatory paradigm. This may well be the point at which we ultimately conclude that we actually do need new hard law to underpin a (future) AI lifecycle governance regime, and therefore we ought to have some people - separate from the debate about the now – already thinking about this point-in-time.

Conclusion

It is a fact of the current geopolitical situation that the negotiation of new hard law to regulate AI in military contexts is almost certainly a non-starter. But it is also clearly arguable that (assuming political will) we already have the necessary tools to develop a functional and appropriate lifecycle governance regime for AI in military contexts. Further, it is likewise clear that in doing so, we ought to steer away from specific technology-based regulatory assumptions; rather, we should focus on processes, patchworks, soft law, existing and well defined metrics, and what is anticipated rather than what is feared. However, there will inevitably come a point at which the technology that creates or supports the use of AI in military contexts will outstrip and render redundant any existing or derivative regulatory regime we can construct with the tools we already have. In order to prepare for that future, we should not let the (currently) enduring viability of any lifecycle governance regime we develop with those tools distract some of us from the need to also be looking at the ‘what’ and the ‘when’ of a future governance paradigm shift.

What might this mean for Army (and indeed for the ADF more generally)? Three thoughts are immediately apparent. First, a focus on creating a lifecycle governance framework out of existing tools, systems, rule-sets, and architectures will require the development of deeply specialist legal officers who can understand and relate to AI operators in their own terms, and also AI operators who are legally aware and thus attuned to the governance potentialities, and threats, of both intra and extra-paradigmatic ripples and developments. Second, wargaming the unlikely, the unexpected, and the unwilled should be built into testing and analysis regimes so as to understand whether and when a paradigm-fracturing Mythos moment might eventuate - and thus be mitigated against - regardless of the chance of actual occurrence. The lessons learned from wargaming the worst scenarios from AI-linked operations will offer a sound basis from which to develop possible indicators and warnings that a Mythos moment may be on the horizon (and thus avoided). Third, further research into how use of AI in military contexts can be infused with chapeau influences such as command intent, political control, and campaign appreciation should be prioritised. This is not an issue of tactical or meaningful human control; rather, it is an essential means to mitigate down to the lowest practicable level the possibility that the Mythos moment arises as a result of the military’s operational loss of responsiveness to, or imbued-ness by, strategic and political oversight.

Endnotes

[1] Professor of Law, Australian National Centre for Ocean Resources and Security (ANCORS).

[2] Research Associate, ANCORS. Both authors are most grateful for the very useful comments provided at the Workshop and by the peer reviewers. All errors remain the authors’ alone.

[3] Natalie Nunn, ‘How Lessons Learned from the CCW Can Help Guide the Regulation of AWS’, Articles of War, 06 February 2026 - https://lieber.westpoint.edu/how-lessons-learned-from-ccw-can-help-guid….

[4] 1899 Hague Declaration on the Use of Bullets Which Expand or Flatten Easily in the Human Body - https://avalon.law.yale.edu/19th_century/dec99-03.asp; ICRC, Customary IHL Study, Rule 77 - https://ihl-databases.icrc.org/en/customary-ihl/v1/rule77#refFn_BAA8276….

[5] ICRC Customary IHL Study, rule 77 -https://ihl-databases.icrc.org/en/customary-ihl/v1/rule77.

[6] 1976 Convention on the Prohibition of Military or any Hostile Use of Environmental Modification Techniques - https://ihl-databases.icrc.org/en/ihl-treaties/enmod-1976.

[7] Silky Kaur, ‘One nuclear-armed Poseidon torpedo could decimate a coastal city. Russia wants 30 of them’, Bulletin of the Atomic Scientists, 14 June 2023 - https://thebulletin.org/2023/06/one-nuclear-armed-poseidon-torpedo-coul…; Brandon Weichert, ‘Russia’s “Poseidon” Nuclear Drone Is Way Scarier Than You Think’, National Interest, 31 October 2025 - https://nationalinterest.org/blog/buzz/russias-poseidon-nuclear-drone-w….

[8] Emily Crawford, ‘Accounting for the ENMOD Convention: Cold War Influences on the Origins and Development of the 1976 Convention on Environmental Modification Techniques’, in Matthew Craven et al (eds), International Law and the Cold War, CUP, 2020, 81, 94.

[9] Callum Hamilton, comment on draft paper, 13 April 2026.

[10] UNGA Resolution 79/239 on ‘Artificial intelligence in the military domain and its implications for international peace and security’, 24 December 2024, para 7 - https://docs.un.org/en/A/RES/79/239.

[11] Report of the Secretary-General, Artificial intelligence in the military domain and its implications for international peace and security (A/80/78), 05 June 2025, para 57 (‘Secretary-General’s Report (2025)’) - https://documents.un.org/doc/undoc/gen/n25/107/66/pdf/n2510766.pdf.

[12] UK and others, Draft articles on autonomous weapon systems – prohibitions and other regulatory measures on the basis of international humanitarian law (CCW/GGE.1/2025/WP.7) 03 September 2025 - https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapo….

[13] Campaign to Stop Killer Robots: ‘We are working to have states adopt an international legal treaty that ensures meaningful human control over the use of force and rejects the automation of killing. It should do this through: ‘Prohibitions’ – banning autonomous weapon systems that do not allow for meaningful human control and banning all systems that use sensors to target humans; ‘Regulations’ – additional rules so that other autonomous weapon systems will be used with meaningful human control in practice.’ - https://www.stopkillerrobots.org/our-policies/.

[14] ICJ Statute, art 38(1) - https://www.icj-cij.org/statute.

[15] Raluca Csernatoni, ‘Governing Military AI Amid a Geopolitical Minefield’, Carnegie Endowment for International Peace, 17 July 2024 - https://carnegieendowment.org/research/2024/07/governing-military-ai-am….

[16] UN Office for Disarmament Affairs, ‘Artificial intelligence in the military domain’ -https://disarmament.unoda.org/en/our-work/emerging-challenges/artificia….

[17] Daniel Thurer, ‘Soft Law’, Max Planck Encyclopedia of Public International Law, March 2009 - https://opil.ouplaw.com/display/10.1093/law:epil/9780199231690/law-9780….

[18] Tan Hsien-Li, ‘No Longer Hard Law’s “Poor Relative”: The Growing Respect for Soft, Non-Binding Legal Instruments in the International Order’, EJIL Talk!, 06 June 2025 - https://www.ejiltalk.org/no-longer-hard-laws-poor-relative-the-growing-….

[19] Eg, see Mathew Montiel, ‘Intangible Threats: How Uncontrolled Knowledge Fuels Proliferation’, Center for Arms Control and Non-Proliferation, 08 October 2025 - https://armscontrolcenter.org/intangible-threats-how-uncontrolled-knowl….

[20] 2013 Arms Trade Treaty, eg, article 7(1)(b)(i) (export and export assessment) - https://treaties.un.org/doc/Treaties/2013/04/20130410%2012-01%20PM/Ch_X….

[21] 1968 Treaty on the Non-Proliferation of Nuclear Weapons -  https://treaties.unoda.org/t/npt.

[22] 1996 Comprehensive Nuclear Test-Ban Treaty - https://www.ctbto.org/sites/default/files/2023-10/2022_treaty_booklet_E….

[23] 1974 Agreement between Australia and the Agency for the application of safeguards in connection with the Treaty on the Non-Proliferation of Nuclear Weapons - https://www.iaea.org/sites/default/files/publications/documents/infcirc….

[24] International Convention for the Suppression of Acts of Nuclear Terrorism - https://www-ns.iaea.org/downloads/conventions-codes-resolutions/suppres….

[25] 2005 Protocol to the 1988 Convention for the Suppression of Unlawful Acts Against the Safety of Maritime Navigation - https://www.unodc.org/cld/uploads/res/treaties/definitions/treaty/proto….

[26] 1977 Vienna Convention on Civil Liability for Nuclear Damage - https://www.iaea.org/sites/default/files/infcirc500.pdf.

[27] 1979 Convention on the Physical Protection of Nuclear Material and 2005 Protocol - https://www.iaea.org/publications/documents/conventions/convention-phys….

[28] IAEA, Regulations for the Safe Transport of Radioactive Material (2018 edition) - https://www-pub.iaea.org/MTCD/Publications/PDF/PUB1798_web.pdf.

[29] Eg, UNGA Resolution 78/66: Comprehensive Test-Ban Treaty, 06 December 2023, OP 4: ‘Urges all States not to carry out nuclear-weapon test explosions or any other nuclear explosions, to maintain their moratoriums in this regard and to refrain from acts that would defeat the object and purpose of the Treaty, while stressing that these measures do not have the same permanent and legally binding effect as the entry into force of the Treaty…’ - https://docs.un.org/en/A/RES/78/66.

[30] Eg, IAEA Code of Conduct on the Safety and Security of Radioactive Sources - https://www-pub.iaea.org/MTCD/publications/PDF/Code-2004_web.pdf.

[31] Eg, Benjamin Jensen and John Paschkewitz, ‘Mosaic Warfare: Small and Scalable are Beautiful’, War on the Rocks, 23 December 2019 - https://warontherocks.com/mosaic-warfare-small-and-scalable-are-beautiful/; Benjamin Jansen and Jake Kwon, ‘The U.S. Army, Artificial Intelligence, and Mission Command’, War on the Rocks, 10 March 2026 - https://warontherocks.com/the-u-s-army-artificial-intelligence-and-miss….

[32] Australia, Australia’s System of Control and applications for Autonomous Weapon Systems (CCW/GGE.1/2019/WP.2/Rev.1), 26 March 2019 (‘Australia’s System of Control and applications for Autonomous Weapon Systems (2019)’) - https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapo….

[33] Ibid, para 3.

[34] See Damian Copeland, Rain Liivoja, and Lauren Sanders, ‘The utility of weapons reviews in addressing concerns raised by autonomous weapon systems’ (2023) 28:2 Journal of Conflict and Security Law 285, 298-300.

[35] UK, Defence Artificial Intelligence Strategy, June 2022, para 5.1.1 - https://assets.publishing.service.gov.uk/media/62a7543ee90e070396c9f7d2…

[36] Alexander Blanchard, Vincent Boulanin, Laura Bruun, and Netta Goussac, ‘Dilemmas in the policy debate on autonomous weapon systems’, SIPRI, 06 February 2025 - https://www.sipri.org/commentary/topical-backgrounder/2025/dilemmas-pol….

[37] https://www.war.gov/News/Releases/Release/Article/2091996/dod-adopts-et….

[38] Shane Reeves, Ronald Alcala, and Amy McCarthy, ‘Challenges in regulating lethal autonomous weapons under international law’ (2020) 27:1 Southwestern Journal of International Law 101, 117.

[39] CA Ford, AI, human-machine interaction, and autonomous weapons: Thinking carefully about taking ‘killer robots’ seriously, 2020 - https://www.newparadigmsforum.com/p2526.

[40] Christopher Menadue ‘Science fiction helps us deal with science fact: A lesson from Terminator’s killer robots’, The Conversation, 23 August 2017 - https://theconversation.com/science-fiction-helps-us-deal-with-science-….

[41] Jean-Baptiste Vilmer, ‘Terminator Ethics: Should We Ban “Killer Robots”?’, Ethics and International Affairs, 23 March 2015 - https://www.ethicsandinternationalaffairs.org/online-exclusives/termina….

[42] John Lewis, ‘The Case for Regulating Fully Autonomous Weapons’ (2014-2015) 124:4 Yale Law Journal 882 - https://www.yalelawjournal.org/comment/the-case-for-regulating-fully-au….

[43] See, eg: Article 36 (NGO), ‘Key elements of meaningful human control: Background Paper’, April 2016 - https://www.article36.org/wp-content/uploads/2016/04/MHC-2016-FINAL.pdf; CCW GGE Rolling Text (18 December 2025), cl III(6): ‘Context-appropriate human judgement and control is needed to ensure the use and effects of LAWS are in compliance with international law, in particular IHL, including the principles and requirements of distinction, proportionality and precautions in attack.’ - https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapo….

[44] Vincent Boulanin, Neil Davison, Netta Goussac, and Moa Carlsson, Limits on Autonomy in Weapon Systems: Identifying Practical Elements of Human Control, SIPRI / ICRC, June 2020, p 3 - https://www.icrc.org/sites/default/files/document/file_list/icrc_sipri_….

[45] Gerald Mako, ‘Legal Accountability for AI-Driven Autonomous Weapons’, Articles of War, 09 March 2026 - https://lieber.westpoint.edu/legal-accountability-ai-driven-autonomous-….

[46] See, eg, James Kraska, ‘Distinction, Proportionality, and Precautions in Attacks at Sea in the New Era of the Law of Naval Warfare’ (2025) 26:1 Chicago Journal of International Law 387, 391: ‘The standard at sea is more lenient because vessels and aircraft, rather than individuals, are targeted. Belligerents are more likely to injure civilians during attacks on land than they are to injure civilians or civilian ships while attacking a warship or other military objectives at sea.’

[47] European Union Artificial Intelligence Act (Regulation (EU) 2024/1689) Chapter II, Article 5 - https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L_202401689.

[48] Robert Kolb, ‘Indirect or Reverberating Excessive Collateral Damage in Modern IHL’, Articles of War, 13 August 2025 - https://lieber.westpoint.edu/indirect-reverberating-excessive-collatera….

[49] Eg – albeit controversially - ICRC, Interpretive Guidance on the Notion of Direct Participation in Hostilities, 2009 - https://www.icrc.org/sites/default/files/external/doc/en/assets/files/o….

[50] Australia, Australia’s Submission to the United Nations Secretary-General’s Report on Lethal Autonomous Weapons Systems (ODA-2024-00019/LAWS, May 2024, para 13 - https://docs-library.unoda.org/General_Assembly_First_Committee_-Sevent….

[51] Marko Milanovic, ‘AI and the Commission and Facilitation of International Crimes: On Accountability Gaps and the Minab School Strike’, EJIL Talk!, 09 March 2026 -https://www.ejiltalk.org/ai-and-the-commission-and-facilitation-of-inte….

[52] ‘Statement from Dario Amodei on our discussions with the Department of War’, 26 February 2026 - https://www.anthropic.com/news/statement-department-of-war.

[53] 1969 Vienna Convention on the Law of Treaties (VCLT) articles 31-32 - https://legal.un.org/ilc/texts/instruments/english/conventions/1_1_1969….

[54] Eg, ILC, Draft conclusions on subsequent agreements and subsequent practice in relation to the interpretation of treaties, 2018, conclusion 5 (conduct as subsequent practice), conclusion 8 (interpretation of treaty terms as capable of evolving over time) - https://legal.un.org/ilc/texts/instruments/english/draft_articles/1_11_….

[55] Eg, ILC, Draft conclusions on identification of customary international law, with commentaries, 2018,  conclusion 2 (constituent elements) - https://legal.un.org/ilc/texts/instruments/english/commentaries/1_13_20….

[56] Michael Scharf, ‘Accelerated formation of customary international law’ (2014) 20:2 ILSA Journal of International and Comparative Law 305, 330-335.

[57] Fabio Tronshetti, ‘Customary international law and space law’, in Mahulena Hofmann et al (eds), Elgar Concise Encyclopedia of Space Law (Elgar, 2025), Ch 13.

[58] For example, via a lifecycle based, continuing appreciation of Article 36 review obligations – see other essays in this collection.

[59] ‘The Mythos moment’, The Economist, 18 April 2025, 9.

[60] Samuel White et al, ‘Artificial Intelligence, Autonomous Drones and Legal Uncertainties’ (2023) 14:1 European Journal of Risk Regulation 31, section III.

[61] Kevin Baker, ‘AI got the blame for the Iran school bombing. The truth is far more worrying’, The Guardian, 26 March 2026 - https://www.theguardian.com/news/2026/mar/26/ai-got-the-blame-for-the-i….