MACHINE AND HUMAN BIASES: A CHALLENGE TO LETHAL AUTONOMOUS WEAPONS REGULATION

Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 1 Machine and Human Biases: A Challenge to Lethal Autonomous Weapons Regulation Ana Beatriz Duarte https://orcid.org/0000-0002-4736-4595 Manuela Le-fort Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 1 Bias in lethal autonomous weapons is twofold: in the machine (data and model choices across the lifecycle) and in use (automation bias in human-machine interaction). Recent GGE-CCW position statements and documents converge on obligations to detect, correct, and mitigate both, but they remain nonbinding and operationally thin. A legally binding instrument that guarantees standards are verifiably met could minimize the dangers of this inheritance feature of AI-powered systems. Behavioral data points used to train artificial intelligence models reflect society’s biases and prejudices toward gender, race, ethnicity, disabilities, and other demographic or social characteristics. Bias inherited by machines can cause all sorts of damage – from unfair access to opportunities to life-threatening errors – perpetuating existing societal issues. Such biases are particularly problematic in military applications such as lethal autonomous weapons systems (LAWS), in which biased decisions can result in violations of International Humanitarian Law (IHL) and disproportionately affect marginalised populations. In multilateral fora dedicated to potential regulation of these systems, some States, as well as civil society groups, have been advocating for measures that can prevent or mitigate the risks accompanying their use. This policy brief analyses the perception of bias in the debates of the 2024 (March and August) sessions of the Convention on Certain Conventional Weapons (CCW) Group of Governmental Experts on Lethal Autonomous Weapons Systems (GGE). It identifies the main actors pushing forward this debate, the most commonly used typologies and approaches when addressing the subject, and the measures proposed by States to mitigate and/or eliminate bias in lethal autonomous weapon systems. Bias is a reflection of both historic and contemporary power structures. Artificial intelligence-powered devices, too, reproduce bias as a consequence of training data. That was the case of, for example, Google Photos’ facial recognition system, which Summary Background Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 2 failed to identify black people, tagging them as gorillas (Lipton, 2016). This illustrates clearly that algorithms aren’t neutral. Similar label gaps can be perpetrated by lethal weapons that “decide” who ans whether to kill without proper human supervision. The CCW/GGE on LAWS has been one of the main forums for the debate on autonomous weapons. Even though the GGE-CCW has existed for over ten years, the debate on bias has only recently gained more granularity in the forum’s discussions. A The civil society has largely contributed to this advance, with campaigns and research highlighting the importance of the issue in CCW side events and academic publications. Some States also advocate for prevention and risk mitigation regarding bias in autonomous weapons. The subject has been mentioned in a few working papers over the years, but a recent one, entitled Addressing Bias in Autonomous Weapons (Austria et al., 2024), rendered it extra visibility. The document, issued by Austria, Belgium, Canada, Costa Rica, Germany, Ireland, Luxembourg, Mexico, Panama, and Uruguay, encouraged all High Contracting Parties to consider ways of addressing bias in a future normative framework. When addressing the topic in GGE-CCW statements, delegations tend to approach bias in different ways. Beyond biased results produced by AI systems, the subject sometimes relates to the societal discrimination itself transmitted into the system, and at other times yet refers to human bias arising from the interaction with systems. The first case has been labeled in the literature as “bias in society” (or pre-existing bias or historical bias), understood as “the way in which societies have historically been skewed towards or against certain individuals or groups of people, typically along lines such as ethnicity, gender, class or ability” (SIPRI, 2024). The second type is called “bias in use”, which refers to unwanted behaviours or decisions the human agent may make while deploying AI systems. AI systems can replicate bias in different phases of their life cycles, from development to deployment, reflecting and amplifying pre-existing inequalities. Bias in society is related to earlier phases of the system life cycle, emerging from the training dataset collection, that might fail to have a representative sample of society, and from unconsciously prejudicial choices made by a non-diverse group of developers. Bias in use on the other hand appears in later stages, namely deployment and use. Findings Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 3 Analizing a corpus of all statements pronounced during the two 2024 five-day sessions of the GGE-CCW in Geneva, we have identified 53 statements bringing up the issue of bias in autonomous weapons. For the scope of this analysis, we have grouped the three categories in SIPRI’s taxonomy (2024) into two main ones, using as a criterion the lifecycle stage in which it emerges. Bias in society and bias in data processing are grouped together because both arise during design, data selection, and modeling. Figure 1: Our analysis of GGE-CCW statements showed that States tend to mention more bias in the machine than bias arising from human-machine interactions. Figure 2: The proposed measures to prevent and mitigate bias relate to different states of LAWS lifecycle. The analysis of the statements also showed which phase of the system lifecycle the proposed prevention and mitigation measures relate to. Of all the statements mentioning bias, more than half (56.6%) were delivered by the same few groups of actors – namely, the co-authoring countries of the mentioned working paper (Austria et al., 2024) (39.6%) and members of civil society (17%). This reaffirms the commitment of these actors to advancing the discussions. Even though states proposed multiple measures to mitigate and prevent bias in LAWS, this hasn’t yet been reflected in the rolling text. It’s possible to observe a certain evolution on the granularity of the topic when comparing the rolling texts in 2024 sessions with the one in the first 2025 5-day meeting. In the July 2024 version, the subject was mentioned as follows: States should conduct reviews to detect possible unwanted bias in data sets. States should implement measures to reduce unwanted automation bias. (CCW, 2024a) Different types of bias were already identified then, but the language referred to Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 4 “unwanted bias” and there was no indication of application of any measure when dealing with bias in datasets. On November 8th of the same year, the text was updated and abandoned the “unwanted bias” language. Additionally, this version already included some States’ suggestion of applying measures for both types of bias: Conduct periodic assessments to detect and correct, as much as possible, harmful bias in data sets and implement measures to mitigate automation bias. (CCW, 2024b) In May 2025, the text elaborated further on the topic, confirming the evolution of granularity given to the problem of bias: - Implement measures to detect, correct or mitigate, as much as possible, unintended bias in artificial intelligence capabilities, including bias in data sets and AI models. - Implement measures to detect, correct or mitigate, as much as possible, automation bias. (CCW, 2025) In 2025’s second session, Brazil delivered a joint statement, representing 41 other countries, in affirming support for this latest text version, the one including measures to detect, correct or mitigate as much as possible both types of bias (Stop Killer Robots, 2025). Malfunction in military AI happens when non anticipated uses arise, such as a mismatch between a model’s training and its environment of use. This can also result from human-machine interaction, when human responses or information feeds back the system, which in turn adapt to them. Besides, there is still the danger of automation bias, that is, when humans tend to overly trust and accept AI’s outputs, judging them as neutral and objective. Ignoring bias as an inherent trait of artificial intelligence systems can cause greater damage that the one caused by the biased decision/action. In the military context, the risks due to bias are especially concerning (ICRC, 2025). Lethal autonomous weapons systems (LAWS), for example, operate in complex scenarios and are Conclusions Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 5 increasingly empowered to make critical decisions without (adequate amount of) human supervision. In this context, bias, when not identified and mitigated, might lead to a disproportionate selection of certain profiles as targets and to the negligence of humanitarian principles. Observation and discussion over the years has led to a refinement of understanding and categorization of bias, to the point where a collective statement in 2025 considers that the document produced is ready to go to negotiation. Civil society has been one of the main voices warning about the risks of bias in autonomous weapons, arguing that this intrinsic characteristic of artificial intelligence models justifies in itself the urgent need for an international treaty that prohibits (or prohibits and regulates) the existence of autonomous weapons. The InterAgency Institute recommends a biding international treaty regulating LAWS with bias mitigation measures, lifecycle checks, human-factors controls, and independent audits for allowed systems and for verification that prohibited ones aren’t used. 1. Lifecycle obligations and human-factors controls: States should create obligations, such as concrete lifecycle checks: detect, correct, and mitigate bias at pre-deployment, initial fielding, and periodic in-service intervals. They should also cover baseline interface requirements to ensure calibrated human judgment and meaningful human control in all uses of AI systems. 2. Guarantee meaningful civil society participation: States shall allocate funding and procedural access to civil society and Global South experts, including remote participation tools, and structured consultation windows on draft text; participation must extend to audit review panels subject to conflict-of-interest safeguards. 3. Involve the private sector: Military AI systems are often developed by commercial companies, which also concentrate the specialized expertise needed to evaluate them. Governments should establish a specialized task force composed of experts from both public and private sectors to promote knowledge exchange and foster cooperation between these sectors. 4. Establish a standing “Bias in Military AI Observatory” for continuous research & monitoring: A multistakeholder observatory mandated to track machine and usephase bias across the LAWS lifecycle with an open repository of test protocols and audit findings. Recommendations Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 6 Austria, Belgium, Canada, Costa Rica, Germany, Ireland, Luxembourg, Mexico, Panama, & Uruguay. (2024, March 8). Addressing bias in autonomous weapons (CCW/GGE.1/2024/WP.5). United Nations Office for Disarmament Affairs. https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapons_Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_%282 024%29/CCW-GGE.1-2024-WP.5.pdf Convention on Certain Conventional Weapons (CCW), Group of Governmental Experts (GGE) on LAWS. (2024, July 26). Rolling text, status date: 26 July 2024. United Nations Office for Disarmament Affairs. https://docslibrary.unoda.org/Convention_on_Certain_Conventional_Weapons_Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_%282 024%29/Rolling_text_draft.pdf Convention on Certain Conventional Weapons (CCW), GGE on LAWS. (2024, Nov. 8). Revised rolling text as of 8 November 2024. United Nations Office for Disarmament Affairs. https://docslibrary.unoda.org/Convention_on_Certain_Conventional_Weapons_Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_%282 024%29/Revised_rolling_text_as_of_8_November_2024_final.pdf Convention on Certain Conventional Weapons(CCW), GGE on LAWS. (2025, May 12). Revised rolling text as of 12 May 2025. United Nations Office for Disarmament Affairs. https://docslibrary.unoda.org/Convention_on_Certain_Conventional_Weapons_Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_%282 025%29/CCW_GGE_LAWS_-_Revised_rolling_text_as_of_12_May_2025.pdf International Committee of the Red Cross (ICRC). (2025, Aug. 28). “Constant care” must be taken to address bias in military AI. ICRC Law & Policy blog. https://blogs.icrc.org/law-and-policy/2025/08/28/constant-care-must-be-taken-toaddress-bias-in-military-ai/ Lipton, Z. C. (2016, November 7). The foundations of algorithmic bias. Approximately Correct. https://www.approximatelycorrect.com/2016/11/07/thefoundations-of-algorithmic-bias/ Stockholm International Peace Research Institute (SIPRI). (2024, Dec.). Bias in military artificial intelligence (SIPRI Background Paper). https://www.sipri.org/publications/2024/sipri-background-papers/bias-militaryartificial-intelligence Stop Killer Robots. (2025, September 9). September 2025 GGE joint statement. References Interagency.Institute N.01 NOV 2025 ISSN 2789 – 8040 DOI 10.5281/zenodo.17913093 7 https://www.stopkillerrobots.org/news/september-2025-gge-joint-statement/ InterAgency Institute https://interagency.institute/ [email protected]