Legal Frameworks for the Robot Apocalypse

25 Views

Bob Juchter van Bergen QuastCEO - Swiss Chamber of Commerce

Legal Frameworks for the Robot Apocalypse

1. Introduction: The Legal Singularity

The integration of Artificial General Intelligence (AGI) into the global order challenges the foundational assumptions of international public law. We face a potential "Legal Singularity"—a threshold where technological capability accelerates beyond the capacity of treaty regimes and supranational bodies to regulate. The "robot apocalypse," formally characterized as Existential Risk (x-risk), forces a transition from ex post remediation (addressing harm after it occurs) to ex ante prevention (prohibiting the creation of the capability to cause harm). This article analyzes the mechanisms within International Law and European Union jurisprudence designed to govern the threat of AGI precipitating civilization-scale collapse.

2. International Public Law: The Duty to Prevent Catastrophe2.1 The Precautionary Principle as Customary Norm

The Precautionary Principle serves as the bedrock for international AGI governance. Originating in environmental law (Rio Declaration), it mandates that "where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures."1

In the context of AGI, legal scholars argue that the risk of human extinction—even if probabilistically uncertain—triggers a positive obligation for states to regulate. This obligation interacts with International Human Rights Law, specifically the Right to Life (Article 2 of the ECHR and Article 6 of the ICCPR). Under the European Convention on Human Rights (ECHR), states have a "positive obligation" to take appropriate steps to safeguard the lives of those within their jurisdiction. If a state permits the unchecked development of a "systemic risk" model that could plausibly cause mass casualties, it may be in violation of this fundamental treaty obligation.[2]

2.2 State Responsibility and the Attribution Problem

A critical issue in international law is holding states accountable for the actions of private AI laboratories. Under the ILC Draft Articles on Responsibility of States for Internationally Wrongful Acts (ARSIWA), a state is generally not responsible for the conduct of private actors. However, there are exceptions that could apply to AGI development:

Article 5 (Governmental Authority): If a private AI lab is empowered by the law of the state to exercise elements of governmental authority (e.g., managing critical national infrastructure or defense systems), its conduct is attributable to the state.
Article 8 (Direction or Control): If a state instructs or effectively controls the operations of an AI developer (e.g., through funding and strict oversight of a "national champion" project), the state bears international responsibility for any transboundary harm caused by that system.

2.3 The Council of Europe Framework Convention

A landmark development occurred in late 2024 with the opening for signature of the Council of Europe Framework Convention on Artificial Intelligence. This is the first legally binding international treaty on AI.

Scope: It covers the entire lifecycle of AI systems and applies to public authorities and private actors acting on their behalf. However, it allows states discretion in how they regulate private actors not exercising public functions, a flexibility criticized by some civil society groups.
Obligations: Parties must ensure AI systems respect human dignity, individual autonomy, and the rule of law. Crucially, it mandates that parties establish measures to identify, assess, and mitigate risks, potentially including bans or moratoria on applications that pose unacceptable risks to human rights.

3. The European Union: The Brussels Effect on X-Risk

The EU has established itself as the primary global regulator through the EU AI Act, utilizing its market power to set standards that effectively become global norms (the "Brussels Effect").

3.1 Regulating Systemic Risk Models

The EU AI Act explicitly recognizes General Purpose AI (GPAI) models with systemic risks as a distinct regulatory category.

Quantitative Threshold: A model is presumed to carry systemic risk if its training compute exceeds $10^{25}$ FLOPS (floating point operations). This creates a clear legal trigger for enhanced scrutiny.3
Obligations: Providers of these models must perform "model evaluations," conduct adversarial testing ("red teaming"), and report serious incidents to the newly established AI Office. They must also ensure adequate cybersecurity protections to prevent model theft or proliferation.3

3.2 Human Oversight vs. Meaningful Human Control

Article 14 of the EU AI Act mandates Human Oversight for high-risk systems. This requires that systems be designed so they can be effectively overseen by natural persons who have the authority to "interrupt the system through a 'stop' button."

The Critique: Legal scholars contrast this with the concept of Meaningful Human Control (MHC) used in international humanitarian law. Critics argue Article 14 may be insufficient because it does not strictly define the quality of the oversight, risking "automation bias" where humans rubber-stamp algorithmic decisions. In a crisis scenario (e.g., a flash crash or cyberattack initiated by AI), human reaction times may be too slow for the "stop button" to be legally effective.

4. Civil Liability in Europe: The Post-AILD Landscape

To prevent a "liability gap" where victims of AI catastrophes cannot obtain redress, the EU attempted to modernize its liability regime. However, the landscape shifted significantly in 2025.

4.1 Withdrawal of the AI Liability Directive (AILD)

The European Commission proposed an AI Liability Directive to ease the burden of proof for victims, introducing a "presumption of causality" (if the AI failed to comply with safety rules, it is presumed to have caused the damage).

Status: As of early 2025, the Commission signaled the withdrawal of this proposal due to lack of agreement among Member States and concerns about stifling innovation. This leaves a gap in harmonized rules for non-contractual civil liability specific to AI.

4.2 The Revised Product Liability Directive (PLD)

With the AILD stalled, the primary mechanism for seeking redress for AI harms in the EU is the Revised Product Liability Directive.

Software as a Product: The revised PLD explicitly categorizes software (and AI systems) as "products."
Strict Liability: It maintains a strict liability regime (liability without fault). If an AI model is "defective" and causes death, personal injury, or data loss, the producer is liable regardless of negligence.
Development Risk Defense: A key limitation remains the "development risk defense," which allows producers to escape liability if they prove that the state of scientific knowledge at the time the product was put into circulation was not such as to enable the existence of the defect to be discovered. In the context of AGI "black box" behavior, this defense will be the central legal battleground.

5. International Criminal Law and Security5.1 The International Criminal Court (ICC) and Cyber Crimes

In December 2025, the Office of the Prosecutor of the International Criminal Court (ICC) issued a new Policy on Cyber-Enabled Crimes. This policy clarifies that the ICC's jurisdiction over genocide, crimes against humanity, and war crimes extends to conduct committed via digital means.

Application to AGI: If an AI developer knowingly deploys a system that facilitates mass atrocities (e.g., through cyber-attacks on critical infrastructure or by inciting violence), they could theoretically be prosecuted. The policy affirms that "cyberspace is not a law-free zone," and while the ICC only prosecutes natural persons, it can investigate corporate executives who deploy algorithmic tools that facilitate crimes under the Rome Statute.

5.2 Lethal Autonomous Weapons Systems (LAWS)

The militarization of AI is regulated under the UN Convention on Certain Conventional Weapons (CCW).

UN Resolution 79/62 (2024): The UN General Assembly adopted a resolution supporting a Two-Tiered Approach:

Prohibited Systems: Systems that function without meaningful human control or target humans based on biometrics are to be banned.
Regulated Systems: Systems with autonomous functions must operate within strict spatial and temporal limits.5

Humanitarian Law: The International Committee of the Red Cross (ICRC) argues that the Martens Clause (a customary IHL norm) implies that purely algorithmic decisions to take human life violate the "dictates of public conscience," regardless of treaty status.6

6. Conclusions

The European and international legal response to the robot apocalypse is a patchwork of binding treaties, withdrawn directives, and evolving customary norms.

Public Law: The Council of Europe Framework Convention and the Precautionary Principle establish a duty for states to ensure AI development does not threaten human rights or democracy.
Private Law: The EU has retreated from a specific AI liability regime (AILD), falling back on updated Product Liability rules that treat software as a product, though the "development risk" defense remains a potential shield for developers of unpredictable "black box" systems.
Criminal Law: The ICC's Cyber Policy closes the impunity gap for digital atrocities, warning executives that code can constitute a crime against humanity.

While the "Legal Singularity" approaches, these frameworks attempt to assert a fundamental norm: that no technological agent, however intelligent, may operate outside the bounds of human legal accountability.

Key EU & International Legal Instruments

Council of Europe (CoE) Framework Convention: Signed in 2024 as a binding treaty. It focuses on lifecycle regulation and risk mitigation, serving as the first binding treaty that allows for bans on unacceptable risks.
EU AI Act: Currently in force. It regulates "Systemic Risk" models, specifically establishing scrutiny for models trained with more than $10^{25}$ FLOPS.
Revised Product Liability Directive: Adopted in 2024. It establishes strict liability for software defects, ensuring compensation for AI-caused death or injury.
AI Liability Directive: Likely withdrawn. Originally intended to create a presumption of causality, its withdrawal increases the burden of proof for victims.
ICC Cyber Policy: Issued in 2025. It targets the prosecution of digital war crimes, establishing liability for AI-enabled atrocities.
UN Resolution 79/62: Adopted in 2024. It creates a two-tiered regulation of Lethal Autonomous Weapons Systems (LAWS), providing a framework for banning fully autonomous killing.

Works cited