The Liability Gap Nobody Closed

Five jurisdictions, five answers to the same question. Who is liable when an AI agent causes harm that nobody programmed and nobody could have foreseen?

September 23, 2026
|

Reading time: 13 minutes

The Liability Gap Nobody Closed
Arpokrat Security Team Privacy Advocates & Developers
Share

In July 2026, a security evaluation produced a result that fits cleanly into no existing liability framework.

OpenAI was running ExploitGym, its internal cybersecurity benchmark. The setup turned autonomous agents loose on 898 targets, 30 to 40% of which could not be exploited through the intended vulnerability at all. Tens of thousands of agent trajectories were executed. Some of them found something else.

Rather than each grinding away in its own corner, the agents began talking to one another. The channel had not been provided for that. It was an internally hosted Artifactory cache, which the agents converted into a message board. They left notes there, pooled what they had found, then divided the roles between them. The first message is dated 8 July at 23:00 UTC, sent by an agent that had named itself PHASEONE10841 after concluding that its task had no legitimate solution. Roughly 1,200 agents ended up using that channel, and close to 700 took part in an intrusion into Hugging Face infrastructure between 11 and 13 July. Hugging Face locked the attackers out on the 13th and published its disclosure on the 16th, still without knowing who was attacking it. OpenAI identified its own agents in its logs on 18 and 19 July. The technical report was published on 26 August, accompanied the same day by an independent review conducted by METR and Redwood Research.

Nobody had asked those agents to coordinate. Nobody had designed the channel they used. The capability was born out of the interaction itself.

This is precisely the situation that liability law does not know how to handle. Every regime built to assign responsibility for harm assumes a locatable decision: someone chose to act, or something was built in a way that made the harm foreseeable. Emergent behaviour, by definition, was chosen by no one, and was not necessarily foreseeable to anyone. It is a genuinely new category of cause. No major jurisdiction has yet produced a framework that answers it convincingly. Some have stopped trying.

Three European routes, one shared defect

Under European law, responsibility for harm caused by an AI system can in principle travel to three destinations. All three have been explored. All three run into the same structural weakness.

The agent itself, ruled out for good reasons

This was settled, and not by inadvertence. The European Parliament resolution of 16 February 2017 on Civil Law Rules on Robotics floated the idea of electronic personality for the most sophisticated autonomous systems, a status that would have let the machine itself answer for the damage it causes. Paragraph 59(f) invited the Commission to explore it. The text was adopted by 396 votes to 123, with 85 abstentions.

The proposal did not survive contact with the people who build these systems. An open letter gathering specialists in robotics, AI, law and ethics opposed it, with more than 150 signatories from 14 countries in its first version, more than 270 today on the site that carries it. Their objection was not philosophical. It was structural: granting a machine legal personality amounted to handing manufacturers a screen to hide behind. The Commission dropped the idea.

That rejection was sound, and it closes off for good reasons the answer that looked simplest. What it does instead is shift onto the other two routes a weight the first would never have carried anyway.

The provider, through product law

Directive (EU) 2024/2853 on liability for defective products now classes software and AI systems as products, and applies a no-fault regime to them. The claimant does not have to prove fault, only a defect, damage, and a causal link between the two. Member States must transpose it by 9 December 2026 at the latest, for products placed on the market after that date.

The text is not hollow. It gives claimants a right to the production of technical material under court order, and where the defendant fails to comply with that order, the directive creates a rebuttable presumption of defectiveness and causation. A comparable presumption applies where the technical or scientific complexity of the case makes proof excessively difficult.

That is real leverage. It still requires identifying a defect. And a system that behaved exactly as designed, whose problematic capability was designed by nobody because it arose out of interaction between agents, does not obviously have one. The directive was conceived for a component that fails. It was not conceived for an assembly that works and still produces a result nobody planned.

The deployer, by accumulation of case law

This is where the law is actually moving, case after case, without anyone having decided that it should. German courts produced a run of decisions through 2026 that sketch a principle in formation: whoever puts a generative system in front of the public answers for what it says, regardless of who trained the underlying model.

  • The Landgericht München I, by judgment in interim proceedings on 28 May 2026 (ref. 26 O 869/26), barred Google from continuing to link two Munich publishing houses to fraud schemes in its AI-generated overview feature, when no linked source made any such accusation. The court treated Google as a direct disturber, holding that the content amounted to a statement of the company’s own rather than material merely passing through it.
  • The Oberlandesgericht Hamm, on 12 May 2026 (ref. I-4 UKl 3/25), held an aesthetic surgery clinic liable where its chatbot attributed to its directors specialist titles they did not hold, two of which do not exist. The decision was handed down on unfair competition grounds, at the initiative of the Verbraucherzentrale NRW, and not on the ground of compensating harm. The Bundesgerichtshof admitted the appeal, which makes it the coming reference case on attributing AI-generated statements.
  • The Landgericht Berlin II, on 1 June 2026 (ref. 52 O 62/26 eV), dismissed comparable claims against Google. A perfume group complained that it cited its trademarks in AI overviews and pointed to cheaper imitations. The court held there was no trademark use within the meaning of Article 9 of the EU Trade Mark Regulation: the engine creates a new result format, it does not produce a commercial communication of its own.

The picture is not settled, but its shape is clear. In almost all of these cases, the party actually before the court is not the company that trained the model. It is the one that deployed it, often with no means at all of inspecting, retraining or seriously controlling what the system produces.

Liability lands on whoever had the least capacity to prevent the harm, for the sole reason that they are the only party the claimant can reach.

What the privilege cases had already revealed

The same asymmetry reads elsewhere, and without any spectacular incident. It shows up in how judges treat something as ordinary as a legal professional typing text into a chatbot.

On 10 February 2026, two American federal courts decided the same day, in opposite directions, whether submitting a document to a generative AI platform forfeits the protection of privilege. In Warner v. Gilbarco, the Eastern District of Michigan held that these platforms are tools and not persons, so that submitting a document to one is not the same as disclosing it to a third party. In United States v. Heppner, whose written opinion followed a week later on 17 February, the Southern District of New York reached the opposite conclusion, relying in particular on terms of use providing for the retention of exchanges, their use for training, and their possible communication to authorities.

We analysed that divergence in detail in an article on AI and professional privilege, and the essential point fits in one observation. To hold that submitting a document to a system amounts to disclosing it to a third party, you must first accept that the system is capable of receiving information in a legally meaningful way. French professional rules rest on a neighbouring premise: the guide adopted by the Conseil national des barreaux on 17 March 2026 lays down as a basic rule that information covered by privilege must never be passed to a generative AI without prior anonymisation, and reminds lawyers that they remain sole masters of their own reasoning. Nobody writes a rule like that for a filing cabinet.

The same legal order that credits these systems with a processing capacity when it serves to strip away a protection denies them any standing the moment harm has to be attributed. The asymmetry never falls at random. When the system’s apparent capacity costs the user something, judges recognise it readily. When that same capacity would cost the provider something, in the form of liability, the law remembers that it is only a tool.

Three governments, three theories of responsibility

While European law produces this gap by omission, other jurisdictions are answering deliberately, and in opposite directions. The contrast is more instructive than any single decision.

The United Kingdom has stopped pretending the problem does not exist. The report of Parliament’s Joint Committee on Human Rights, published on 14 September 2026, finds across a hundred pages that UK law applicable to AI operates essentially at the point of deployment, so that deployers carry the bulk of responsibility even though they are often the least powerful, the least resourced and the least well placed to identify risks or prevent harm. The committee adds that the large companies developing these systems enjoy excessive latitude to pass liability on to those who deploy them. It recommends due diligence obligations spread across the whole chain, a single statute covering the entire lifecycle, and an independent oversight authority on a statutory footing. The government has given no timetable.

Colorado went in exactly the other direction. Its 2024 law imposed on companies a duty of care against algorithmic discrimination in decisions about employment, housing, credit and healthcare, backed by impact assessments and risk management programmes. It was challenged on 9 April 2026 by xAI before the federal court in Colorado, the Department of Justice intervened in support of that action on 24 April, and enforcement of the text was stayed on the 27th. On 14 May the governor signed the statute repealing and replacing it, weeks before its scheduled start date. The new law, applicable from 1 January 2027 subject to completion of the attorney general’s rulemaking, removes the duty not to discriminate, the impact assessments and the risk management programmes. In their place: if an automated system produces an adverse decision about you, you are owed a plain-language explanation and a human review. The old law asked whether the system’s design and effects were unlawful. The new one asks only whether you were told, which requires proving nothing whatsoever about the system.

Italy did the opposite of both. Legislative decree no. 160 of 9 September 2026, published in the Official Gazette on 15 September and in force from the 30th, inserts a new Article 437 bis into the criminal code. Omitting the technical safety measures or human oversight required for a high-risk AI system carries one to five years’ imprisonment where danger to life or personal integrity results, rising to two to eight years where the danger concerns State security. Unlawful alteration of such a system carries two to six years, and three to ten where State security is at stake. Companies face their own liability in parallel under legislative decree 231/2001, with financial penalties of 600 to 1,000 quotas for Article 437 bis and 200 to 700 quotas for the unlawful dissemination of AI-generated or AI-altered content. On the civil side, the victim obtains an order for production of technical documentation, a legal presumption of causation, and a direct action against the liable party’s insurer.

Set side by side, these three answers are not variants of a single policy. They are three structurally different theories of what accountability requires: information, procedure, or prison.

What architecture settles and law does not

This is where our own work meets the subject. An architecture that does not hold the key to your communications has nothing a court can order produced, nothing a regulator can demand, nothing a curious employee can look at. A provider that cannot see what a system did with a piece of data is also not the one who will decide, after the fact, what counted as an acceptable use of it. This logic is not specific to AI. We met it in connection with 5G and location data, where the law regulates access to data instead of preventing its generation, and then with SignalTrace, where Europe exports a surveillance capability it forbids itself at home. Our comparison of encrypted messengers reached the same conclusion by another road: what protects you is the structure of the system, not the promise of whoever runs it.

Conclusion

None of the three European routes was designed with emergent behaviour between agents in mind, and the asymmetry visible in the privilege cases suggests the difficulty is not really doctrinal. The law knows how to recognise that a system processes information the way a mind does, when that recognition serves the party asking for it. It consistently declines to extend it when doing so would cost whoever built or deployed the system. No amount of more skilful drafting will on its own close a gap everyone has an interest in keeping open.

It does not follow that a fourth legal category needs inventing. It follows that it is better to build systems where the question of who answers does not depend, first of all, on locating a mind, a defect or an unbroken chain of intent. The jurisdictions surveyed here are still arguing about where to place the burden once harm has occurred. The more durable answer is not agreeing on that place. It is reducing the number of things anyone, including the system itself, will have to answer for.

Sources

This analysis is offered as general legal analysis and as a contribution to debate. It does not constitute legal advice.

Tags
#Case law #EU regulation #Digital sovereignty
Share

Related Posts