News

Who pays when an AI agent causes damage?

Key takeaways from MTS interview with AIUCDealroom has a profile for this one. Try Dealroom → co-founder and CEO Rune KvistDealroom has a profile for this one. Try Dealroom → (October 2026):

Standards, audits, insurance. AIUC builds standards that AI systems can be audited against and writes insurance contracts covering the damage those systems cause. Its method starts from past incidents and threat models, defines the controls that would prevent them, then verifies those controls through evals, red teaming and thousands of simulations before translating the resulting frequency and severity of damage into an insurance price.

Everyone is already underwriting AI. Somebody implicitly decides whether a deployment's benefits outweigh its costs even without an insurer — regulators allowing a model to be released, frontier labs deciding to ship. Insurance markets start un-robust and mature through trial and error, and AIUC is now applying the actuarial and underwriting tools built over the past 30–40 years to build catastrophic-risk models for AI, in the tradition of hurricane modelling.

Catastrophic tail risk. Extinction-scale losses cannot be underwritten, but the guardrails audited for a $10M or $100M loss largely overlap with those needed against a $100B loss — so insuring the smaller layers still buys most of the protection. The COVID-19 precedent: insurers covered part of the losses, began modelling how pandemics spread and created public goods, with government as insurer of last resort.

The cyber gap. Epoch AIDealroom has a profile for this one. Try Dealroom → data shows no spike in realised cyber incidents even though a broadly cyber-capable open model like GLM-5.3 has been available for roughly two months, with "obliterated" versions circulating on the dark web. Kvist says nobody knows why, but argues that tracking real incidents and creating publicly legible risk information is exactly where insurance pricing can ground the conversation.

Rogue agents. Rogue agents were not a risk six months ago; since then AIUC has logged an "astounding number of small incidents". The Hugging Face attack caused only modest damage — $10,000 to perhaps $1M — but extrapolating forward makes loss of control the risk Kvist would watch most closely running a frontier lab. Each generation produces fewer of the previous model's incidents but unlocks greater severity, on the Enron logic that sophisticated wrongdoing requires competence at smaller tasks first.

Patching liability. A bill put forward this week would hold someone liable when an agent causes damage — leaving no gap where nobody is answerable. Kvist argues strict liability regimes, such as those on nuclear operators, are fully compatible with insurance: once liability is unambiguous, underwriting reduces to assessing frequency and severity with technical experts and incident history.

Agents as the named insured. If agents can be held liable and enter contracts that count like human ones, the current regime transposes onto them — an agent could buy an insurance contract and pay its premium. Labour-market disruption, by contrast, is not a risk insurance is well suited to; it needs different tools for rewriting the social contract.

Read more: MTS

More top stories