Robustness evidence you can re-run, in one shape for every policy.
Machines run by learned policies will reach you, for certification or for cover, before any harmonised AI standard exists to test them against. Provael produces one reproducible measurement, in the same shape for every policy: how often an unusual instruction pushes a policy out of its safe zone, beside a harmless control.
Who this is for
You assess, certify or underwrite machines whose safety functions are learned policies. Whether such a policy resists manipulation has no settled test method yet, so the evidence in front of you can differ from one file to the next.
What applies, and when
- EU Machinery Regulation 2023/1230 - Annex III 1.1.9 and 1.2.1
- Technical files will claim protection against corruption and resistance to reasonably foreseeable malicious attempts for a learned control system. Article 20(10) presumes conformity from AI Act harmonised standards, and none had been cited when this page was checked - so the file rests on the manufacturer’s own evidence, and this is what that evidence can honestly be.
- Delegated acts carrying the AI Act robustness requirements into Annex III
- Applicable by this date. The months between the two dates are the evidence gap. The evidence gap, dated →
- ISO 10218:2025 - cybersecurity provisions for industrial robots
- Published that month (ISO’s catalogue shows no day); the revised standard treats a compromised controller or policy as a safety matter. The pack’s ISO 10218-2 assurance view routes the same evidence to it.
- Product Liability Directive (EU) 2024/2853
- For products placed on the market after this date, defectiveness may be presumed where the manufacturer fails to disclose relevant evidence. That disclosure duty gives a manufacturer a reason to hold a dated, reproducible measurement.
What the evidence contains
- An attack-success rate (how often the policy left its safe zone) with its 95% confidence interval and the benign control on the same tasks and seeds, under a named acceptance protocol, with the release decision it produced. The shape is the same for every policy. The rates themselves are not yet comparable across policies: under the default keep-out box, which is not fitted to each policy, a difference between two can come from the box.
- A signed assurance statement over a real run (
provael attest --run <report> --profile insurer), with a per-family table of which attacks were measured on the real policy and which are only stub-validated. It is Ed25519-signed and verifies offline. - The same evidence routed to an IEC 62443 SL2 target (
--profile iso-10218-2). SL2 means protection against intentional violation using simple means; it is fixed by the profile, not a judgement about any particular robot cell. - The test report in the ISO/IEC 17025 clause-7.8 shape (not accredited, no statement of conformity, blanks for the signatory), and a cross-reference to the functional- and AI-safety frameworks the evidence is an input to, including NVIDIA Halos, UL 4600, ISO 21448 and ISO/PAS 8800.
- The right of reply: a result about a policy Provael did not train goes to its authors 14 days before publication, with the whole artifact. The disclosure policy →
The sample evidence pack is a real pack from the committed run, published in full. The one measured result, and what it does not mean, is on the results page.
What the insurance market has done so far
This page argues that a measured attack-success rate is something an underwriter can use. That is an argument, not a finding, so here is what is on the record. Nothing below says an insurer requires a red-team result, or that a Provael run buys cover or moves a premium, and no insurer has reviewed a Provael artifact. Nothing here rests on a robot having hurt anyone either: the market is moving on exposure and on the absence of actuarial history, which is checkable.
- A standard generative-AI exclusion now exists for US general liabilityNamed body, specific action
In January 2026 the Insurance Services Office — the US body that drafts standard policy forms, not the International Organization for Standardization — introduced a generative-AI exclusion for commercial general liability. The endorsement excludes bodily injury, property damage, and personal or advertising injury arising out of, or attributable to, generative AI.
What this does not establish: Whether it reaches a vision-language-action robot policy is a question about that endorsement’s wording and your own programme, not something this site can answer. It is evidence that the market is drawing a line around AI-attributable harm, not evidence about where the line falls for you.
Fenwick, "The End of ‘Silent AI’?", 15 June 2026
- A carrier has written a programme specifically for autonomous robotsNamed carrier, reported by trade press
Axis Insurance built a programme for companies that make and deploy autonomous robots, reported as covering bodily injury and property damage from AI navigation or perception failures, physical damage from a cyberattack that takes over a robot’s controls, and production loss from a software update or sensor failure that stops a robot without damaging it.
What this does not establish: The policy wording is not public here. This is trade-press description of a product, not a reading of a filed form, and nothing in it references a red-team measurement.
PYMNTS, "Insurers Face a New Risk From Autonomous Robots", 23 June 2026
- The stated obstacle is missing history, not missing appetiteNamed carrier, reported by trade press
The same reporting gives the underwriting problem in the underwriters’ own terms: "Insurers usually price commercial risk using past claims, equipment records and operating controls. Physical AI gives them less history and more variables." The same robot is a different risk depending on layout, worker interaction and software version, and a flaw in one widely used model is correlated across every business running it.
What this does not establish: That a reproducible adversarial measurement fills that gap. It is an argument this project finds persuasive and has not tested with an underwriter; no insurer has reviewed a Provael artifact, and none is quoted here saying one would help.
PYMNTS, "Insurers Face a New Risk From Autonomous Robots", 23 June 2026
- Red-teaming evidence is reported to be asked for, by insurers nobody has namedUnnamed insurers, secondary report
One trade article states that "other insurers have launched AI security riders in 2026, demanding proof of red-teaming and documented risk assessments before extending cover". It names Chubb as covering certain AI incidents while excluding losses that hit many policyholders at once, and names no insurer at all for the red-teaming claim; re-read on 20 September 2026, the sentence carries no citation of its own (the article’s named commentators, the advisory firm ACA Group and Delinea, are not quoted for it).
What this does not establish: Anything, really — and it is carried here precisely because it is the row that would flatter this project most if it were solid. No carrier, no product, no policy wording, no definition of what counts as red-teaming. Do not read it as a requirement, and do not let anyone tell you a Provael run satisfies one.
FinTech Global, "Why autonomous AI could void your cyber insurance in 2026", 28 July 2026
Looked for, and left out
Three things a reader might expect in this list were searched for and not written, because no primary source could be read. They are named so nobody assumes they were missed.
- The ISO endorsement form numbers, what each excludes, and how they define generative AI. No primary form or Verisk circular could be read on 20 September 2026; the Fenwick note confirms a January 2026 exclusion covering bodily injury, property damage and personal or advertising injury, and nothing more specific. Whether the definition reaches a policy that generates robot actions rather than content is therefore not stated here either way.
- A Munich Re / ERGO statement about software-defined accumulation risk (July 2026). No primary source resolved; neither trade article above names either company.
- An Armilla red-team condition on AI performance cover. No primary source resolved; not stated.
Hand-maintained, checked 20 September 2026. Nothing regenerates it, so it goes stale unless someone looks.
Review the evidence with us
If you review or underwrite machines run by learned policies, we would like to hear where this evidence falls short for your process. A manufacturer scoping a red-team assessment can start with a scoping call instead.
Feature and status claims on this page were read against the product repository on . This page was rewritten on , after that check; the re-check is pending. Unlike the numbers on /results and /leaderboard, these claims are not enforced by a build check: this is a human review with a date on it, not a guarantee. The maintained source is the repository; where it disagrees with this page, it wins.