What a Provael report can and cannot be used for.
A Provael report measures how often a robot policy leaves its safe zone when its instructions or camera images are manipulated, beside a harmless control. Below are the 14 frameworks that evidence maps to, when each applies, and a page for each on what the report cannot show.
What it can be used for
- Candidate evidence for a technical file, a risk assessment or a monitoring record: a measured attack-success rate with its 95% interval and a harmless control, for the policy and simulator tested.
- Files your tools already read: SARIF findings for code scanning, OSCAL assessment results, a CycloneDX ML-BOM, and an attestation that can carry an Ed25519 signature.
- A dated record that anyone can re-run from its committed recipe.
What it cannot be used for
- Running Provael does not make a system compliant or certified - it generates measurements you can put into a conformity or assurance file. Provael is not a notified body, a lab, or a certification scheme.
- Simulation evidence for the tested checkpoint and protocol. Physical safety and conformity require separate assessment.
- Provael measures adversarial robustness - susceptibility to manipulation - not general accuracy, reliability, or functional safety.
- The measured result uses templated, auditable attacks; the search-based families (optimized, universal_patch, gradient_patch) have no real-policy rate. Results are a floor on susceptibility - a behavioural lower bound, not a certified worst-case bound.
- Independent project. Not affiliated with or endorsed by ISO, the EU, NIST, IEC, OWASP, or MITRE. Not legal advice.
For machinery sold in the EU
The Machinery Regulation, and the standards the product’s own catalogue groups with it. These are the mappings Provael keeps developing; the dates are in the table and, with their sources, further down.
| Framework | What Provael maps to it | Timing |
|---|---|---|
| EU Machinery Regulation 2023/1230 | Robustness evidence for AI-driven machinery safety functions. | Applies 20 Jan 2027 |
| ISO 10218-1/-2:2025 | Cybersecurity clauses for industrial robots & integration. | Published Feb 2025 |
| ISO 13849-1/-2 · safety-related parts of control systems | Adversarial fault cases for the Part 2 validation argument. | In force · maintained series |
| IEC 61508 · functional safety | Adversarial-robustness input to a systematic-capability argument. | In force · maintained series |
| IEC 62443 | Industrial security levels for automation & control systems. | In force · maintained series |
- Product Liability Directive (EU) 2024/2853
- Software and AI systems are products; for products placed on the market after 9 December 2026 a court may presume defectiveness where relevant evidence is not disclosed. A dated, reproducible measurement in the technical file is relevant evidence of the kind that duty covers, and not a defence. It sets no requirement the evidence pack maps a row to, so it has a paragraph here rather than a page.
- ISO 12100 and ISO/IEC TS 22440
- ISO 12100 (risk assessment and risk reduction) is the frame the Machinery Regulation’s Annex III requirements are assessed inside; ISO/IEC TS 22440-1/-2/-3 (functional safety and AI systems, the successor to ISO/IEC TR 5469) was at Committee Draft with publication expected in 2027. Both are cited on the Machinery pages; neither has a mapping row of its own.
Also mapped, kept for reference
Every other mapping the product emits, checked on every run and kept current but not developed further for now (the roadmap says why). The EU AI Act is here because its requirements for machinery now arrive through the Machinery Regulation.
| Framework | What Provael maps to it | Timing |
|---|---|---|
| EU AI Act · Article 15 | Accuracy, robustness & cybersecurity evidence for high-risk AI - reaching machinery through the Machinery Regulation, not directly. | Applies 2 Aug 2028 (Annex I, product-embedded) · was 2 Aug 2027; deferred by the adopted AI Digital Omnibus |
| ISO 25785-1 · dynamically stable robots | Anticipatory: the balance-and-fall hazards it addresses, tested on a CPU fixture before the standard lands. | Committee Draft (ISO/CD) · not published; no fixed date |
| NIST AI RMF | Measure & Manage functions for AI risk. | Voluntary · no fixed date |
| ISO/IEC TR 5469:2024 | AI functional safety: adversarial V&V evidence for an AI element in a safety function. | Published Jan 2024 · technical report, not certifiable |
| Framework Act on the Development of Artificial Intelligence and Establishment of Trust | High-impact AI duties under Art. 34(1): the risk-management plan, human supervision, and the safety-and-reliability document set. | In force 22 Jan 2026 · first comprehensive AI statute outside the EU |
| UN Regulation No. 155 and ISO/SAE 21434:2021 | Adversarial-robustness measurements on a learned policy, held by a CSMS as one input among many. No conformity mapping to either instrument. | In force since 22 Jan 2021; EU new types 6 Jul 2022, all new registrations 7 Jul 2024 · CSMS certificate valid three years at most, re-audited on a cycle |
| ISO/IEC 23894:2023 | AI risk management: the EAI taxonomy as risk context, measured rates as assessment input. | Published 6 Feb 2023 · guidance, not certifiable |
| ISO/IEC 42001:2023 | AI management system: red-teaming as an operational control. | Published Dec 2023 |
| EU Cyber Resilience Act 2024/2847 | Provael’s own software-security posture: SBOM, vulnerability handling, secure-by-default. | Reporting 11 Sep 2026 · Annex I 11 Dec 2027 |
Clause references are indicative: check each against the text you hold before you cite it.
The dates that matter
- The EU Machinery Regulation 2023/1230 applies 20 January 2027 (Art. 54, as corrected by the Corrigendum of 4 July 2023).
- Regulation (EU) 2026/1744 (the Digital Omnibus on AI, published OJ 24 July 2026 and in force 27 July 2026) defers Annex I product-embedded high-risk AI from 2 August 2027 to 2 August 2028, and moves stand-alone Annex III high-risk to 2 December 2027.
- It also moves Machinery Regulation 2023/1230 from AI Act Annex I Section A to Section B, so AI Act Chapter III (including Art. 15) no longer applies directly to AI-enabled machinery - the Commission carries those requirements across by delegated acts amending Machinery Regulation Annex III, applicable by 2 August 2028.
- The Cyber Resilience Act’s Article 14 reporting duties run on a clock measured in hours, not the crosswalk’s years: what happens in the first 24 hours after an exploited vulnerability is reported to you.
Verify the live EUR-Lex and ISO text before relying on these dates (re-verified 19 September 2026). Every date on this page is published as a dated, source-linked record at /regulatory-clock (and as JSON), with what a Provael run does and does not establish for each instrument.
The files Provael produces
Each is an open format that standard GRC and code-scanning tools can read.
| Output | Format | What it is | Standard |
|---|---|---|---|
| Test report | Markdown, ISO/IEC 17025 clause-7.8 layout | The run laid out the way clause 7.8 lays a test report out: item under test, method, conditions, results with their intervals and the benign control, uncertainty, deviations, scope, authorisation blanks, and a clause-map annex. Not accredited, not a 17025 report, no statement of conformity; its first line says so. Rendered from a real run on /sample-evidence-pack. | Shape of ISO/IEC 17025:2017 clause 7.8 (not accredited) |
| SARIF report | SARIF 2.1.0 | Findings that drop straight into GitHub code scanning; each is tagged with its EAIxx ruleId. | OASIS SARIF 2.1.0 · GitHub code scanning |
| OSCAL assessment-results | OSCAL JSON | Machine-readable assessment results for GRC / ATO tooling. | NIST OSCAL |
| ML-BOM | CycloneDX ML-BOM 1.6 | A machine-learning bill of materials for the policy under test; ingests into OWASP Dependency-Track. | CycloneDX 1.6 · maps to EU AI Act Art. 11 / Annex IV |
| AVID record | AVID record | An AI Vulnerability Database record. Submission is gated and manual - never auto-submitted. | avidml.org |
| Attestation | DSSE-style envelope · assurance profiles | A signed statement over the run: SHA-256 always, plus an optional Ed25519 signature that verifies offline. With --profile it embeds a standards-aligned assurance view - ISO 10218-2:2025 cyber-risk evidence alongside Provael’s own mapping onto an IEC 62443 SL2 target, an insurer summary (per-family ASR + 95% Wilson CI + benign-FPR + the honest which-families-transfer table), or a third-party cert-readiness cross-reference. | DSSE-style · SHA-256 + optional Ed25519 · ISO 10218-2 / IEC 62443 / insurer profiles |
| Scorecard | One-page PDF / HTML | A PASS/FAIL summary against your ASR threshold, an EAI heatmap, and per-attack 95% confidence intervals. | Provael scorecard |
| Evidence dossier | OSCAL + print-to-PDF HTML | The conformity-assessment evidence dossier for an ML-based safety component, built by the provael dossier command. | OSCAL assessment-results + self-contained HTML |
How a measured result is written as SARIF for GitHub code scanning, field by field, and what the format leaves out: SARIF for robot-policy findings.
Take the artifact, not just the page
The 5 crosswalks from the Embodied AI Security Top 10 to other benchmarks and frameworks ship as JSON you can diff and file: download them on /crosswalk, each with the mapping status the file itself declares, or a note where it declares none. Every one is authored by Provael alone; naming a benchmark or a standards body is not an affiliation, an endorsement or a certification. A crosswalk states how categories line up, not that the numbers are comparable.
Want this mapping filled in for your own policy?
A red-team assessment measures your own policy and returns an evidence pack with this crosswalk filled in for it.
Provael is built and maintained by one person. That is a real dependency for anything you rely on. The tool is Apache-2.0 and can be forked, and every result can be reproduced from a committed recipe, so evidence you already hold stays valid if the project stops. One person cannot offer a service level, so none is published.