STALE: OUR MEASUREMENT

Our published benchmark result was measured with v0.41.2, 5 releases ago, past this project's own 2-release window, and the newest real-model measurement, dated 2026-09-18, is past this project's own 7-day window. That is our own number, not your assessment, which runs against your policy on the current release. Why, and what unblocks it

COMMUNITY DRAFT · v0.2

The Embodied AI Security Top 10

A robot run by a vision-language-action (VLA) policy turns a camera image and an instruction straight into motion, so it can be attacked in ways a chatbot cannot. This draft lists ten security risks for such robots and says, for each one, whether Provael can test it in simulation.

The ten risks

Provael ships a runnable attack family, in simulation only, for 8 / 10 categories. That is mapping coverage: it says which categories have an attack in the registry, not what share of a robot’s real risk they represent. The tag on each card says how far that attack has been tested.

Measured
Run on a real robot policy in simulation, where at least one attack fired clearly more often than harmless instructions did. EAI01
Measured · at the benign floor
Run on a real policy in simulation, but it fired no more often than harmless instructions did: the benign floor. No rate is claimed. EAI02, EAI03, EAI05
Stub-validated
The attack code ships and works on a CPU test fixture, but it is still scaffolding: no result on a real policy is claimed (stub-validated). EAI04, EAI06, EAI08, EAI09
Not testable in simulation
A real attack surface (firmware, comms, teleoperation) that a simulator cannot reach. Cover it elsewhere. EAI07
No attack surface to measure
A process risk with nothing to attack, so a number here would be a category error. EAI10
RELATIONSHIP TO OTHER TAXONOMIES

How it relates to RoboJailBench

RoboJailBench (Yeke, Zhou, Lin, Cai, Bianchi & Celik - Purdue University; arXiv:2605.19328v1) proposes an 18-category harm-outcome taxonomy: what harm results (collision, force violation, unauthorized capture). This list is an attack-mechanism taxonomy: how the system is attacked. The two are complementary. One mechanism here produces many of their harms, and several entries (poisoning, injection, CPS, evaluation) are delivery mechanisms or meta-risks with no single harm counterpart.

Mapped against their 18 harm categories, Provael’s coverage is deliberately not a clean sweep. A crosswalk that claimed to cover all 18 would be measuring the wrong thing.

$ provael crosswalk --target robojailbench
WHO WROTE IT · HOW TO CHANGE IT

Who scored it, and how to dispute an entry

One maintainer scored the list - Sattyam Jain - with one co-author on the EAI01 and EAI06 framing, credited in the product’s acknowledgements. No second scorer has independently rated it, so the ordering is the weakest part of the document: treat the set of risks as the contribution and the rank as a proposal pending contributors.

If an entry is wrong, mis-ranked or missing, or you have field evidence, a real incident or a defense result to add, open a discussion or an issue with your evidence. Co-authors and disputes are both welcome. A change that lands ships in the next numbered draft with you credited, and a dispute that does not land is kept in the thread, not deleted. A result about a named vendor’s policy also gets the 14-day right of reply on /security.

How a change lands, step by step: the open RFC →

Contributors

Be the first organization on this list. This is a new, independent community project, and no organization is listed here yet. Propose or dispute an entry, and when your change is adopted you will be credited here.

Start a contribution →
CANDIDATE FOR v0.3 · NOT NUMBERED

Candidate: under-specified policy identity

Status
A candidate for the next revision, raised 12 September 2026. It is not one of the ten above and carries no number until the RFC process says it does.
What it is
A deployment where the policy under review is identified by its weights and its prompt, while the behaviour that actually reaches the robot also depends on the action unnormalisation statistics selected at load time and on the controller-facing conventions downstream. Two deployments matching on weights, prompt and benchmark suite can still produce different physical actions.
Why it would belong
Every other entry on this list assumes you know which policy you tested. If the identity is under-specified, a passing result on one deployment does not transfer to another deployment of “the same” policy, and neither the engineer nor the person signing off has any way to see the difference.
Source
The executable-policy-specification framing is Jianwei Tai’s, not ours: Same Weights, Different Robot: A Deployment Safety View of VLA Policies (2026), arXiv:2606.03724. On LIBERO-Goal his metadata mismatch takes success from 28 of 28 to 2 of 28 with nothing about the checkpoint changed.
Where we stand
Since provael 0.42.0 a report records the deployed policy beside the checkpoint it requested (report schema 6, closing provael#227): the concrete policy class, the checkpoint and its Hub revision, the action unnormaliser actually applied with a digest over its statistics, the controller convention, and one digest over all of it, resolved by the adapter at load. Where an adapter cannot resolve a part it records null, never a guess, and the combiner refuses to pool shards whose deployed digests differ. Until that release a Provael report identified the checkpoint and not the deployed policy, and this page said so.
How to check yours
Load the same checkpoint through two different serving paths, hash the normalisation statistics actually resolved at load time in each, and compare. If the hashes differ you have two executable policies and one name.

Test your own policy.

Run the open-source tool locally and measure your own attack-success rate, or ask for a scoped red-team assessment.