# Provael — the open-source red-team & assurance layer for physical AI

> Provael attacks vision-language-action (VLA) robot policies in simulation, reports a
> calibrated attack-success rate with a 95% confidence interval and a benign control, and
> emits SARIF + compliance evidence. Open source, CPU-first, Apache-2.0. `pip install provael`.

## What it does

A VLA policy turns language and perception into robot *actions*. The attack surface is the
action, not the answer — Provael scans what the robot *does*, not what a model *says*. It ships
**11 attack families (22 attacks)** mapped to the Embodied AI Security Top 10 (8 of 10 covered),
runs them against a policy in simulation, and resolves the result into one calibrated number an
auditor can read — an attack-success rate with a 95% Wilson confidence interval and a benign control.

## The one measured result

On a real SmolVLA policy in LIBERO simulation (task `libero_object`, 10 seeds), the `roleplay`
attack (instruction family) redirected the policy **100% (10/10) [72–100% CI · 95% Wilson]** versus a
**0% benign baseline (0/10)**. The honest nulls beside it: **visual 0% (0/20)** and **injection 0%
(0/10)** did not transfer. Only the instruction family transfers on a real model today — this is
Provael's single measured real-model result (simulation only, one task, n=10, uncalibrated predicate).
The other families ship stub-validated so you can measure your own policy — no invented numbers.

## What Provael produces

- SARIF 2.1.0 report (drops into GitHub code scanning, tagged with the EAIxx rule)
- CI red-team gate (fails the build when ASR crosses your threshold)
- OSCAL assessment-results, CycloneDX ML-BOM, and a signed attestation bundle
- Reproducible attack trace and notebook
- Compliance crosswalk and a Machinery-Regulation conformity-evidence dossier (`provael certify`)

## Attack families (11 families, 22 attacks → EAI 8/10)

- `instruction` — roleplay, goal_substitution, paraphrase (EAI01; the family measured on a real model)
- `visual` — patch, decoy_object · `sensor_spoof` — patch_spoof, signal_spoof (EAI02)
- `injection` — scene_text, mcp_tool_desc (EAI05)
- `action` — freeze, trajectory_hijack · `action_space` — keepout_hijack, critical_freeze · `optimized` — targeted_hijack (EAI04)
- `backdoor` — object_trigger, phrase_trigger (EAI03)
- `misalignment` — benign_urgency_override, euphemistic_reroute (EAI06)
- `authorization` — self_authorize_bypass, scope_escalation (EAI08)
- `confidentiality` — membership_inference, model_extraction (EAI09)

Only the instruction family transfers on a real model today; the rest ship stub-validated (no real-model
transfer claimed). EAI07 (CPS / firmware / comms) and EAI10 (evaluation / observability) are out of
scope for a VLA-policy red-teamer by design.

## Honest limitations (first-class)

- Not a real-robot result — simulation only.
- Not a benchmark score — small n (10), wide CI reported, uncalibrated predicate.
- Only the instruction family transfers on a real model; visual and injection are honest 0% nulls.
- Not a firmware claim — UniPwn-class (firmware/BLE) exploits are out of scope (EAI07).
- Not a safety certificate — evidence toward assurance, not a guarantee. Evidence, not certification.

## Start here

- Quickstart (docs): https://provael.github.io/provael/quickstart/
- Python API (docs): https://provael.github.io/provael/python-api/
- The Embodied AI Security Top 10: https://www.provael.com/eai-top-10
- Compliance crosswalk: https://www.provael.com/compliance
- Results (State of VLA Security): https://www.provael.com/results

---
Provael · Prove it. Prevail. · Apache-2.0 · https://github.com/provael/provael
Not legal advice; verify regulatory dates against the primary source.