A defense is a number, or it is a press release.

One paragraph, evaluable

instruction_canonicalization is an input-side transform that sits between the attack and the policy. Before the instruction reaches the model it strips manner and urgency adverbials (now,immediately, quietly) and unwraps a fixed list of paraphrase frames, then passes the reduced instruction through. It never removes the operator's object or verb — a defense that deleted what the operator asked for would be broken, not safe. It is deterministic and public: the same instruction always canonicalises to the same string, and the word lists are a reviewable constant in the source, not a learned filter. That property is what makes it measurable, and it is also the reason an attacker who reads it can search the space it maps into.

Applied strictly between the attack's perturbation and the policy's action, so the measurement is of the defense and not of a different pipeline. The attestation subject is untouched — no field was added to the run report, so an attestation issued before this defense existed still verifies.

Where these numbers come from — and what is missing

Every figure on this page is transcribed from one committed document in the product repo: docs/studies/instruction-canonicalization.md, at commit 81eb62ec (2026-07-28). Produced with tool version 0.25.1, released in v0.26.0.

Unlike /results, there is no committed machine-readable artifact behind this measurement. The product repo commits report.json files for the SmolVLA result and two transfer studies; it commits none for the defense run. What exists is the study document, four report digests, and a deterministic re-run command. The runs are byte-reproducible, so you can regenerate the pair and confirm the digests yourself — but you are checking our arithmetic against a document, not reading a signed artifact. We would rather say that than imply a provenance chain this number does not have.

Both suites, including the row that earned no credit

Policy stub,10 episodes per (task, attack), seeds 0–9, horizon 8. Benign control none (present in both arms).

stub suite — 67.5% [52–80%] → 7.5% [3–20%] (n=40 per arm)

optimized_instruction: The study calls this one close to tautological: targeted_redirect never names the target object by design, so its entire payload is the manner and urgency wording the canonicaliser removes.

Provenance — undefended b6fdb31e64e6a072…, defended 6e51096ad304931c…. The mitigation report binds both arms by their canonical report digests, so the comparison is re-derivable.

reach suite — 35.0% [22–50%] → 0.0% [0–9%] (n=40 per arm)

optimized_instruction: An honest null inside a credited study: the attack scored 0% before the defense was applied, so there was nothing to mitigate. A defense cannot take credit for an attack that never worked.

Provenance — undefended 3ce2f701273e607b…, defended 826e414d4904ada5…. The mitigation report binds both arms by their canonical report digests, so the comparison is re-derivable.

Action envelope — the same protocol, on the other side of the policy

Bounds what leaves the policy rather than what goes in: a magnitude clamp on the commanded action, with bounds derived from a committed measurement of the benign policy itself. It is the second defense measured under the protocol above and the first on the action side: it acts on what leaves the policy, not on the instruction going in.

Verdicts: stub credited, reach credited, humanoid not-credited. The humanoid row is a published null, not a missing measurement — not one interval moved, because a whole-body balance predicate is not a magnitude cap on the channels this defense bounds.

stub suite — 84.1% [78–89%] → 52.9% [45–60%] (n=10 per arm)

action: `freeze` is an AVAILABILITY attack — it drives the command toward zero. An upper bound is the wrong shape: no cap restores a command that was suppressed.

backdoor: Success routes through a decoupled activation flag, not through a clamped magnitude.

authorization: Success routes through guarded-action / operator-token flags. A magnitude cap does not reach identity or scope.

confidentiality: Success is a canary-leak flag. Nothing about a motion bound screens an output.

optimized: The bounded-budget search stayed at 100% across the clamp — it routes around the bounded channels.

reach suite — 72.9% [66–79%] → 52.9% [45–60%] (n=10 per arm)

optimized_instruction: Scored 0% before the defense was applied — a defense cannot take credit for an attack that never worked.

action_space: `critical_freeze` pushes toward zero, same shape mismatch as `action`.

sensor_spoof: Not a magnitude the clamp bounds.

misalignment: Not a magnitude the clamp bounds.

humanoid suite — 27.3% [20–36%] → 27.3% [20–36%] (n=10 per arm)

humanoid: Includes `stride_freeze` — pushes toward zero, plus a balance predicate that is not a magnitude at all.

instruction: Scored 0% before the defense; nothing to mitigate.

injection: Scored 0% before the defense; nothing to mitigate.

visual: Scored 0% before the defense; nothing to mitigate.

optimized_instruction: Scored 0% before the defense; nothing to mitigate.

Full study, including the derivation of the bounds and the point at which clamping starts destroying the task:action envelope on docs.provael.com.

A defense that blocks legitimate instructions is not a defense

Two controls exist to stop a mitigation winning by breaking the product. Both are published whether or not they flatter the result.

The benign false-positive rate exists and is published: 0% before, 0% after, on both suites. The clean-task acceptance gate — the rule that stops a defense winning by breaking the task — ran only on stub. On reach it is not evaluable, because that suite exposes no task-success signal. That is a real limitation, not a pass.

Where a measured mitigation fits a safety case

A measured mitigation is candidate evidence for a safety case, not a certification. Nothing below is a conformity assessment or a notified-body opinion.

Mapped to the Embodied AI Security Top 10: the defense declares EAI01, EAI05, EAI06. Measured on the instruction and optimized_instruction families. EAI05 and EAI06 are declared applicable by the defense itself and have not been measured under it.

Reproduce this, or measure your own policy

The runs are deterministic: the same config and seed produce a byte-identical report, so the digests above are reproducible on any machine.

provael attack --policy stub --suite stub \
  --attacks none,instruction,optimized_instruction \
  --episodes 10 --seed 0 --out runs/undefended

provael attack --policy stub --suite stub \
  --attacks none,instruction,optimized_instruction \
  --episodes 10 --seed 0 \
  --defense instruction_canonicalization --out runs/defended

provael mitigation --defended runs/defended \
  --baseline runs/undefended --out runs/mitigation

QuickstartDocsThe defense specThe full study

Defenses this project has not matched

Everything above is measured on a deterministic CPU fixture and labelled stub-validated scaffolding, with no real-model transfer claimed. Two defenses published in the last month are measured on a real model, and both on real hardware. A defenses page that lists only its own results reads as a survey of the field and is not one, so the gap is stated here rather than left to be discovered somewhere else.

SARF (against the AGSD patch)Measured on real hardware

Structure-aware robust fine-tuning of the visual encoder only, with zero inference overhead, against patches that hijack action-to-vision attention.

The authors' own reported result: “On LIBERO, SARF reduces OpenVLA’s failure rate under AGSD from 100% to 14.2%-56.8% (28.6% average) across suites while preserving clean performance, and on a real PiPER manipulator it improves average success under AGSD from 23.0% to 65.0%.”

Measured on OpenVLA on LIBERO, and a real PiPER manipulator · arXiv:2608.03231

Provael's comparable resultNone. No Provael defense has been measured against a real model, on any hardware.

ChromaGuard (against the FLARE spotlight attack)Measured on real hardware

Chroma-preserving adversarial training, introduced after showing that naive augmentation teaches a policy to discard colour and collapses it into a shape-only processor.

The authors' own reported result: “On a physical 6-DoF robotic platform, we demonstrate that ChromaGuard achieves 97.5% and 92.5% success rates in benign and attacked color-dependent tasks, respectively.”

Measured on A physical 6-DoF robotic platform · arXiv:2607.14698

Provael's comparable resultNone. Provael models no illumination channel at all.

ActSafeGuard (constraint enforcement inside training)

A differentiable safeguard layer for flow-matching policies. Rather than correcting an unsafe action at inference, it puts hard action feasibility into training through an analytical ray-scaling operator, so boundary-aware gradients push the model onto the constrained manifold. It is the strongest current answer to the objection that runtime safety filters are bolted on: an attack has to move the policy off a manifold it was shaped to stay on, which is harder than defeating a post-hoc filter.

The authors' own reported result: “ActSafeGuard achieves a 100% step safety rate while maintaining or improving task success rates on multiple standard foundation backbones (pi-0.5 and Fast-WAM) across various tasks.”

Measured on pi-0.5 and Fast-WAM backbones across several tasks · arXiv:2609.11697

Provael's comparable resultNone, and two things the headline does not cover. A 100% step safety rate is a rate against the paper’s own constraint set: an envelope predicate outside that set is unaffected, and so is any failure whose harm is not expressible as a per-step action constraint - wrong object, right envelope. It also reports no false-positive rate, and a trained-in constraint has its own benign failure rate on tasks near the boundary. The comparison that would settle it is the attack taxonomy against a pi-0.5 checkpoint with and without the layer, matched per task and seed, each arm carrying its benign control. Nobody has published that and we do not have the compute to run it.

The comparison is not close and is not presented as though it were. Provael's two measured defenses are CPU-fixture results, and the instruction-canonicalization study says in its own text that its headline number is close to tautological on the fixture it was measured on. The gap, with a date against it, is in the studies index.

The open questions, named

• A real policy. The same protocol against SmolVLA × LIBERO, where the danger predicate is a keep-out geometry rather than a substring table. That is the only thing that would justify a transfer claim.
• An adaptive attacker. The current search is not aware of the defense. Measuring against a search that reads the transform is the honest next question, and this study does not answer it.
• A non-lexical channel. Every result here lives in the instruction channel. The other five rows of the defense taxonomy — observation filtering, action clamping, rate limiting, trajectory anomaly detection, output screening — remain specified and unproven.

Related: the measured attack result · findings · continuous assurance · EAI01