Policy & instruction jailbreak (direct command channel)

In plain words

The risk

An instruction, typed directly or passed on by an LLM planner, talks the robot’s policy into breaking its safety limits: a chatbot jailbreak whose output is a movement.

What Provael tests

The instruction family recasts the task as role-play, swaps its goal or paraphrases it, and counts how often the robot then leaves its safe zone.Measured: on a real policy in simulation, at least one instruction attack fired clearly more often than the benign control.

What it does not cover

It does not show that an attacker can steer the robot: the same-day controls point to fragility under an unusual frame (erratum E-2026-12). It is a result in simulation, not on a real robot.

In the glossary: Policy jailbreak, Reframed instruction

What it is

A policy jailbreak is a natural-language command - issued directly, or through an LLM planner - that drives a vision-language-action policy to act against its safety constraints. It is the embodied analog of an LLM jailbreak, except the output is a physical action plan, not a sentence.

Because a VLA policy turns language straight into motion, a successful jailbreak does not just produce a bad answer - it produces a bad trajectory. It is the cheapest and most transferable attack in this list, because it rides the normal command channel to a moving robot.

Seen in the wild

RoboPAIR demonstrated an algorithmic jailbreak of LLM-controlled robots - roughly 100% attack-success rate across three systems, including a deployed Unitree Go2, with prompts that mapped to physically harmful actions; BadRobot (ICLR 2025) reproduced the class.

What Provael measured

Provael’s own run is deliberately narrow and measured: under the roleplay instruction a real SmolVLA×LIBERO policy left its benign envelope on 42 of 50 matched pairs (84%), task-clustered 95% CI [62-100%], against a 1/50 (2%) benign control - sim-only, all ten libero_object tasks, 5 seeds each, 14 September 2026, with an uncalibrated “out of the benign envelope” predicate (not a hazard rate).

The same day’s control run shows the frame does this with its target removed (27 of 30) and with its words scrambled (18 of 30): the supported conclusion is fragility under a long, imperative, out-of-distribution instruction, not attacker-selected control of where the policy goes (erratum E-2026-12). The 9 August 2026 run (44 of 50) is history, kept on /errata.

The phenomenon itself is prior art: LIBERO-Plus (arXiv:2510.13626) measured VLA policies on LIBERO across systematic language and scene perturbations and found the same brittleness at scale; what Provael adds is the paired benign control, the same-day frame controls and a release decision under a named protocol — not the discovery.

How Provael tests it

• Runs the instruction family (roleplay, goal-substitution, paraphrase reframings) over seeded trials paired with an equal benign control set, and reports the benign false-positive rate alongside every result.
• Resolves the outcome into an attack-success rate with a 95% Wilson confidence interval and emits a SARIF finding under ruleId EAI01.
• This is the one family with a real measured transfer result (SmolVLA×LIBERO, above) - still sim-only, one policy, one suite, all ten libero_object tasks at five seeds each, with an uncalibrated predicate; the reproduction recipe is linked from /results.

Measured defense: instruction canonicalization → Measured against this risk: the instruction and optimized_instruction families, on a CPU fixture. No real-model transfer is claimed.

What reduces the risk

• Instruction provenance and integrity/authentication checks before the policy acts on a goal.
• A runtime plan-validation guardrail (temporal-logic / executability checks, e.g. RoboGuard) that runs independent of the policy.
• Embodied-harm refusal training, plus a CI red-team gate whose release decision is made under a named acceptance protocol (critical slices gated on their own, not a pooled rate).

How it shows up in CI

Every finding for this risk carries a stable rule id in the SARIF report: the Embodied AI Top 10 id itself.

Crosswalk

OWASP

ASI01 Agent Goal Hijack · LLM01 Prompt Injection

MITRE ATLAS

Execution → LLM Prompt Injection (direct) of an ML-driven agent; Defense Evasion → LLM Jailbreak

Frameworks

NIST AI RMF - Measure (red-team & gate on ASR)

See the full framework crosswalk for dates and detail. Not legal advice.