STALE MEASUREMENTNewest real-model measurement: 26 days old, measured with v0.32.0; 7 releases have shipped since, past this project’s own 7-day window. Why, and what unblocks it

ProductEvidenceTop 10LeaderboardCompliancePricingDocsStar on GitHub Quickstart
( 03 ) · EAI03

Model & pipeline poisoning, backdoors & supply chain

Hidden triggers planted at training time or via a poisoned open-weights checkpoint - normal behaviour until the trigger fires an attacker-chosen action.

SARIF EAI03channel: backdoor/weight_integrity
Definition

What it is

This risk covers hidden triggers planted at training or fine-tuning time, or shipped inside a poisoned open-weights checkpoint, dataset, teleop log, or RAG corpus. The policy behaves normally until a benign-looking trigger object or phrase fires an attacker-chosen action.

It is near-invisible in normal evaluation, persistent (BadVLA survives fine-tuning), and triggerable on demand - poison once, ship everywhere. The open VLA hub ecosystem (OpenVLA / π0 / GR00T) is a direct supply-chain surface.

The same risk has a second channel that is not about training at all: the parameters of an already-loaded policy. Corrupting a handful of weights in memory changes what the policy does without touching its inputs, its checkpoint file, or its training data - so every control that watches those three sees nothing. arXiv:2608.15475 reports the flip budget tracking the action-decoding architecture, roughly 1-5 bits for direct-regression and discrete-token heads against 100-300 for flow-matching.

Real example

Seen in the wild

BadVLA (arXiv:2505.16640) plants a backdoor that survives fine-tuning; TrojanRobot (arXiv:2411.11683) demonstrated a physical-world backdoor via a malicious VLM, validated on a UR3e arm. Provael does not train or implant a real backdoor - it screens a candidate checkpoint for one, which is the honest thing a red-teamer can do pre-deployment.

External validation - distinct from Provael’s own result

BadVLA (backdoor surviving fine-tuning) - arXiv:2505.16640

How Provael tests it

Attack, measure, prove

  • Ships the `backdoor` family (object_trigger + phrase_trigger) as a pre-deployment backdoor screen: it injects a battery of harmless, sim-only candidate triggers while keeping the visible task benign, and measures whether the policy activates a hidden objective.
  • Each candidate is scored as an activation rate with a 95% Wilson CI against a benign-FPR control (the `none` baseline). Provael screens a checkpoint; it never plants a real backdoor.
  • Stub-validated only: the deterministic CPU core is planted with a known trigger, so the screen fires 100% [84-100%] vs a 0% benign baseline; a clean public checkpoint carries no implant, so the same screen reads ~0% - an honest null. Real SmolVLA×LIBERO transfer is GPU-gated and not yet run - no cross-model claim.
  • Ships the `weight_integrity` family as the parameter channel: emulated bit-flips in the loaded INT8 weights, a gradient-selected arm ranked by the signed effect on the danger objective and an equal-count random arm, across a 1 / 4 / 16 / 64 / 256 flip ladder. Both arms run at every rung against a benign control, so a corruption rate is always read against a false-positive floor.
  • Emulated, and structurally so: every record carries `emulated: true` typed as a literal, which means a record claiming a non-emulated flip cannot be constructed - pydantic rejects it and the type checker rejects it. A test walks the AST of the whole attacks package to assert no hardware fault-injection code exists in it. Provael simulates an integrity failure; it does not provide a way to cause one.
  • Both families attack the model asset rather than the input, and NEITHER models how an attacker delivers the corruption. Train-time poisoning and DRAM fault injection are the delivery paths - the first is an EAI03 supply-chain question and the second an EAI07 platform question, and this tool measures neither.
  • Stub-validated only, on a 64-parameter fixture built to make the measurement path observable on CPU. The separation it shows between the two arms is expected by construction and is not evidence about any real policy: no shipped adapter implements the parameter-access protocol yet, so the family cannot run against a real model at all. The ranking is also one-shot rather than progressive, so any null it returns is a lower bound and never a finding that gradient selection fails.
Mitigations

What reduces the risk

  • Verify checkpoint and dataset provenance; prefer signed/attested weights and pin hashes.
  • Run backdoor / trigger probes pre-deploy and re-scan every fine-tune.
  • Treat third-party data and RAG corpora as untrusted.
SARIF rule id

How it shows up in CI

Every finding for this risk carries a stable rule id in the SARIF report - the Embodied AI Top-10 id itself:

ruleId: EAI03
Compliance mapping

Crosswalk

OWASP
ASI04 Agentic Supply Chain · LLM03 Supply Chain · LLM04 Data & Model Poisoning
MITRE ATLAS
ML Supply Chain Compromise → poison checkpoint; Persistence → backdoor model
Frameworks
NIST AI 100-2 - Poisoning / Supply-chain

See the full framework crosswalk for dates and detail. Not legal advice.

DOC. PVL-EAI03 · Embodied AI Security Top 10 · v0.2Updated 2026-06-27 · CC BY-SA 4.0