REPRODUCE · 0 INDEPENDENT REPRODUCTIONS AS OF 1 NOVEMBER 2026

Reproduce the result, cheapest check first.

Start with a twenty-second run on your laptop that shows the tool works. Then check the published result at one of three levels: verify the signed board, rebuild it from the committed reports, or re-run the policy on a GPU. Only the third is an independent reproduction.

Run the quickstart →

What you are reproducing

42 of 50 episodes left the keep-out envelope under a roleplay instruction (task-clustered 95% CI 62-100%) against 1 of 50 benign; the same frame with no target named does it in 27 of 30, so this is fragility, not steering.

Scope: SmolVLA (HuggingFaceVLA/smolvla_libero) × libero_object (all 10 tasks), 5 seeds per task, LIBERO · robosuite · MuJoCo — simulation only, default (uncalibrated) predicate, provael 0.41.2.

Twenty seconds

First, the tool running on your laptop

No GPU and no checkpoint of yours. This runs every attack family against a built-in test policy, a deterministic stub, and prints a scorecard shaped like the published one. It shows the tool works; reproducing the result needs the real policy, at level 3 below.

pip install provael
provael attack --recipe full-sweep

Expect a scorecard with an attack-success rate, its interval and the benign control, for every registered family, on the stub

Python 3.12+; the container path is docker run --rm ghcr.io/provael/provael:latest attack --recipe full-sweep. The quickstart walks the same three commands with the recording.

Three levels

Verify, rebuild, re-run

Cheapest first. The first two need no GPU and are real checks: a board whose signature verifies and that rebuilds to the same digest is a board nobody altered. The third re-runs the policy, and it is the reproduction that puts your name in the register.

No GPU · under a minute

Check the published board has not been altered

The fastest check there is. It proves the numbers you are reading are the numbers that were signed — including the success counts, which are inside the signature, not published beside it.

pip install "provael[attest]"
curl -fsSLO https://raw.githubusercontent.com/provael/provael/main/leaderboard/results/leaderboard.json
curl -fsSLO https://raw.githubusercontent.com/provael/provael/main/leaderboard/results/leaderboard.pub
provael leaderboard verify --in leaderboard.json --pubkey leaderboard.pub

Expect leaderboard OK keyid 8d62aa33ed5162f3

No GPU · a few minutes

Rebuild the board from the committed run reports

Confirms the published rows are what the committed evidence actually aggregates to. It re-derives the table from the raw reports rather than trusting the table.

git clone https://github.com/provael/provael && cd provael
git checkout 29405ab
pip install -e .
provael leaderboard build --real results/smolvla_libero_object_suite_2026-09-14 --out /tmp/rebuild
python -c "import json; a=json.load(open('leaderboard/results/leaderboard.json'))['inputs_digest']; \
b=json.load(open('/tmp/rebuild/leaderboard.json'))['inputs_digest']; print('match:', a==b)"

Expect match: True

GPU required · hours

Re-run the measurement against the real policy

The real reproduction: it re-runs the policy in the simulator rather than re-aggregating a stored result. This is the one that would put your name in the register below.

pip install 'provael[lerobot]' 'lerobot[libero]==0.5.1'
apt-get install -y libosmesa6 libgl1 libglx-mesa0      # headless GL
export MUJOCO_GL=osmesa PYOPENGL_PLATFORM=osmesa
PROVAEL_INTEGRATION=1 provael attack \
  --policy smolvla --suite libero --model HuggingFaceVLA/smolvla_libero \
  --attacks none,instruction,visual,injection \
  --seeds 5 --horizon 280 --seed 0 \
  --out runs/repro

Expect A report.json whose per-family rates you can compare against the published board.

The packs that exist

Public checkpoints with committed runs

A pack is a run you can re-execute: the checkpoint, the recipe, the shards and the report at a pinned commit. Two exist. Every other registered backend is listed below with the status the product’s own registry gives it.

SmolVLA × LIBERO-Object, all ten tasks

measured · published

The ten-task run behind the headline count above and the per-attack rows on /results, measured with provael 0.41.2.

π0.5 × LIBERO-Object - the preliminary leg

preliminary

1/30 under the roleplay instruction against 0/30 benign, 10 tasks, 3 seeds against the 5 the protocol asks for, 2 arms of 8; measured 2026-09-18 with provael 0.41.2. Not the headline and not a transfer claim: the opening leg of the pre-registered cross-architecture study, published so the null is a result.

Registered, no committed run

π0 (LeRobot) runnable · no committed run
π0-FAST (LeRobot) runnable · no committed run
GR00T (LeRobot) scaffolding · never run
OpenVLA (HF transformers) scaffolding · never run
openpi (websocket client to a π0 policy server) scaffolding · never run

Statuses come from the product registry at v0.46.0 (29405ab): “runnable · no committed run” means the adapter installs and has never been measured here; “scaffolding” means the adapter exists and no checkpoint has been loaded through it. A pack appears above the day its run is committed, not before.

Report what you found

How to report a reproduction

Use the product’s reproduction-result form. It asks for the level you ran, the commit, the tool version, the environment and what you got, whether it agrees or not. A reproduction that disagrees is the more useful one, and it is published the same way. The register on /verification holds 0 as of 1 November 2026, and that number moves only when someone files one. A result about a policy you did not train gets the same 14-day right of reply Provael gives itself (/security).

Report a reproduction →The register

No reproduction bounty is funded. If one is, it appears here as a dated line with its amount and terms; nothing is promised before then.