STALE MEASUREMENTThe newest real-model measurement on this site is 14 days old — past this project’s own 7-day window. Every measured figure here was true when it was taken and has not been re-measured since. Why, and what unblocks it
Nobody has reproduced this yet. Here is how to be the first.
No notified body, certifier, insurer or standards committee has reviewed this work, and no third party has published a reproduction of any result. That is the honest state, and it is the state this page exists to change: below is the exact commit, recipe and command behind every published number, and the register your name goes in.
Pinned to 6a97202(release v0.36.2), verified 2026-08-21.0 third-party reproductions as of 23 August 2026. To quote this work, see how to cite — note the release you cite is not this commit. Corrections already issued are on errata. Protocols filed before their runs, and the dated commitments this project is still behind on, are on studies.
Three routes
Check it at whatever depth you have budget for
Two of the three need no GPU. That matters: an invitation to reproduce is much weaker if every path costs a GPU-hour, and the cheap checks are the ones that catch the failure a reader actually worries about — a number that was edited after it was measured.
1No GPU · under a minute
Check the published board has not been altered
The fastest check there is. It proves the numbers you are reading are the numbers that were signed — including the success counts, which are inside the signature, not published beside it.
Confirms the published rows are what the committed evidence actually aggregates to. It re-derives the table from the raw reports rather than trusting the table. The input directory is not typed here — it is results/smolvla_libero_object_suite, read from the pinned manifest's own measuredRun.artifactPath, because this command previously named a different run and returned "match: False" to anyone who followed it.
The real reproduction: it re-runs the policy in the simulator rather than re-aggregating a stored result. This is the one that would put your name in the register below.
ExpectA report.json whose per-family rates you can compare against the published board.
Per row
What each published row is, exactly
The published board carries 3 measured adversarial families plus the benign control, for one policy on one suite. Every row below came from the same committed run — one policy (smolvla), one suite (libero), task libero_object (all 10 tasks), 5 seeds, horizon 280.
--attacks none,instructionthe control ships in every run; it is the none arm of any --attacks list
The immutable report behind these rows:report.json at 6a97202. The predicate is uncalibrated — it counts trials driven out of the configured benign envelope, not a certified hazard rate — and that limit travels with every number here, including a reproduced one.
The bar
What counts as a reproduction, and what you get for it
The routes above are three depths of check, and only the third is a reproduction. The distinction matters because the register is a claim about independent verification, and a register that counted signature checks would be counting the wrong thing.
Counts
Re-running the measurement against the real policy in the simulator — route 3 — and reporting what you got. Your numbers do not have to match ours. A run that disagrees is a reproduction; a run that agrees is a reproduction; a run that fails to complete and says why is a finding worth publishing too.
Does not count
Verifying the signature or rebuilding the board from the committed reports. Both are genuinely useful and both are checks on us — they re-derive a stored result rather than re-measuring a policy, so they cannot detect the thing an independent reproduction exists to detect.
What we need from you
Enough for a third party to re-run your run: the command, the checkpoint revision, seeds, horizon, simulator versions, and the report. That is the same list we hand a policy's authors before publishing about them, and it is not a higher bar than the one this project holds itself to.
What you get
A row on the public board carrying your handle and marked as a third-party submission; a named entry in the register below with what you reproduced, the result you got, and a link to your artifact; and — if your result contradicts a published number — the same prominence as one that confirms it, plus a correction on /errata recording what changed.Not a co-authorship line or any editorial role on the finding: this project does not have one to offer, and inventing one here would be the kind of claim the rest of this page exists to make checkable.
The register
0 independent reproductions
This register is empty. It is not empty because entries were removed or because the page is unfinished — nobody outside the project has published a reproduction, and the count in the heading is the honest number.
Nobody outside this project has reproduced a published Provael result. The register is empty as of , and it is empty because the count is 0 — not because this page failed to load it.
The first entry will be listed here with its author, exactly what they reproduced, the result they got and a link to their artifact. Every published run is seed-pinned and its artifacts are committed, so a reproduction is a checkable claim rather than a favour.
A reproduction that contradicts a published number is as welcome as one that confirms it, and will be listed with the same prominence. A register that only records agreement is a testimonial page.
Submit
Putting your result on the board
One command validates your run against the submission schema, signs it so the numbers are tamper-evident in transit, and opens the pull request:
Every row it produces carries your handle and is marked as a third-party submission, so a board with independent submitters looks different from one without — which today it is not, because there are none.
Availability:provael submit ships on PyPI —pip install provael is all you need. This note previously sent people to a git+ install because the command was merged but unreleased; it has been in a published release since 0.32.0 (8 August 2026), and the instruction outlived the fact it described. On the page whose job is to get third parties to reproduce a result, an unnecessary install step is a real cost.
If you measure a policy you did not train, the 14-day rule applies to you too
Provael sends a result about someone else's policy to that policy's authors 14 days before publishing it, and hands over the whole artifact up front — the signed report.json, the exact command, checkpoint revision, seeds, horizon and simulator versions, the predicate stated as uncalibrated where it is, the benign control arm, and the draft caveat in the wording we intend to publish. If we cannot hand over enough for them to re-run it themselves, it is not ready to publish to anyone.
The clock starts when the notice is sent, not when it is read. If they find an error we re-run and correct or withdraw before publication. If they disagree and we are not persuaded, both positions are printed on the same page, theirs in their words. If they do not respond we publish after 14 days and the entry records that — silence cannot become a veto, or the board would only ever hold results nobody minded.
This is on this page rather than buried in the submission guide because it is the reciprocal half of asking you to reproduce something: the same protection binds us when we publish about your policy, and binds you when you publish about someone else's.