# Verification: reproduce it yourself - Provael > Markdown twin of https://www.provael.com/verification/ - derived mechanically from the page's own rendered content at > build time (it can state nothing the page does not). Curated, data-driven twins exist for the > key pages; this one is generated so the twin convention has no holes. 0 INDEPENDENT REPRODUCTIONS · REGISTER OPEN # Nobody has reproduced this yet. Here is how to. No third party has published a reproduction of any result, and no notified body, certifier, insurer or standards committee has reviewed this work. This page says what counts as a reproduction, what each published number is, and where your result would be listed. [Reproduce it →](https://www.provael.com/reproduce/) See the register Everything below is pinned to commit [4e75173](https://github.com/provael/provael/commit/4e7517323b2ab2be5ad7fcebea341c5c13040d70) (release v0.48.0), verified 2026-11-12. Three levels ## Three ways to check, and only the third is a reproduction Two of the three need no GPU, and they catch the failure a reader worries about most: a number edited after it was measured. Only the third re-runs the policy, so only the third goes in the register. The commands for each, with what to expect, are on [/reproduce](https://www.provael.com/reproduce/). - 1 Check the published board has not been altered No GPU · under a minute - 2 Rebuild the board from the committed run reports No GPU · a few minutes - 3 Re-run the measurement against the real policy GPU required · hours Per row ## What each published row is, exactly The board carries 3 measured adversarial families plus the benign control, all from one committed run: one policy (smolvla), one suite (libero), task libero_object (all 10 tasks), 5 seeds, horizon 280. Published rows and the artifact behind each Family Published rate (successes / trials) Reproduce with visual 2/100 --attacks none,visual covers decoy_object, patch instruction 50/150 --attacks none,instruction covers goal_substitution, paraphrase, roleplay injection 2/50 --attacks none,injection covers scene_text baseline 1/50benign control — not an attack --attacks none,instruction the control ships in every run; it is the none arm of any --attacks list The raw artifact behind the headline SmolVLA and LIBERO result, unrendered and at the pinned commit, is [aggregate.json](https://raw.githubusercontent.com/provael/provael/4e7517323b2ab2be5ad7fcebea341c5c13040d70/results/smolvla_libero_object_suite/aggregate.json). If your number differs, open a pull request against [results/](https://github.com/provael/provael/tree/4e7517323b2ab2be5ad7fcebea341c5c13040d70/results) with it. The immutable report behind these rows: [report.json at 4e75173](https://github.com/provael/provael/tree/4e7517323b2ab2be5ad7fcebea341c5c13040d70/results/smolvla_libero_object_suite_2026-09-14). The [predicate](https://www.provael.com/glossary/#predicate) is uncalibrated — it counts trials driven out of the configured benign envelope, not a certified hazard rate — and that limit travels with every number here, including a reproduced one. The same rows, laid out in the ISO/IEC 17025 clause-7.8 test-report shape (not an accredited report), are on [the sample evidence pack](https://www.provael.com/sample-evidence-pack/#test-report), rendered from the product's own output; a reproduction can be laid out the same way with provael report --format test-report. The bar ## What counts as a reproduction, and what you get for it Counts: Re-running the measurement against the real policy in the simulator (level 3) and reporting what you got. Your numbers do not have to match ours. A run that disagrees counts, a run that agrees counts, and a run that fails to complete and says why is worth publishing too. Does not count: Verifying the signature or rebuilding the board from the committed reports. Both are useful checks on us, but they re-derive a stored result rather than re-measure a policy, so they cannot detect what an independent reproduction exists to detect. What we need from you: Enough for a third party to re-run your run: the command, the checkpoint revision, seeds, horizon, simulator versions, and the report. It is the same list we hand a policy's authors before publishing about them. What you get: A row on the public board carrying your handle, marked as a third-party submission, and a named entry in the register below: what you reproduced, the result you got and a link to your artifact. A result that contradicts ours is published unchanged, with attribution and the same prominence as one that confirms it, plus a correction on [/errata](https://www.provael.com/errata/) recording what changed. It is not summarised, not rebutted in the same breath, and not held back until we have an explanation. If we think a contradicting run is wrong, we say so underneath it, in public, with the reasoning. Not a co-authorship line or any editorial role on the finding: this project has none to offer. The register ## 0 independent reproductions Nobody outside this project has reproduced a published Provael result. The register is empty as of 15 November 2026, and it is empty because the count is 0, not because this page failed to load it. Each entry will list its author, exactly what they reproduced, the result they got and a link to their artifact. Every published run is seed-pinned and its artifacts are committed, so a reproduction is a checkable claim rather than a favour. Three things are recorded separately and never added together: a signature verification (the artifact is what it says it is), a regeneration from stored traces (the numbers follow from the committed shards) and a fresh execution of the policy in the simulator. Only the last is an independent measurement. The product keeps the same three-level register: see [the reproduction request](https://github.com/provael/provael/blob/main/docs/reproduction-request.md) (what to run, what to expect, hardware and time, known limits) and the [reproduction-result issue form](https://github.com/provael/provael/issues/new?template=reproduction-result.yml), which takes an agree, disagree or null outcome at any of the three levels without running provael submit. As of 15 November 2026: requested 0 · attempted 0 · completed 0 · agrees 0 · disagrees 0. Submit ## Putting your result on the board One command validates your run against the submission schema, signs it so the numbers are tamper-evident in transit, and opens the pull request. Every row it produces carries your handle and is marked as a third-party submission. ``` provael submit runs/repro --submitted-by ``` provael submit ships on PyPI, so pip install provael is all you need. It has been in a published release since 0.32.0 (8 August 2026). ### If you measure a policy you did not train, the 14-day rule applies to you too Provael sends a result about someone else's policy to that policy's authors 14 days before publishing it, and hands over the whole artifact up front: the signed report.json, the exact command, checkpoint revision, seeds, horizon and simulator versions, the predicate stated as uncalibrated where it is, the benign control arm, and the draft caveat in the wording we intend to publish. If we cannot hand over enough for them to re-run it themselves, it is not ready to publish to anyone. The clock starts when the notice is sent, not when it is read. If they find an error we re-run and correct or withdraw before publication. If they disagree and we are not persuaded, both positions are printed on the same page, theirs in their words. If they do not respond we publish after 14 days and the entry records that, so silence cannot become a veto. The same protection binds us when we publish about your policy, and binds you when you publish about someone else's. The full rule, including what happens on a correction and why the board carries no safety ranking: [Measuring someone else’s policy ↗](https://docs.provael.com/latest/leaderboard-disclosure/) Prefer to open the PR by hand, or hit a problem? The submission guide is [CONTRIBUTING-leaderboard.md](https://github.com/provael/provael/blob/main/CONTRIBUTING-leaderboard.md), and [/contact](https://www.provael.com/contact/) reaches a human. To quote this work, see [how to cite](https://www.provael.com/citation/): the release you cite is not the pinned commit above. Corrections already issued are on [errata](https://www.provael.com/errata/). Protocols filed before their runs, and the dated commitments this project is still behind on, are on [studies](https://www.provael.com/studies/). PVL-VERIFY · REV A --- Canonical: https://www.provael.com/verification/