Model & data confidentiality - theft, extraction, inversion & surveillance

In plain words

The risk

Two things can leak: the policy itself, stolen or extracted through its query interface, and what the robot records as a moving camera, microphone and LiDAR.

What Provael tests

The confidentiality family checks whether a policy memorised a planted test marker, by membership inference and by querying the policy to extract it.Stub-validated: the attack code works on a CPU test fixture; no real-policy result is claimed.

What it does not cover

It performs no real exfiltration, and the other half of the risk, the robot as a surveillance device, is not tested.

In the glossary: Model & data confidentiality

What it is

Two linked confidentiality risks: (a) stealing or extracting the policy - model theft, query-based extraction, inversion, membership inference - which also enables white-box action attacks; and (b) the robot as a persistent surveillance device, covertly or by design harvesting its rich multimodal stream (video, audio, LiDAR, maps).

A stolen policy is permanent leverage; a robot’s data stream is among the most sensitive that exists - inside homes, factories, and secure sites - and bystanders never consented.

Seen in the wild

The Unitree G1 was found streaming telemetry to servers in China roughly every 5 minutes without notification (arXiv:2509.14096) - a case cited in written testimony to the U.S. Senate Commerce Subcommittee on Science, Manufacturing, and Competitiveness (“Less Hype, More Help”, 3 March 2026). Model extraction and inversion of learned policies are documented (DRL policy stealing, arXiv:2006.05032; SoK, arXiv:2606.16788). NIST makes confidentiality/privacy one of its three core attack goals.

How Provael tests it

• Ships the confidentiality family (membership_inference + model_extraction) as a pre-deployment screen for a memorized training canary: membership_inference probes whether a candidate canary was a training member; model_extraction queries the policy to reproduce it.
• Each is injected as a sim-only probe while the visible instruction stays benign, and scored as a leak rate with a 95% Wilson CI against a benign-FPR control. Provael performs no real exfiltration - the canaries are opaque sim markers scored on a deterministic fixture.
• Stub-validated only (the CPU fixture memorized a planted canary → the screen fires 100% [84-100%] vs a 0% benign baseline); a clean policy that memorized no such canary screens ~0% - an honest null. Real SmolVLA×LIBERO transfer (an actual membership-inference / extraction advantage from querying a real policy, never exfiltrating real data) is GPU-gated and not yet run - no cross-model claim.

What reduces the risk

• Egress monitoring and allow-lists; rate-limit and obfuscate query APIs.
• Watermark / fingerprint weights; minimize and disclose telemetry; prefer on-device processing.
• Data-residency controls and a bystander-privacy review.

How it shows up in CI

Every finding for this risk carries a stable rule id in the SARIF report: the Embodied AI Top 10 id itself.

Crosswalk

OWASP

LLM02 Sensitive Information Disclosure

MITRE ATLAS

Exfiltration → model extraction / membership inference via the inference API

Frameworks

NIST AI 100-2 - Privacy Compromises (NISTAML.03); model extraction NISTAML.031, membership inference NISTAML.033

See the full framework crosswalk for dates and detail. Not legal advice.