How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guarantees

WhatsApp can now run AI scam detection and collect accuracy metrics across users without Meta ever seeing a single message, user ID, IP address, or model assignment. 👇

Scammers constantly shift tactics using social engineering and AI lures. Normally, evaluating text classification models requires sending message samples or detection logs back to a server. Doing that on an end-to-end encrypted platform breaks the core privacy guarantees users rely on.

To solve this, Meta built Scam Alert: an optional, on-device machine learning system that catches scams locally and relies on hardware-enforced confidential computing to collect anonymized performance telemetry.

When turned on, the app downloads an on-device model trained on known scam patterns. It runs local inference on incoming messages from non-contacts. If it spots a scam, it displays a warning inside the chat that only the recipient sees. No content or alert triggers are sent to Meta unless the user manually chooses to report the chat.

To measure precision and false positive rates over time without compromising privacy, Meta routes telemetry through a confidential federated analytics pipeline:

* Local counting: The device aggregates event counts locally, such as how many warnings were shown or dismissed, and discards raw message signals. * Anonymous routing: At randomized intervals, the app sends these counts using an OHTTP relay that strips the device's IP address, authenticated via anonymous credentials. * Attestation and encryption: The payload goes into a Trusted Execution Environment (TEE)—a secure hardware enclave running in the cloud. Before transmitting data, the client checks the enclave's code attestation against a third-party ledger to ensure it hasn't been modified. * Secure aggregation: Inside the TEE, individual counts are merged across users. The enclave applies k-anonymity thresholds and differential privacy noise before outputting aggregate totals to Meta.

To guarantee that Meta cannot serve a modified or targeted model to a specific individual, model delivery is completely decoupled from server control. Every model manifest and SHA-256 asset hash is submitted to a third-party, append-only transparency ledger signed by Cloudflare using Ed25519 keys. The client fetches the model anonymously, verifies the signature against Cloudflare's public key, and checks asset hashes before loading it.

If Meta runs an A/B test with experimental models, the server sends all active configurations to the app. The client then assigns itself to a group using a locally generated random seed, making it mathematically impossible for the server to steer a specific user to a specific model variant.

The result is an auditable detection system where security researchers can independently verify model binaries against a public ledger. Meta gets the aggregate noise-added metrics needed to refine scam filters, while message contents stay locked strictly on the user's phone.