WhatsApp、プライバシー保護型オンデバイス ML で詐欺検出テスト開始

WhatsApp is testing Scam Alert, an optional on-device machine learning feature that warns users when messages from non-contacts match patterns associated with scams. The limited beta is accompanied by a technical design that keeps message content on the device during classification while using confidential computing, differential privacy, and transparency mechanisms to measure model performance and prevent targeted model delivery.

Once enabled, Scam Alert downloads a small machine learning model to the device and evaluates incoming messages from non-contacts using conversational structure and linguistic signals. Meta says the model was trained on patterns observed in scam conversations from reports users previously sent to WhatsApp. If the model identifies a likely scam, the user sees a warning that is not visible to the sender and can block, report, or continue the conversation. Users can also mark a chat as trusted, after which Scam Alert will not flag that conversation again. They can optionally share the last 5 messages received from a trusted chat with WhatsApp to help improve the feature.

The architecture separates message classification from performance measurement. The device aggregates warning events and user actions locally into counts. These metrics are transmitted through an Oblivious HTTP relay using anonymous credentials and processed inside confidential virtual machines, a form of trusted execution environment. The client verifies the code running in the confidential environment and checks privacy parameters before transmitting data. The aggregation process applies minimum cohort thresholds and differential privacy before only approximate population-level statistics are made available to WhatsApp.

Meta also treats model distribution as a security boundary. Each production or experimental model version and its SHA-256 hash is published to a third-party append-only transparency ledger before deployment. Clients verify the ledger entry, model signature, freshness, and hash before loading a model. Model downloads use anonymous credentials and OHTTP, while experiment assignment occurs locally, preventing the server from selecting a particular model variant for an individual user.

The approach builds on Meta’s PAPAYA federated analytics system, presented at USENIX NSDI 2025. PAPAYA uses trusted execution environments and on-device processing for privacy-preserving federated analytics, distinguishing analytics workloads from federated learning. Scam Alert applies that foundation to a consumer messaging system where the telemetry itself is treated as sensitive.

A comparable approach exists in Google Messages, which provides real-time spam protection for scams and phishing and uses privacy-preserving mechanisms for detection. Google also documents on-device detection for some message safety features. The implementations differ in their architectural boundaries and the types of processing performed locally or through Google services.

Stephanie Lio, Head of AI Product Marketing at WhatsApp, described the Scam Alert design on LinkedIn as on-device only and emphasized no automatic reporting and that Users remain in control.

Meta says the limited beta will allow the company and its Bug Bounty community to stress test the implementation before broader availability. The company is expanding its Bug Bounty program to cover the confidential federated analytics pipeline and will publish the confidential virtual machine binary and privacy-relevant source components for independent security review.

About the Author

Leela Kumili

Leela is a Lead Software Engineer at Starbucks with deep expertise in building scalable, cloud-native systems and distributed platforms. She drives architecture, delivery, and operational excellence across the Rewards Platform, leading efforts to modernize systems, improve scalability, and enhance reliability.

In addition to her technical leadership, Leela serves as an AI Champion for the organization, identifying opportunities to improve developer productivity and workflows using LLM-based tools and establishing best practices for AI adoption. She is passionate about building production-ready systems, enhancing developer experience, and mentoring engineers to grow in both technical and strategic impact. Her interests include platform engineering, distributed systems, developer productivity, and bridging technical solutions with business and product goals.

Show moreShow less