OpenAI、犯罪スキャム運用を妨害する取り組みを発表

はじめに

Earlier this year, we disrupted a Cambodia-based scam operation that used ChatGPT to support investment, romance, gambling, and law enforcement impersonation schemes. We began investigating this activity following a lead from our peers at WhatsApp and have since shared additional threat signals with industry partners and relevant authorities.

The operation illustrates an important reality about modern scam networks: organized criminal groups rarely restrict themselves to a single type of scam. Instead, they opportunistically employ whatever narratives, personas, and tactics they think will be most effective to deceive victims. In our investigations we routinely observe actors moving between scam types, or combining multiple scam techniques within a single operation.

Some users in the network also generated content suggesting links to human trafficking and forced criminality. These observations are consistent with extensive public reporting⁠(opens in a new window) describing⁠(opens in a new window) organized crime groups in Southeast Asia that recruit workers with promises of legitimate employment before trapping them in systems of debt bondage and coercion. While we cannot independently determine the circumstances of every individual involved, the activity serves as a reminder that the people conducting scams can themselves be victims of exploitation.

Actor

The network used our models to create and support the operation of fake online personas, generate and translate messages sent to scam targets, create promotional content for their fraudulent schemes, and assist with day-to-day operations.

As with past⁠(opens in a new window) scam networks we have disrupted, a subset of users also employed ChatGPT for administrative work, including drafting internal announcements, translating messages between staff, and documenting matters that appeared related to recruitment, immigration status, working conditions, and employee discipline.

Behavior

The network simultaneously conducted multiple types of scams, often blending elements from different schemes. For instance, operators used dating personas to build trust before introducing fraudulent investment opportunities involving cryptocurrencies and spot gold trading. Other users engaged in lengthy romantic conversations with targets using fictitious identities, posed as representatives of online gambling platforms offering fake bonuses and winnings, or impersonated law enforcement agencies to tell targets they needed to pay fines for committing serious criminal offenses.

Although the narratives varied, users across the network consistently displayed the same underlying pattern of deceptive behavior. For example, they created and operated fake dating profiles, fictitious investment experts, and fraudulent law enforcement personas. They also generated images of forged documents, including passports, legal notices, stock-purchase confirmations, and gambling platform interfaces.

• The ping: The network used ChatGPT to translate and generate conversations with targets on messaging platforms such as WhatsApp and Telegram. Scammers also created social media content and researched dating profile material to support their fake personas.
• The zing: Scammer messages frequently relied on emotional pressure and trust-building techniques. Examples included promises of guaranteed returns and “risk-free” investments, romantic language, instructions to keep conversations secret, and urgent requests for action before fictional bonuses expired.
• The sting: The scammers instructed victims to make deposits to unlock purported rewards, pay activation fees, settle fictitious fines, and then provide screenshots of transfers or account information as proof of payment.

Human Trafficking Indicators

In addition to scams, some users generated content suggesting involvement in other violative activities that we detect and disrupt, such as human trafficking or forced labor. This included creating social-media advertisements for “chatter” jobs in Poipet that promised flights, accommodation, meals, visas, and work permits.

Other activity appeared to relate to the administration of workers inside the operation. Users maintained records of employee debts, salary deductions, disciplinary fines, and loan repayments, and translated discussions about immigration status, work permits, visa overstays, and recruitment incentives.

Some conversations also referenced apparent detention, escape attempts, and potential criminal liability for people who had been trafficked and forced to work in scam operations. While these conversations do not allow us to determine the circumstances of any particular individual, they are consistent with extensive public reporting⁠(opens in a new window) describing⁠(opens in a new window) the activities of organized crime groups in Southeast Asia.

Impact

We banned the ChatGPT accounts associated with this operation, shared relevant indicators with industry partners and relevant authorities, and took steps to make it harder for these actors to regain access to our products and services.

The full scale of financial losses associated with the network is unknown, but based on the scammers’ own communications, the operation may have interacted with hundreds of targets across multiple scam types. User conversations referenced individual victims losing thousands of dollars, although we are unable to independently verify those claims.

More broadly, this case reinforces two trends. First, organized scam networks can be highly diversified, operating multiple fraud schemes simultaneously rather than narrowly adhering to a single scam type. Second, the boundaries between online fraud, organized crime, and human trafficking are often blurred. Effective disruption therefore requires targeting not just the victim-facing scam activity, but also the criminal organizations that orchestrate and profit from it.