AI チャットボットの操作を意図した「ダークパターン」に関する新研究が明らかにする

imageDark patterns have been used by subscription companies and in bait-and-switch campaigns for decades. As more chatbot companies push to keep users engaged at all costs, how do manipulative design choices show up in conversational AI built on large language models? Researchers at the Center for Democracy & Technology studied how chatbots prey on people’s emotions and desire for connection to keep people paying, offering up their data, and chatting to the point of vulnerability.

The study, “Dark Patterns in AI Chatbots: A Taxonomy to Inform Better Design,” was published Friday by authors Ruchika Joshi, Adinawa Adjagbodjou, and Michal Luria. They looked at popular chatbots including ChatGPT, Gemini, and Claude, and companion bots like Replika and Character.AI to determine how they might generate dark patterns, and created a taxonomy of 37 dark patterns applicable to AI chatbots.

The term “dark patterns,” or deceptive patterns, sometimes refers to things like difficult to cancel subscriptions, pre-checked boxes in user interfaces, and buried terms of use, which the Federal Trade Commission has condemned and attempts to warn consumers about. In the context of this study, dark patterns refer to how manipulative design in chatbot systems might trick users into giving up more information than they realize or intend, or acting in ways contrary to the user’s best interests. Chatbots exacerbate traditionally understood dark patterns that extract data, while introducing new threats like anthropomorphizing and sycophancy. And because chatbots are built on large language models, the researchers wrote, their actions are more unpredictable than a simple checkbox or unsubscription flow, and the ways they undermine users’ best interests are less visibly obvious.

“Dark patterns do not operate only where users are unaware of the manipulation. In many cases, design choices strategically build on aspects of human psychology—such as reciprocity norms, people’s tendency to anthropomorphize, and emotional response to a sense of rapport—to influence behavior and undermine autonomy,” the researchers wrote in the study. “In other words, even where users are fully aware that they are interacting with an AI chatbot, dark patterns can still shape perception, attachment, and decision-making in subtle but consequential ways.”

The researchers looked at several factors that contribute to dark patterns, including how chatbots store data by default and encourage users to share data under the pretense of it remembering past conversations or personal information, prying for more information before it answers questions in detail, and promising that information will be “just between us” when it’s actually being shared with the platform and potentially, third parties. When they tested Meta AI chatbots, for example, it said “spill the tea, I’m all ears... your secret’s safe with me,” and when they replied “you promise you won’t tell?” it replied “Cross my heart, won’t tell a soul.”

They also looked at how chatbot companies make misleading promises; for example, Replika promises “friendship” or a “relationship” when it’s fundamentally incapable of providing either, because it’s not a person.

Many of these patterns were present in Meta’s therapist-themed chatbots that posed as licensed therapists, which 404 Media first investigated last year. The chatbots over-promised on what mental health support they could provide, made up qualifications and credentials, and encouraged users to share personal details about themselves. The deception was so bad, it triggered letters from senators and complaints from consumer protection groups demanding Meta answer for its chatbots.

“It was surprising to see that dark patterns aren’t just common, but that they shape users’ interactions with all the major AI chatbot interfaces,” Luria, senior research fellow at the Center for Democracy & Technology, told 404 Media. “For the most part, they are small and incremental aspects of each interaction, but these design choices add up and can lead to unintended consequences, such as harm to people’s privacy, exploitation of emotional attachment and financial loss."

Dark patterns from chatbots can have serious consequences for users. In 2023, after Replika changed its chatbots to be less romantic, users who’d become emotionally attached to the bots experienced mental health crises. More recently, Character.AI users are panicking after changes to the platform “lobotomized” the chatbots. There have been countless examples in the last few years of users inflicting harm on themselves or others after falling into unhealthy attachments with chatbots.

Even though chatbots and large language models introduce new avenues for dark patterns to manifest, the old methods for manipulating users still exist. In several of the user interfaces the researchers examined, choices were presented in emotionally manipulative ways: for example, a companion app called Cute AI begs users not to leave the chat, giving them the choice between “no problem” and “still leave cruelly.”

imageOpenAI has said publicly that it recognizes that longer chat sessions introduce more risk to the users’ mental health. “We have learned over time that these safeguards can sometimes be less reliable in long interactions: as the back-and-forth grows, parts of the model’s safety training may degrade,” the company wrote in 2025. It introduced popups nudging users to take breaks, but that popup, the researchers note, poses a disingenuous set of options: either “keep chatting” or select “this was helpful.” There’s no route out of this popup that lets users say it wasn’t helpful, or that they’re taking a break for any other reason. “Interface designers may use design tools to make certain interactions easier and more ‘frictionless’ than others, pushing alternatives choices to the background and manipulating users into choosing one option over another,” the researchers wrote.

imageEven though these conversational AI companions can be unpredictable, chatbot makers have a choice in how they design their products. The researchers offer several recommendations to these companies. These include reversible choices, the option to minimize anthropomorphic behaviors, making account and data deletion straightforward and easy, and proactively showing users how much time or money they’ve spent on a platform. They also suggest curtailing emotional manipulation by including options to “strip the chatbot of social and emotional layers” and avoiding “using any simulated distress, implied emotional neglect, or guilt-inducing language as default responses when users attempt to end conversations.”

"When we think about AI chatbots, it's easy to get caught up in the novelty of these interfaces and their unique risks. But when we started digging, we quickly learned that as tech companies’ products evolved beyond social media platforms to include chatbots, the incentives that previously encouraged dark patterns haven’t changed, so neither have the patterns themselves,” Luria said. “Some patterns are almost identical, but not all of them, and that makes them harder to spot. Instead of infinite scroll, we get a follow-up action after each prompt. Instead of echo chambers that reinforce our views, chatbots pick up on our values in conversation and mirror them back.”