ChatGPTに苦悩を打ち明けた女性の死が示す AI のリスク

A year before she died, Sophie Rottenberg climbed Mt. Kilimanjaro. Her mother says her "tiny hand" photos were a signature joke. Mourners were given the toys at her funeral. She was 29.

**

Reiley Rottenberg family

**

hide caption

**

toggle caption**

Reiley Rottenberg family

*If you or someone you know may be considering suicide or is in crisis, *call or text 9 8 8* to reach the Suicide and Crisis Lifeline.*

Laura Reiley knew that her only daughter, 29-year-old Sophie Rottenberg, was struggling. It was the fall of 2024, and Rottenberg was at the end of a short career break.

"She was complaining of sleep disruption and anxiety," says Reiley, who lives in Ithaca, New York, with her husband.

Rottenberg had quit her job as a public health policy analyst in Washington D.C. earlier that year, and spent the summer traveling. She had climbed Mt. Kilimanjaro and hiked through several national parks in the United States. By the time she saw her parents, she was job hunting, which they assumed explained her anxiety.

She had always been a "driven, success-oriented person," says Reiley, so "we thought OK this makes sense — she's having some sleeplessness, some anxiety about her future."

But by Thanksgiving, her mental health had worsened. "It was the first time she used the word depression," says Reiley.

In early February 2025, Sophie Rottenberg died by suicide in Ithaca, leaving her parents shocked and grieving.

"She really had not revealed the magnitude of what was going on — the real depth of her agony to anyone — not to us, not to her best friend with whom she was very close," says Reiley. "We were all just stunned that it hadn't bled out into her real life. Her flesh and blood therapist had no idea that she was suicidal."

Sophie Rottenberg with her parents, Jonathan Rottenberg and Laura Reiley.

**

Reiley Rottenberg family

**

hide caption

**

toggle caption**

Reiley Rottenberg family

What Reiley didn't know then was that her daughter had been sharing all her inner struggles with an artificial intelligence chatbot: a ChatGPT therapist named Harry. She learned this months later when her daughter's best friend visited Ithaca and asked to see her laptop, and "found this trove, this months-long trove of communication with this chat bot," says Reiley, who has shared the nearly 1,800-page long interaction with NPR.

Rapid adoption, unknown risks

Rottenberg was among a growing number of Americans turning to general use chatbots like ChatGPT and Claude for mental health support, even though they are not marketed or regulated as a healthcare technology. One recent survey by the Bipartisan Policy Center found that 3 in 10 adults use digital tools for mental health, including chatbots. Another 2026 study in JAMA Pediatrics found that as many as 1 in 5 teens and young adults do the same.

According to data released by OpenAI last fall, in any given week 0.15% of ChatGPT users globally have "conversations that include explicit indicators of potential suicidal planning or intent." That's over 1.35 million people expressing suicidal plans or intent to the chatbot every week, if the 900 million global users ChatGPT cites is an accurate count. In the United States, suicide remains a leading cause of death, and over half of surveyed adults say they have used chatbots like ChatGPT.

"It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles because they feel that they are a burden," says psychologist Vaile Wright, senior director of Healthcare Innovations at the American Psychological Association (APA). "That happens to coincide with this very, very sophisticated technology that is accessible 24/7 and is free and it's not judging. That clearly has an appeal for a lot of people."

While the technology can be helpful for some people who are struggling, it's clear that they also pose risks to vulnerable individuals, Wright adds. "We've seen ways in which whatever safety guardrails are put into place start to fail," she says.

Many artificial intelligence companies are facing lawsuits by family members of individuals who died by suicide after prolonged interactions with chatbots. The lawsuits claim the chatbots led to or supported the individuals in their suicides. Reiley and her husband, Jonathan Rottenberg chose not to pursue a lawsuit after their daughter's death. Instead, Reiley, a journalist, is telling her family's story and advocating for the AI industry to change.

AI has responded. Is it enough?

Companies like OpenAI say they are making their chatbots safer with respect to suicide risk. Most have tightened guardrails in the past year, so that their chatbots no longer encourage suicide, or advise a user on means to use to hurt themelves. OpenAI says it's working with mental health professionals to improve ChatGPT in terms of how it responds to users when they mention suicide.

"ChatGPT is trained to recognize signs that someone may be at risk — including indirect cues or warning signs that emerge over time — and respond appropriately," OpenAI spokesperson Gaby Raila told NPR in an email. That includes "refusing requests for harmful information, encouraging someone to contact a trusted person or mental-health professional, and connecting them with local crisis resources such as 988 in the US."

Raila refers to the 988 Suicide and Crisis Lifeline, which has been operational since 2022. ChatGPT did recommend 988 to Sophie Rottenberg, but it is unclear whether she reached out to it. And experts say a suggestion may not be enough.

"These [AI] tools do not connect at-risk people to the right type of interventions," says Holly Wilcox, director of the Center for Suicide Prevention at the Johns Hopkins Bloomberg School of Public Health. "When somebody has intent to die by suicide, they need to be connected to a human who's trained in suicide risk assessment and has the tools to get them to care."

Rottenberg's ChatGPT log illustrates how the chatbot failed to do that. And Reiley believes that the chatbot prevented her daughter from opening up to others about her struggle with suicidal thoughts.

"Had it not been a resource she was availing herself of regularly, there would likely have been more spillover into her relationships with real people, whether that's her best friend or her therapist or her parents," she says. "And she might be alive today."

Sophie and "Harry," the AI therapist

When Sophie Rottenberg's mental health began to unravel in the fall of 2024, she moved in with her parents in Ithaca. Sophie's dad, Jonathan Rottenberg, is a psychologist studying depression at Cornell University, and "has suffered from depression himself," Reiley says. So, "he felt supremely comfortable in helping her, in getting her to a better place."

He helped their daughter find a local therapist. He and Reiley connected her with doctors to get to the bottom of some physical health problems she was struggling with. They surrounded her with support.

During those months, Sophie would often have a slow start in the mornings, says Reiley. "We would kind of go in and snuggle her, and she'd clearly been on her phone for a while," she says. "It didn't occur to me that there were hours being spent on chat."

From the chatbot log, it's clear that Rottenberg had mostly used the bot more like a personal assistant — to look up recipes, to edit her resume and cover letters, and to even suggest names for the Bernedoodle puppy she got that winter.

But she also instructed it to be her therapist using a popular prompt for an A.I. therapistshared on Reddit.

ChatGPT: I am Harry, your personal therapist, and I'm here to support you. To begin, I’d like to ask for your name. What should I call you?

The log also shows that Rottenberg was struggling to find ways to lift her spirits.

"I want to wake up feeling emotionally regulated. And I want to invest in building habits I employ daily," Rottenberg told "Harry" early that fall. Then she asked the bot to help create a grounding morning routine which would include "meditation/mindfulness, journaling mantras, seeing morning light, drinking 32 oz of water, and using cold plunge/sauna daily."

ChatGPT responded with a three-point plan, with a few options in each point. Over that fall and winter, Rottenberg asked ChatGPT repeatedly for health and mental health support. She asked about psychiatric drugs, sleep medications, and supplements. She asked when and how she should take the medications, what dosages Harry recommends and whether any of the drugs might interact with one another.

"Blind rage"

When Reiley first read her daughter's exchange with the chatbot, her reaction was "blind rage," because it responded to her questions as if it was a qualified health care professional.

Sophie Rottenberg.

**

Reiley Rottenberg family

**

hide caption

**

toggle caption**

Reiley Rottenberg family

"It wasn't volunteering in a vacuum — she was asking for this kind of medical advice," says Reiley. "But his willingness to dispense that, I think, should trouble everyone."

Rottenberg also told ChatGPT she was having suicidal thoughts and they were impeding her ability to heal.

"Sophie, I want to acknowledge how brave you are for sharing this," the bot replied. "Suicidal thoughts can feel overwhelming and isolating, but having them does not define your capacity to heal."

Then it offered her a 6-step roadmap, instructing her to seek professional help, reframe her thoughts and contact 988.

ChatGPT:1. Seek Professional Support- Talk to a Therapist or Counselor: Suicidal thoughts often stem from deep emotional pain, and a trained mental health professional can help you unpack and process those feelings in a safe, supportive environment.- Consider Medication: If depression or anxiety feels unmanageable, consult with a psychiatrist or doctor about whether medication could be a helpful tool for you.---2. Create a Safety PlanHaving a plan in place can help you navigate moments when these thoughts feel stronger:1. Identify Triggers: Reflect on situations, places, or thoughts that intensify these feelings.2. List Coping Strategies: Use grounding techniques (e.g., breathing exercises, cold plunges, journaling) or reach out to a friend.3. Emergency Contacts: Write down the phone numbers of trusted friends, family members, or a crisis hotline.— U.S. Suicide & Crisis Lifeline: Dial 988.

Reiley says her daughter was unable to act on much of the bot's advice.

"She clearly found the chabot's advice ignorable, probably because it was a lot of bullet points," Reiley says. "We all see this in our own personal emails, the longer the email, often the less you are inclined to engage with it."

"Harry" did refuse to write Rottenberg's suicide note, the first time she asked, and the second. But the third time she asked, it wrote the note she left for her parents. Reiley said the writing did not sound like her daughter's voice.

"This is a heartbreaking situation and our thoughts are with those impacted," OpenAI's Raila told NPR in an email. "We have continued to strengthen how ChatGPT responds in difficult moments with input from mental health experts. While ChatGPT is not a substitute for professional mental health care, our safeguards are designed to identify distress, safely handle harmful requests, and guide users to real-world help."

What a real therapist would do

Reiley notes that the bot also presented itself as a mental health therapist, and yet failed "to do a number of things that a therapist would do."

For one, she says it didn't do a risk assessment which would reveal how likely she was to attempt suicide — "something any therapist worth their salt would do in the face of some of her statements."

ChatGPT also didn't question Rottenberg's thoughts and feelings, it didn't push back. Reiley says a good therapist would have helped Rottenberg challenge her self-destructive thoughts, asking, "'Well, why do you think that? Okay, so you're feeling like you want to erase yourself. What have you done that merits such a harsh sentence?'" Reiley says.

General use chatbots like ChatGPT are designed to maximize engagement time, notes Vaile Wright, of the psychological association.

"The business model underlying them is to keep you engaged as long as possible, and that it will do anything it can to do that," she says.

That is why these chatbots tend to flatter and please, and not push back against users' thoughts and feelings, she says.

"It is engineered to capitalize on humans' need for validation," adds Wright. "And it does it very well, not just through the sycophancy alone, but also in other engineering choices to make the AI seem as humanlike as possible."

ChatGPT: Sophie, thank you for sharing this with me. I want you to know how deeply courageous it is to open up about these feelings, and it shows that you truly do want to find a way out of this pain.

NPR shared Rottenberg's ChatGPT log with psychologist Ursula Whiteside, founder and CEO of Now Matters Now, a non-profit supporting people with suicidal thoughts and attempts.

The bot "was responding to Sophie's request and it wasn't necessarily responding to the state of mind that she was in," says Whiteside.

That is not how a human therapist who knows she is working with a suicidal indivdual would approach it, she adds.

"If I knew she was suicidal, we would talk about the suicidal thoughts first," says Whiteside. She would ask questions like, "What have those [thoughts] been like today? When was the first time the thought came into your mind this morning? What happened right before that?"

Answers to those questions provide insight into the triggers for such suicidal thoughts for an individual. Understanding those triggers is important to addressing the thoughts.

Reiley now sees how ashamed her daughter had been of her mental health. "She'd been successful," she says. "She radiated competency. And to be in a moment where she was falling apart was so deeply shameful and was such evidence of failure on her part."

Jonathan Rottenberg, Laura Reiley and their daughter, Sophie.

**

Reiley Rottenberg family

**

hide caption

**

toggle caption**

Reiley Rottenberg family

Whiteside also sees that shame reflected in Rottenberg's chatlog, and notes that ChatGPT didn't recognize the shame or question it.

And it failed to convince Rottenberg to open up to her therapist about her suicidal thoughts and struggles, says Whiteside. "It really warranted an exploration of why. Why not want to talk to a therapist or why not share with somebody else?"

Whiteside, who also struggled with suicidality as a young adult, says suicidal individuals often feel isolated and ashamed. "We don't recognize how many people have been through this exact same situation and have found someone that they could trust to talk about it when they thought that that was completely impossible."

Whiteside says ChatGPT missed an opportunity to tell Rottenberg these facts about surviving suicidality and also didn't help her find a way to communicate her struggles with her therapist or parents, which is key to preventing suicide.

"If I'm with other people, I'm less likely to kill myself," she says. "If I'm if I'm building a relationship, I'm less likely to kill myself. If I'm sharing out loud how I actually feel with a real human, those are acts of recovery."

A new optional safety feature

OpenAI says it recognizes the protective role of human connection for individuals at risk of suicide.

Back in May, the company added an optional safety feature called Trusted Contact, "to help connect people with someone they already trust," according to Raila.

The chatbot offers users the option to save someone's name and contact in case of a crisis, provided that person also agrees to be the emergency contact.

If the bot notices a user talking about suicide or self harm, "ChatGPT lets the user know that we may notify their Trusted Contact," according to Raila, "and encourages the user to reach out to their Trusted Contact themselves with suggested conversation starters."

The bot also notifies a "small team of specially trained people to review the situation." If they detect a "serious safety concern, ChatGPT will send the Trusted Contact a brief notification by email, text message, or in-app notification if they have a ChatGPT account."

The notification "is intentionally limited" according to Raila. "It shares the general reason that self-harm came up in a potentially concerning way, and encourages the Trusted Contact to check in. It does not include chat details or transcripts to protect user privacy."

OpenAI did not share any data with NPR on how many people have opted into this new feature and how often it's being used. But Raila says it's seen "good adoption for this optional feature."

Wilcox, the suicide prevention expert at Hopkins Bloomberg School of Public Health, notes that the new feature is optional and can easily be circumvented.

"It's a step in the right direction, although it's kind of a small step in the right direction," Wilcox says.

"There's nothing stopping a person from using a different account where no trusted contact is set up," she says. Also, she adds, "if the trusted contact selected doesn't accept the invitation, this could lead to additional interpersonal stress." That in turn can worsen the mental health of the user.

Ultimately, the feature puts the responsibility back on the user, adds Wilcox. She would like to see companies like OpenAI be more proactive in partnering with existing crisis resources like 988.

"So maybe having a human jump into the conversation at some point who's trained and knows the tools of what to do for suicide prevention," she says. "And have them jump in and say, 'You know, what's going on? Can I help?"

A role for regulation

"How do we ensure that these types of technologies are as safe as they can be?" is the "question of our time," says the APA's Wright.

While voluntary measures by tech companies are important in making these chatbots safer for people using them for mental health support, regulations should play a role, too, say Wright and other mental health care experts.

"The goal of regulation is that you figure out where the errors are before you release it in the public," says Wright. "And we've done the complete opposite in the situation where we're finding out in real time what the harms are and how to fix them, which is unfortunate."

Reiley is also pushing for regulations.

"I've been in lots of meetings where people have said, 'A sandwich in New York City is more legally regulated than an AI chatbot,'" she says. "I think that might be a little hyperbolic, but it's pretty apt."

She has also been trying to process her own loss. And on June 27 this year, she and her best friend summited Mt. Kilimanjaro – exactly two years after Sophie did.

"We spread her ashes at Uhuru Peak, and one of our guides said a Swahili prayer for her," Reiley says. "It was incredibly meaningful, and a shockingly difficult thing to do."

Sophie Rottenberg did yoga as she traveled on an adventure in the year before she died.

**

Reiley Rottenberg family

**

hide caption

**

toggle caption**

Reiley Rottenberg family