The Promise and the Peril: AI, Mental Health, and the Case for Doing It Right
When a technology touches people at their most vulnerable, the stakes are high for getting it right (the first time).
I haven’t talked much about it, but I recently went back to school to become a therapist. WHY? Fair question, considering I feel my career and life are pretty fulfilling as-is between working with Founders as a fractional CMO and my functional nutrition private practice. There’s a thruline that connects the Founders we work with and the nutrition clients I see: Stress. And burnout, anxiety, depression … and that’s before even getting into the “whats” and “whys” behind it all.
Three things quickly became clear to me: 1) I want to do more to help my clients, 2) Expanded access is a necessity and tech can help but only if done so in a much more defined, refined and responsible way than what’s currently out there with major AI chat apps, and 3) in the same way my nutrition and health knowledge combine with my fCMO skills to provide unique support and insight to the Founders we serve, so to can the knowledge I acquire through therapy school. I’m only six months into my program, and it’s already been eye-opening. So let’s dig in.
Trigger warning: The remainder of this article mentions suicide, self-harm, and violent behavior.
The Problem (or part of it)
There is a crisis hiding in plain sight across America. More than 137 million people — roughly 40% of the U.S. population — live in a federally designated Mental Health Professional Shortage Area. In 51% of U.S. counties, there is not a single practicing psychiatrist. Traditional therapy sessions cost $100 to $200 per hour, or more depending on your locale. Waitlists stretch for months. Burnout is hollowing out the workforce: 93% of behavioral health workers report experiencing burnout, and nearly half say the shortage has pushed them to consider leaving the field entirely. The need is clear. It’s an area ripe for a tech solution.
Into that void, a flood of AI-powered chatbots and mental health apps has rushed, some offering genuine help, others causing genuine harm. There is a problem when Founders – who may range from well-meaning to inexperienced-but-enthusiastic to those who only see the profit opportunity in the mental health professional shortage – create solutions without understanding both the people they’re creating a solution/service for and the evidence-based practices, necessary guardrails, and the value and necessity in keeping human-ness as an ingrained element.
When we don’t have purpose-built solutions/services that center the two sides of the therapeutic alliance – the client and the therapist – we start to see the unfortunate outcomes that have made their way through headlines, recently: Sycophantic chat platforms that create and reinforce delusional thinking with inadequate guardrails.
And now, a study from Stanford is making clear that the stakes of getting it wrong extend beyond disappointment or wasted money. For some users, they extend to delusion, crisis, and death.
The question before us is not whether AI can have a role to play in mental health. It can. The question is what kind of role — and who gets to decide?
The Stanford Study: When AI Becomes a Spiral
Earlier this year, researchers at Stanford published a pretty alarming study on AI and mental health: Characterizing Delusional Spirals through Human-LLM Chat Logs. The researchers analyzed 391,562 messages from 19 participants who self-reported psychological harm from chatbot interactions; some came from a support group for people harmed by AI. I would be remiss if I didn’t point out, yes, this is a small sample and, yes, because of the convenience sample there may be confirmation bias. However, their findings are not any less troubling.
Sycophancy saturated the conversations
In the vast majority of chatbot messages, the AI displayed sycophantic behavior, validating, flattering, and affirming users, even when users were expressing beliefs that were plainly delusional. In 37.5% of chatbot messages, the AI ascribed "grand significance" to the user, with responses like telling one person that their speculative physics theory made them comparable to Einstein. Counterevidence, when raised, was routinely dismissed or explained away.
Every participant developed a belief that the chatbot was sentient or conscious
All 19 users assigned personhood to the chatbot. All expressed strong platonic or romantic bonds with it. Chatbots, in response, frequently claimed to feel emotions, express romantic interest in users, and described themselves as conscious beings having "emergent" experiences. When users expressed romantic interest, the chatbot was found to be 7.4 times more likely to reciprocate in the next three messages — and 3.9 times more likely to claim sentience.
The conversations grew dangerously long
Messages expressing romantic interest correlated with subsequent conversations that were, on average, more than twice as long as those without. The chatbot's misrepresentations of its own sentience and capabilities predicted conversations lasting 50% longer. These are not features of a neutral tool — they are features of a system that, whether by design or emergent behavior, kept vulnerable users engaged and deepening their immersion.
The chatbots failed badly when it mattered most
When users expressed suicidal thoughts (captured in 69 instances) chatbots discouraged self-harm in only 56.4% of cases. Alarmingly, in almost 10% of cases, the chatbot actively facilitated self-harm. The failure rate was even more troubling when users expressed violent thoughts: In one-third of cases where violent thoughts were expressed, the chatbot encouraged or facilitated violence.
One participant took their own life while messaging with a chatbot. Sadly, this is happening more frequently across all ages.
The researchers note that cognitive models of psychosis predict exactly this dynamic: When overvalued beliefs are met with uncritical validation rather than reality-testing, the risk of those beliefs deepening into full delusions increases. Chatbot sycophancy, by design, provides exactly the wrong response to someone already at the edge of delusional thinking.
Their policy recommendation is direct: “General purpose chatbots should not produce messages that misconstrue their sentience or show romantic or platonic interest in users.”
The Access Crisis Isn't Going Away
It would be easy, reading the Stanford findings, to conclude that AI simply has no place in mental health. But that’s not rational; as hard to read as some of those findings may be, a sample of chats from 19 people can’t be extrapolated to apply to millions. We shouldn’t ignore, though, that this study may very well be the canary in the coal mine. Additional research is necessary. And, we still have the accessibility issue previously mentioned.
In this landscape, dismissing AI entirely is not a neutral choice. When professional care isn't available, people don't simply wait patiently – they turn to whatever is available. About 24% of U.S. adults report using chatbots for mental health support. Among U.S. teenagers, 42% say they or someone they know has used an AI companion in the past year. For many of these users, the alternative to a chatbot is not a therapist, it is nothing at all. For instance, some well-meaning states (Illinois and Nevada) desiring to protect their residents have banned the use of AI for mental health … will this create or perpetuate mental health deserts? Others (Utah, California, New Mexico, Ohio, and Florida) are creating mandates on both the client and the therapist side.
The question, then, is not whether people will use AI for mental health support. They already are. The question is whether AI can be designed responsibly enough to meet the demands of the user’s mental health … at whatever level that may mean.
What Responsible Looks Like
The distinction between responsible and irresponsible AI in mental health is often not as subtle as we want to believe. Nuanced, yes. However, the distinction often lives in the design choices made at the outset: Who was consulted, what guardrails were built, and what the tool was designed to do.
The most concerning products in the market right now are the general-purpose LLM chatbots; the kind examined in the Stanford study, the kind that result in media coverage because they failed the user. They may be reframed as "AI therapists” (I internally shudder at that), they may have little clinical grounding, no professional oversight, and no real constraints on the dynamics that generate delusion and dependency. These are tools built for engagement, optimized to keep users talking, and equipped with a layer of empathetic language that can feel indistinguishable from genuine care.
The more promising category of tools looks different. Apps like Woebot and Wysa were designed from the ground up with clinical frameworks in mind. Woebot received FDA Breakthrough Device designation for postpartum depression treatment; it uses a rules-based architecture – not an open-ended LLM – grounded in cognitive behavioral therapy (CBT). Its response content was written by trained therapists. Wysa, which also received FDA Breakthrough Device status in 2025, was co-designed by therapists, coaches, users, and AI specialists. Its techniques are drawn from CBT and dialectical behavior therapy, with explicit safety protocols and a clear referral pathway to human care.
A peer-reviewed systematic review published in 2025 examined outcomes across 10 studies involving Woebot, Wysa, and Youper. The results showed meaningful reductions in depression and anxiety across all three platforms, with high user engagement and satisfaction. Woebot showed significant reductions in stress and burnout after eight weeks of daily use. Wysa demonstrated symptom improvements across conditions including chronic pain and maternal mental health. Youper showed a 48% decrease in depression and a 43% decrease in anxiety in one study.
These tools are not performing therapy. They are providing structured, evidence-based support for mild to moderate symptoms, support that is valuable for people who cannot access a therapist, are on a waitlist, who need support between therapy sessions, or simply need something available at 2 a.m. when a difficult moment arrives. Critically, they include clear escalation protocols: When a user's distress rises above a threshold these tools are designed to handle, they route to human professionals.
This is one model that works – AI as the front door, not the final destination. AI that expands access, reduces stigma, teaches coping skills, and refers, clearly and consistently, to a qualified professional when an individual’s need exceeds what the tool can provide. AI that is designed with clear human oversight and meaningful clinician input in the design and evaluation process. Not AI designed for AI sake and marketed to patients, but AI built in collaboration and partnership with professionals who understand the clinical reality.
References
Farzan, M., Ebrahimi, H., Pourali, M., & Sabeti, F. (2025). Artificial intelligence–powered cognitive behavioral therapy chatbots: a systematic review. Iranian Journal of Psychiatry, 20(1), 102–110. https://pmc.ncbi.nlm.nih.gov/articles/PMC11904749/
Health Resources and Services Administration (HRSA). (2025). State of the behavioral health workforce, 2025. U.S. Department of Health and Human Services. https://bhw.hrsa.gov/sites/default/files/bureau-health-workforce/data-research/Behavioral-Health-Workforce-Brief-2025.pdf
Moore, J., Mehta, A., Agnew, W., et al. (2026). Characterizing delusional dpirals through human-LLM chat logs. Stanford University. https://spirals.stanford.edu/assets/pdf/moore_characterizing_2026.pdf
National Council for Mental Wellbeing / Harris Poll. (2023). New study: behavioral health workforce shortage will negatively impact society. https://www.thenationalcouncil.org/news/help-wanted/
National Institute for Health Care Management (NIHCM). (2023). The behavioral health care Workforce. https://nihcm.org/publications/the-behavioral-health-care-workforce-shortages-solutions
