Mental Health AI Apps and the Sensitivity of Therapy Conversation Data

Mental health AI apps collect the most sensitive disclosures a person can make: depression, trauma, suicidal ideation. These apps operate almost entirely outside the confidentiality rules that bind a licensed therapist, and most people who type into them at two in the morning have no idea that gap exists.

What people actually disclose to mental health AI apps, and why those disclosures are unlike any other data

Picture the moment someone downloads one of these apps. It's rarely a calm, deliberate decision made after reading reviews side by side. It happens at 2 a.m., after a bad day, before a panic attack fully takes hold, or in the silence after a fight with a partner. In that moment, the app competes with doing nothing at all, and so it gets access to material that a person might not tell a spouse, a doctor, or a close friend. A 2026 CHI study of 20 UK adults living with mental health conditions found that people chose large language models for support specifically because of non-judgment, self-paced disclosure, immediacy, cognitive reframing, and relational engagement. No therapist's office is open at 2 a.m., no friend wants to hear the same spiral for the fourth time that week, and a chat window doesn't flinch.

But the same qualities that make the disclosure feel safe are what make the data so dangerous once it leaves the conversation. Depression, trauma narratives, suicidal ideation, and addiction histories are not like a shopping history or a location trail. Exposure of a browsing history embarrasses. Exposure of a trauma narrative can end a career, unravel a custody case, or hand an abuser leverage over a victim who thought that story was safe. The stigma attached to mental illness means even the bare fact that someone uses one of these apps carries risk. If an employer, insurer, or estranged family member learns that a person downloaded a depression-support app, that single fact discloses a psychological struggle the person may never have chosen to share with them. Suicidal ideation and addiction disclosures raise the stakes further still, because in the wrong hands this data converts into physical danger, not just embarrassment.

Why users reasonably assume their conversations are protected, and why that assumption is wrong

Mental health apps tend to borrow the visual and verbal language of therapy. Soft color palettes, words like "confidential," "private," and "safe space" sit throughout onboarding screens, and the entire design borrows the emotional posture of a therapist's office. None of that language carries the legal weight it implies. If a licensed therapist discloses a patient's conversation without authorization, they risk losing a license, face civil liability, and in some cases face criminal charges. No such consequence attaches to the team that built the app on someone's phone.

The reason is structural: a design gap, not bad actors slipping through the cracks. HIPAA protects health information collected during clinical encounters, but most direct-to-consumer mental health and AI companion apps are not clinical encounters in the legal sense and don't qualify as HIPAA covered entities. What governs them instead is the FTC Act, assorted state consumer protection statutes, and general product liability law, none of which were written with psychotherapy-grade disclosures in mind. That mismatch becomes concrete at the moment of first contact with the app: nearly half the apps in the 2026 study asked mental health screening questions before a user even created an account, so the most sensitive disclosure of all, an admission of depression or anxiety severity, can happen before anyone has seen a privacy policy, let alone read and understood one.

What these apps actually collect, conversation content, behavioral signals, and device access

Conversation transcripts are only the most visible layer of what gets collected. A hidden second layer, behavioral signals that most users never see, reveals just as much as the conversation content itself. How often someone opens the app, how long a session runs, which features get used at 3 a.m. versus noon, all of that produces a usage pattern that, on its own, can tell an outside party that a person is likely struggling with depression, anxiety, trauma, addiction, or loneliness, without that party ever reading a single message. A 2026 study of 25 popular Android mental health apps found that every one of the 25 contained at least one tracker that was never disclosed in the app's privacy policy. One app alone embedded 20 separate trackers while disclosing none of them.

Device permissions add a third layer. Most of the apps examined in that study requested camera access, microphone access, or both, and researchers found cases where these "dangerous permissions," the industry's own classification for access this sensitive, were requested with no disclosure in the privacy policy and no clear functional reason the app would need to record audio or capture images. Onboarding compounds the exposure further: around four in five of the accessible apps required an email address before setup even began, tying a real identity to the account from the first screen, and, again, nearly half asked mental health screening questions before that account existed. Once data like this is collected, only a minority of apps offer deletion. Only a minority of the apps studied offered any in-app deletion tool. Most required a user to send an email request into some unknown queue, and two apps offered no deletion mechanism of any kind. Looked at together, conversation content, behavioral inference, device permissions, and onboarding data form a single picture rather than four separate problems: an app can know what someone said, when they said it, what their face and voice sound like while saying it, and who they are by name, often before that person has read a word about how any of it will be used.

Where the Data Goes

Collection is only the first half of the exposure. The second half is where that data travels afterward, and the clearest documented example of the full chain is BetterHelp's 2023 FTC action. BetterHelp had collected sensitive health information through intake questionnaires, the kind of material a person fills out expecting it to inform their therapy match, and shared that data with advertising platforms including Facebook and Snapchat. The FTC ordered the company to pay millions of dollars in consumer redress. That case matters as a template: intake data meant for a therapeutic purpose ended up inside an advertising pipeline built for something else.

Other apps extend that same logic further. Rosebud disclosed that datasets and trained models derived from user conversations could be commercialized or licensed to outside organizations. A user's account of a specific trauma can become a component of a product sold to a party the user never agreed to deal with. Once a disclosure like that is folded into an AI training dataset or baked into a model's weights, removing it after the fact may not be technically possible, even if the user later deletes their account. Deletion of an account record doesn't guarantee deletion of what that account already taught a model. The data broker market is simply the broader version of the same BetterHelp pattern: sensitive psychological data, once it leaves the app that collected it, can be resold, relicensed, or repurposed by parties the original user never knew existed and never consented to in any meaningful sense.

The security vulnerabilities that make the collection and routing problems worse

Even an app with no intention of sharing data with advertisers or brokers still has to secure that data against people trying to take it without permission, and many of these apps fail at that task on purely technical grounds. High download counts and strong app store ratings tell you nothing about the quality of the underlying security architecture. Some of the apps with large user bases and favorable reviews turned out to carry severe technical vulnerabilities, the kind that let an attacker pull data the company never intended to share with anyone.

This compounds the policy failures. A user's exposure runs along two separate tracks at once: what the company chooses to share deliberately, with advertisers or AI partners, and what an outside attacker can extract regardless of the company's intentions. Fixing one track does nothing for the other. An app could stop selling data to advertisers tomorrow and remain just as vulnerable to the kind of breach that doesn't ask permission.

What Real Harm Looks Like

The worst-case scenario already happened, at a psychotherapy provider, in 2020. A criminal broke into the patient database of psychotherapy provider Vaastamo, demanded ransom, was refused, and released the therapy session notes of more than 33,000 patients. These were session notes, the specific content of what people had told their therapists in confidence, now public. Many of the patients affected were severely traumatized by the exposure itself, independent of whatever brought them to therapy in the first place, and some died by suicide in the aftermath. Vaastamo sets the floor for how bad this can get when therapy data escapes its intended boundary.

Contemporary cases show how the same forces operate at a smaller scale and under regulatory scrutiny rather than criminal breach. The fine and investigation involving Replika matter because they mark a shift in what regulators are willing to examine. Earlier enforcement, like the BetterHelp case, focused on how companies handled data they already had. Scrutiny of Replika asks whether using a person's disclosures to train a model is itself the violation, whether or not that data was ever sold or leaked to a third party. That is a meaningfully different question, and it signals that regulators are starting to treat the training process itself as a point of exposure, not just the storage and sharing of raw conversation logs.

The Regulatory Gap: Why HIPAA Does Not Apply

The reason these failures recur is structural. The United States has no federal privacy framework that imposes therapist-equivalent confidentiality obligations on mental health AI apps. HIPAA covers health information gathered in clinical encounters, and a clinical encounter has a specific legal meaning that most direct-to-consumer wellness and AI companion apps simply don't meet. What applies to those apps instead is the FTC Act, a patchwork of state unfair-and-deceptive-practices statutes, newer state AI laws, and ordinary product liability theories, none of which were built around the particular sensitivity of a therapy disclosure.

The FTC's own role illustrates the limit clearly. The agency can act once harm has already occurred, as it did with BetterHelp, but it has no authority to impose the kind of prospective confidentiality regime that governs a licensed psychologist or social worker before any harm happens. States have begun moving into that space, unevenly. A proposed North Carolina bill would require a 30-day data self-destruction timeline for AI chat data across healthcare, financial services, legal, government, mental health support, and education sectors, paired with stronger encryption requirements and limits on data reuse. A 2025 50-state legislative review published in JMIR Mental Health found that transparency mandates built specifically for mental health AI remain uncommon, and that the lack of specialized protection falls hardest on non-HIPAA-covered entities, the exact app developers this piece has been describing. Illinois, Nevada, Utah, and California each passed materially different laws in 2025 and 2026, and the resulting patchwork creates compliance complexity that smaller companies may struggle to manage, which could push the market toward larger incumbents with the legal staff to handle fifty different sets of rules. None of this amounts to a closed gap. It amounts to evidence that lawmakers recognize the gap exists, at an early and inconsistent stage of response.

The Access Argument

One might argue that strict new privacy rules would simply raise costs and slow down a category of tools that fills a genuine, urgent need. Therapist shortages are severe, and stigma keeps plenty of people away from professional care entirely, so an app that offers immediate, judgment-free support at low cost is solving a real problem, not a manufactured one. That argument deserves to be taken seriously rather than waved off, because it's true as far as it goes.

But what if the privacy failures described in the previous sections are themselves undermining the access they're supposed to protect? The 2026 CHI study found that people used LLMs for mental health support because of self-paced disclosure, immediacy, and a sense of non-judgment, and the study's authors also noted that a user's sense of control over pacing may be something of an illusion. That benefit evaporates the moment a user learns their disclosures were routed to an advertiser or a data broker. The access case is strongest exactly where the stakes are highest, in communities with the fewest therapists and the most stigma around seeking help, and those same users are the least likely to have read, let alone understood, a terms-of-service document before they start typing. That makes structural protection more necessary in precisely the populations the access argument is trying to serve, not less. Access and privacy are aligned, since a tool nobody trusts with their secrets is a tool people stop using at the moment they need it most.

What meaningful protection would actually require

A better privacy policy is not the fix. Enforcement after the fact, the way the FTC pursued BetterHelp, is not the fix either, since by the time an enforcement action lands, the disclosures are already out and, in some cases, already baked into a model's weights. Real protection has to start at the architecture level, before a single conversation happens, with systems built so that sensitive disclosures are never collected in forms that can leak, get resold, or train a commercial model without specific, informed consent tied to that exact use.

That distinction, between an app that adds a privacy policy on top of an existing data pipeline and one that is built from the ground up so sensitive data never flows somewhere it shouldn't, is not just a theoretical nicety. That distinction appears in concrete, checkable facts: whether trackers are disclosed, whether deletion actually removes data from training sets, and whether a mental health screening question appears before or after a user has seen a privacy policy. The emerging certification model, the kind the PMC analysis points toward, offers a path to make that distinction visible to an ordinary user who has no technical ability to inspect an app's backend. An independent audit that verifies architecture rather than merely checking a box on a policy document gives users something they currently have no way to get on their own: a way to tell, before they type a single word about their own suffering, whether the app on the other end was actually built to keep that word safe.