What AI Systems Infer About Users Beyond Stated Data

AI chatbots don't need users to say they're depressed, gay, pregnant, or in debt. They infer it, quietly, from word choice and timing and questions people think are unrelated to anything personal. This piece maps how that inference actually works, what data it runs on, and why the gap between what a person types and what a model learns has become one of the least understood risks in consumer AI.

Why AI profiling at scale is no longer a niche concern

ChatGPT crossed 1 billion weekly active users in August 2026, making it the largest standalone AI chatbot by that measure. An AI assistant integrated into several messaging and social apps reports 1.2 billion monthly users across those apps combined. Roughly half of the country's. adults, 49%, now use an AI chatbot at all, and about a quarter use one daily. Separately, 12% of U.S. adults use one several times a day and 4% describe their use as almost constant.

Scale like that changes what a "conversation" actually is. A person asking a chatbot about a strange rash, or venting about a bad month at work, or working through a budgeting problem, isn't having a private moment the way they might with a friend. They're generating a data point that a model processes, weighs, and in many cases retains. Most public concern about AI privacy still centers on the obvious stuff: what happens to the resume someone uploads, or the medical details they type in full. But that's only half the exposure. What a system figures out on its own, from information nobody thought they were disclosing in the first place, is the other exposure.

How AI inference differs from ordinary data collection

Privacy law, as it's mostly built today, governs collection, storage, and transfer: the stuff a company knowingly gathers because a user handed it over. Cookie consent banners, data breach notification laws, the right to request a copy of your file. All of that assumes there's a specific fact somewhere sitting in a database, waiting to be protected or exposed.

AI systems break that assumption. A large language model doesn't need a user's income entered into a form field. It can guess it, from sentence structure, from the brands someone mentions, from how they phrase a complaint about rent. This works through what researchers call proxy learning: the model learns that certain surface features, word choice, topic selection, even response timing, correlate with attributes the user never stated. Nobody has to program the model to hunt for income or age. If those traits appear as statistical patterns anywhere in its training data, the model absorbs the correlation the same way it absorbs grammar or sentence rhythm. It doesn't distinguish between "this pattern predicts sentence structure" and "this pattern predicts sexual orientation." A correlation is a correlation. It means the system never tracks the sensitivity of an inference. The model isn't hiding what it knows. It doesn't know that what it knows is sensitive.

Demographics, personality, and political orientation inferred from text

Start with the plainest case: age, gender, location, education level, income bracket. Researchers have shown that research has shown that brief, ordinary text samples fed into commercial LLMs can be enough to recover all five of those attributes with notable accuracy. That's an off-the-shelf capability sitting inside models people query every day for unrelated tasks, not a lab curiosity requiring custom tooling. It's an off-the-shelf capability sitting inside models people query every day for unrelated tasks.

The signal runs deeper than word choice, too. Research has found that demographic stereotypes can attach to surface-level cues in a prompt, shaping model outputs in predictable, patterned ways even when the substantive content is unchanged.

Personality inference follows a similar arc. A 2026 study using real conversational agent data from 668 users found personality trait inference performing around 60% better than a random guess, depending on the model and the data source feeding it. That echoes older but foundational work by Kosinski and colleagues, who showed that mundane digital signals predict personality, sexual orientation, and political leaning with striking accuracy. Since then, the same pattern has repeatedly shown up in the literature: emoji usage carries more signal about private traits than users assume, gaming telemetry carries more signal than users assume, the shape of someone's social graph carries more signal than users assume, and even the star ratings they leave on products carry more signal than users assume.

Political orientation might be the sharpest case, because it's inferred not from anything a user says about politics but from what researchers call "kitchen sink" data, ordinary, conceptually unrelated information that just happens to correlate with the views of people who did disclose their politics somewhere. That's a strange kind of exposure. Political identity is fluid and often contested even by the person holding it, which makes a silent, third-party inference about it almost impossible to contest or correct. This isn't hypothetical: X's Community Notes system, studied empirically across 13 countries on five continents, works by inferring each rater's ideological position from their past rating behavior. That's the mechanism the system needs in order to function. It's the mechanism the system needs in order to function. Meta, YouTube, and TikTok have each announced or built their own versions of a bridging-consensus system, meaning this kind of systematic political inference is now standard across the industry rather than one platform's experiment. It's becoming standard infrastructure across the industry.

Sexual orientation, mental health, and health conditions inferred from behavior

LGBTQ+ identity sits in a particularly uncomfortable spot on this map. Systems can infer it from social connections, location history, search queries, linguistic patterns, and engagement with certain kinds of content, building a profile a user has no way of knowing exists. GLAAD's 2026 AI Report puts it bluntly: modern models are smart enough to guess a user's sexual orientation or gender identity just from behavioral patterns and proxy data, no explicit disclosure required.

That's not an abstract concern. Human Rights Watch has documented governments using digital targeting, fake online profiles, social media monitoring, device searches, to identify and target LGBTQIA+ people in countries where same-sex relationships are criminalized. Inference here is a safety problem, and in some places, a life-or-death one. It's a safety problem, and in some places, a life-or-death one.

This claim isn't uncontested. Some researchers argue that predicting sexual orientation from behavioral or facial signals is fundamentally impossible as a matter of science, even when a pattern-matching system confidently produces an answer. A model outputting a guess isn't the same thing as that guess being a valid measurement of anything real. The controversy just means the outputs deserve more skepticism than they usually get. It just means the outputs deserve more skepticism than they usually get.

Mental health inference runs through a different but related pipeline. In teletherapy and virtual consultations, AI systems can combine facial micro-expressions, vocal tone, and the actual content of speech to estimate emotional state in real time. An umbrella review covering 29 systematic reviews, scoping reviews, and meta-analyses published between 2013 and 2025 flags data privacy, algorithmic bias, and user trust as recurring, unresolved concerns across this entire research area. Separately, AI frameworks built to flag mental health risk have been shown to pull from electronic health records, brain imaging, speech patterns, and social media activity, stitching together a picture from sources that were never meant to talk to each other.

Physical health and financial vulnerability follow the same logic. Researchers have noted that AI can identify individuals from travel records, social media activity, X-rays, ECGs, MRIs, even gait, or from almost any combination of about three ordinary transactions. A recommendation algorithm doesn't need a diagnosis typed into a search bar to flag a health condition; browsing behavior alone can do it. A pricing model can read financial vulnerability off cart behavior, no income disclosure needed. And anonymized or aggregated data offers less cover than most people assume. Machine learning models have repeatedly demonstrated the ability to re-derive sensitive attributes from data that was supposedly stripped of anything identifying.

Inference from images, video, and non-text signals

Faces carry more signal than most people would guess. Facial recognition systems have been used to predict emotion, age, gender, ethnicity, personality, political orientation, and sexual orientation, traits that feel, intuitively, like they shouldn't be readable off a photograph. Patents filed by companies ranging from small startups to Xerox demonstrate working systems built to infer exactly these kinds of traits from facial data.

Kosinski's research includes a claim that gets cited often and disputed just as often: that a single facial photo can be as informative about political orientation as a full 100-item personality questionnaire. Other researchers push back hard on this, arguing that what these models actually detect isn't some stable underlying trait but stylistic correlates, haircut, grooming, glasses, expression habits, that happen to track political affiliation in a given sample without reflecting anything essential about the person. Both things can be true at once: the correlation appears reliably in the data, and the causal story behind it remains genuinely unsettled.

Video adds another layer. A 2025 study titled "Through Their Eyes" examined how users themselves perceive the ability of visual language models to infer sensitive traits from social media videos, an early look at a gap between what people think a video reveals and what a model actually extracts from it. Gesture recognition systems, separately, have been shown to read body movement for traits like extraversion, openness, and agreeableness. Physiological signals, heart rate variability among them, have been linked in the literature to inferred emotional states through machine learning models trained on that kind of biometric data.

Even something as far removed from personal disclosure as a music-streaming playlist carries risk. Research under the heading "From Beats to Breaches" found that music and playlist data alone can reveal sensitive personal information, a domain almost no one associates with profiling. The pattern across all of this is the same: any behavioral trace a model can access, whether it's a search query, a workout heart rate, or a weekend-night playlist, functions as a proxy signal for something the user never chose to share.

What happens to conversation data once it leaves the chat window

A systematic measurement of web tracking conducted at UC Davis examined 20 popular AI chatbots and found that 17 of them share information with at least one third party during a single, ordinary chat session. Three chatbots were found sharing plaintext conversation content, both the user's prompt and the model's response, with Microsoft Clarity through session replay tooling. Fifteen shared conversation URLs or chat identifiers with third-party advertising, analytics, or social media endpoints. Several exposed user identity directly: names, email addresses, or hashed emails, leaking out through embedded support widgets, analytics scripts, advertising pixels, and session replay tags.

This matters more than typical website tracking for a fairly specific reason. People tell chatbots about health scares, relationship problems, financial stress, and mental state in ways they rarely type into a search engine's search bar. And when chatbot use happens while logged in, tracking can attach to account-level identity rather than the anonymous browser fingerprint that drives most web ad tracking. Research on chatbot interactions has documented that users regularly volunteer health, financial, professional, and emotional detail in ways that go well beyond what most people assume they are disclosing. A Cisco benchmark study found 64% of respondents worry about accidentally sharing sensitive information with generative AI tools. That worry is reasonable, but it's aimed at the wrong half of the problem. What a person chooses not to type is only one layer of exposure; what the system infers anyway, and what happens to that inference once it leaves the chat window, sits entirely outside anyone's typing decisions.

How commercial advertising systems operationalize sensitive inference

Inference doesn't stay theoretical for long. It gets sold. An investigation by AI Forensics found that X's advertising system let advertisers target and exclude users based on sensitive personal categories, including political opinion, sexual orientation, religious belief, and health condition. The investigation identified more than 30 major brands engaged in this kind of targeting.

Specific cases from that investigation are: TotalEnergies excluded users interested in Green party politicians and environmental activists from certain ad campaigns. TotalEnergies excluded users interested in Green party politicians and environmental activists from certain ad campaigns. Other brands ran targeting based on specific medications, sexual orientation, and religious faith. Nine NGOs have since filed formal complaints with national Digital Services Coordinators over these practices.

X's own gender-targeting system illustrates the mechanism at a technical level. It uses gender data from users who disclosed their gender directly to infer the gender of users who never did, based on account similarity, and the platform reports roughly 90% accuracy for that inferred signal. That's a genuinely high hit rate for a trait nobody agreed to share.

This explicit ad-targeting apparatus is really just one visible end of a much longer spectrum. At the far end, recommendation algorithms and conversational bots encode ideology and identity into user profiles silently, as a byproduct of chasing engagement or just processing text efficiently, with no advertiser ever placing an order for that specific inference. Advances in AI explainability have started to make this spectrum harder to deny: researchers can now identify which internal variables inside these models correspond to political opinion or other protected categories. That closes off the easiest defense, the idea that this kind of profiling happens by accident, or without anyone's awareness that it's occurring.

Why users cannot reliably protect themselves through careful phrasing

The instinct to protect yourself by rewording a sensitive message doesn't hold up well under testing. A 2026 CHI Conference paper, "Beyond PII" by Wang and colleagues, ran a study with 240 participants from the same country. participants who were shown text snippets and asked to judge inference risk, rate their own concern, and try rewriting the text to block whatever the model might infer from it.

The results aren't encouraging. Participants struggled to predict what a model could infer in the first place, performing only marginally better than random guessing. When they did attempt rewrites, those rewrites succeeded in blocking inference only 28% of the time. That's better than the automated sanitization tool Rescriber managed on its own, but worse than what ChatGPT produced when asked to do the same rewriting job itself. The most commonly used strategy, plain paraphrasing, was also the least effective one. Techniques like abstraction and deliberately adding ambiguity worked better but weren't the ones people reached for instinctively.

One behavioral detail from that study stands out. Participants treated the task as editing, staying close to the original sentence, preserving its coherence and overall meaning, rather than starting over from a blank page. That habit is exactly why the rewrites kept failing: shaving off a few identifying words doesn't remove the deeper pattern a model is actually keying on. People consistently underestimated how much signal survives light editing.

A separate 2026 research project, described in the paper "When Are LLM Inferences Acceptable?", built something called the Reflective Layer: a visualization tool that shows users the inferences a model has drawn from their own ChatGPT history, sorted by whether each one is a stated fact or a confidence-weighted guess. Most participants shown this tool had no idea these inferences existed until the interface put them in front of their own eyes. That gap, between what a system has quietly concluded and what a user believes they've revealed, is the throughline of the entire piece. Careful phrasing can't close a gap that the user can't even see.

Where regulation currently draws lines

Most existing privacy law was built around the idea of a fact sitting in a file: a Social Security number, a home address, a diagnosis code. Rules like consent requirements, breach notifications, and data access rights all assume there's a specific, identifiable piece of information to protect. Inference breaks that model at its foundation, because the sensitive attribute in question was never stored anywhere as a fact. It was derived, computed fresh from ordinary signals that, individually, look like nothing.

That leaves a real question hanging over every example in this piece. If a model never collects someone's sexual orientation but reliably guesses it anyway from behavioral proxies, has any data protection law actually been triggered? Frameworks built around explicit categories of sensitive data, health records, religious belief, political opinion, tend to falter when the attribute in question was inferred rather than disclosed, because enforcement usually depends on proving that a specific piece of protected data was collected or processed. Proving that a model inferred something is a different, much harder, and much less settled kind of claim.

That gap between what regulation was built to catch and what modern inference actually does remains unresolved. The mechanics laid out across this piece, demographic inference from a single sentence, political profiling built into the plumbing of content moderation systems, health status leaking out of ordinary browsing behavior, show a pattern that current rules weren't designed around. Whether the next generation of privacy law catches up to that pattern, or keeps chasing the older, simpler idea of data as something a person hands over on purpose, remains an open question. The answer will decide who gets to know what about someone before that someone has said a word.