People increasingly use AI chatbots to discuss medical symptoms, financial stress, workplace problems, legal questions and relationship concerns. The conversation may feel private because it appears inside a one-to-one chat window, but a new study warns that consumer AI services are not protected by anything comparable to doctor-patient confidentiality, attorney-client privilege or professional secrecy.
The research compared the consumer versions of five major chatbots: ChatGPT, Google Gemini, Anthropic’s Claude, xAI’s Grok and DeepSeek. It examined how their published policies and interface controls allow providers to reuse conversations for model training, permit human review, support advertising or profiling and share data with vendors or connected services.
The findings do not claim that every conversation is read by an employee or sold directly to an advertiser. Instead, they reveal a gap between the privacy many users assume and the broad rights companies reserve in their policies.
AI Conversations Can Contain Highly Sensitive Information
A traditional web search may contain only a few keywords. An AI conversation can become a detailed personal record developed over dozens of messages.
A user may describe symptoms, upload medical results, paste confidential workplace documents or ask the chatbot to analyze a private argument. Follow-up questions can reveal names, locations, diagnoses, financial problems and emotional vulnerabilities.
The study’s author described chatbots as systems that encourage sustained and intimate disclosure while offering no equivalent of legally protected confidentiality. The research focused specifically on ordinary consumer accounts rather than enterprise products, which may come with stronger contractual and technical protections.
Separate research from Stanford reached a similar conclusion after examining the privacy documentation of Amazon, Anthropic, Google, Meta, Microsoft and OpenAI. Stanford researchers warned that sensitive information entered into a chatbot, including material inside an uploaded file, may be collected and used for training depending on the service and settings.
The Stanford report on AI chatbot privacy advises consumers to think carefully before sharing sensitive information and to opt out of model training where that control is available.
Training on Consumer Conversations Is Now Common
One of the study’s most significant conclusions was that all five providers examined allow consumer chat data to be used for model training by default under at least some ordinary account configurations.
Training allows developers to improve how models answer questions, follow instructions and respond safely. However, it also means that a private-seeming conversation may become part of a broader development process rather than remaining limited to the immediate exchange.
The study found that users can generally opt out, but the controls differ considerably. Some settings are easier to find than others, while certain choices may affect chat history, personalization or data retention.
The researchers also found that feedback buttons can create additional complexity. On at least two services, submitting a thumbs-up or thumbs-down response may allow the associated conversation to be used for evaluation or improvement even when the user previously disabled general training.
This does not mean that every sentence entered into a chatbot will later appear in another user’s answer. Model training does not function like a searchable archive of conversations. Nevertheless, highly specific, identifying or confidential information should not be entered on the assumption that the session will never be reused.
Human Review Is Part of the System
Every provider examined in the five-chatbot study reserved the ability for authorized humans to review at least some conversations.
Human reviewers may examine interactions to investigate abuse, evaluate response quality, improve safety systems or label examples used in model development. Access may be provided to company employees or contractors operating under internal rules and confidentiality agreements.
The study described human review as a structural feature rather than a rare exception. It also found that Gemini was the only service among the five that displayed a direct interface warning telling users not to enter confidential information or anything they would not want a reviewer to see.
Human review does not necessarily mean someone is reading live conversations as they happen. Providers may sample a small proportion of chats or review only interactions flagged for safety, feedback or quality purposes.
The privacy concern comes from the uncertainty. Most users cannot easily determine whether a particular conversation might be selected, who could access it or how long a reviewed copy may remain stored.
Reports involving Meta’s chatbot ecosystem have demonstrated that this possibility is not theoretical. Contractors working on AI evaluation said they encountered real conversations containing names, phone numbers, email addresses and intimate personal disclosures.
Deleting a Chat May Not Remove Every Copy Immediately
Deleting a conversation from the visible chat history does not always mean that every underlying copy disappears at the same moment.
Providers may retain data temporarily for security, fraud detection, legal compliance or system troubleshooting. A conversation previously selected for human review may also follow a separate retention schedule from the copy visible in the user’s account.
The comparative study found substantial variation in retention periods. In one service, agreeing to model improvement could extend backend storage from 30 days to five years, although manually deleting the conversation could override that longer period. It also reported that certain reviewed Gemini conversations may be retained for up to three years after deletion, while some safety-related records may remain longer.
These details can change as companies update their products and privacy policies. Users should therefore check current settings rather than relying on an article, screenshot or instructions written months earlier.
Temporary, incognito or private-chat modes can reduce exposure. These modes generally keep sessions out of normal history, disable training and limit personalization, though they may still permit short-term retention for security and abuse prevention.
AI Conversations Are Beginning to Support Advertising
The study also examined how chatbot interactions can become part of advertising and commercial profiling.
This area differs significantly among providers. Some companies state that they do not sell chat content or use it for personalized advertising. Others operate chatbots inside wider advertising ecosystems where conversation-derived interests may influence recommendations, personalization or ads.
The study noted that advertising entered ChatGPT in the United States through testing for logged-in adult users on the Free and Go plans. It reported that ad personalization could draw on prior chats when the relevant settings and memory features were enabled.
The research did not conclude that advertisers receive complete transcripts. Monetization can occur indirectly, such as by deriving interests or categories from a conversation and using those signals to select advertising.
That distinction matters, but it does not eliminate the privacy concern. A person discussing debt, pregnancy, addiction, health symptoms or family problems may not expect those topics to influence commercial profiling.
Consumers should review both chatbot settings and the broader advertising controls of the company operating the service. For example, a chatbot integrated into a search engine, social network or account ecosystem may participate in data flows that extend beyond the individual conversation.
“We Don’t Sell Your Data” Does Not Mean No One Else Can Access It
Companies often emphasize that they do not sell users’ personal information. That promise can be meaningful, but it does not necessarily mean that chat data stays exclusively inside one isolated system.
Providers may share information with cloud-hosting companies, security vendors, analytics providers, contractors and other processors needed to operate the service. Data may also move between affiliated products when a chatbot is integrated with email, search, social media, maps or other services.
The five-chatbot study found that Gemini and Grok were especially connected to broader service ecosystems. In these arrangements, information entered into a chatbot may interact with additional services or account data, sometimes automatically.
The author’s concern was not that operational sharing is always improper. Modern online services cannot function without infrastructure and specialist providers. The concern was that users are rarely given a simple, understandable view of who may gain access, under what restrictions and for how long.
The complete academic paper, You Trust Your Chatbot With Everything. Should You?, proposes clearer controls and a “sealed mode” designed to prevent training, advertising use and unnecessary human access within one understandable privacy setting.
Memory Features Create a Growing Privacy Risk
AI memory can make a chatbot more useful by allowing it to remember preferences, previous projects and recurring needs. It can also create a long-term profile built from months of conversations.
The study identified memory as an emerging privacy boundary because stored information may influence future responses, personalization, training signals and advertising. It may also create another data repository that could be exposed through a breach, legal demand or account compromise.
Users should review what the chatbot has remembered rather than assuming memory contains only harmless preferences. A system might retain details about employment, family members, health concerns or long-term plans because they appeared relevant in an earlier conversation.
Turning off memory may stop future personalization, but users may also need to delete existing saved memories separately. Removing a chat and removing a saved memory are not necessarily the same action.
How Users Can Reduce Their Exposure
The safest approach is to treat a consumer AI chatbot as an online service rather than a confidential professional adviser.
Names, phone numbers, addresses, account numbers, passwords and government identification details should be removed before submitting text or files. Workplace documents should be anonymized unless the organization has approved that specific AI service and account type.
Temporary-chat modes are preferable for sensitive but non-identifying questions. Training controls, memory settings and advertising preferences should also be reviewed rather than left at their defaults.
Users should avoid uploading complete medical records, legal evidence, unreleased business strategies or documents containing information about other people. Even when a provider offers strong controls, another person’s private data should not be shared without permission.
Organizations handling confidential information should use approved business or enterprise tools with clear contractual protections, access controls and retention terms rather than ordinary consumer accounts.
AI Chats Are Useful, but They Are Not a Private Diary
The study’s central warning is not that people must stop using AI chatbots. These tools can help users understand information, draft documents, organize ideas and explore difficult questions.
The danger comes from assuming that the conversational interface creates confidentiality.
Consumer chats may be stored, used for training, inspected by authorized reviewers, processed by contractors, connected to wider account ecosystems or used to support commercial personalization. The exact combination depends on the provider, account type, settings and features enabled.
The five services offer privacy controls, and some have introduced temporary sessions, opt-outs and clearer data-management tools. Yet the research concludes that those protections remain fragmented and difficult for ordinary users to evaluate.
An AI chatbot can feel like a trusted confidant because it responds instantly and without judgment. Legally and technically, however, it remains a corporate data service. Until providers offer stronger default protections, users should assume that anything entered could be retained or reviewed beyond the immediate conversation.
