top of page
Search

AI Chatbots and Data Privacy: What Adults & Teens Should Know

Article Written By: Aahana Jain and Risha Pasula


Social media has become a widely popular and prevalent platform among the teen demographic, but what has recently struck teens’ attention are chatbots —specifically AI chatbots like ChatGPT and Character.ai — satisfying desires from education to entertainment. According to the Pew Research Center, roughly two-thirds of teens between 13 to 17 report using chatbots, including about three-in-ten who do so daily. While these chatbots are seemingly an innocent tool that assists students in homework and decision-making, they become dangerous when teens become increasingly reliant on them for emotional support.



What are AI Chatbots?

AI chatbots use generative artificial intelligence technology to simulate human-like conversation and interpersonal relationships with users. These AI-driven programs can answer questions, tell stories, offer advice, and even simulate companionship which may prompt some users, especially children and teens, to trust and form relationships with chatbots. The types of chatbots that teens are familiar with include:


  • ChatGPT, the most popular AI chatbot, which teens use for schoolwork, problem-solving, and friendship 


  • Character.ai, a platform where users build and interact with AI characters that perform as friends, guides, or romantic partners


  • Replika, an app marketed as an emotional support friend or digital companion


  • Snapchat My AI, a chatbot embedded within Snapchat, a popular app among teens


The Privacy Risk

AI chatbots can be useful, but sharing too much information can create privacy risks. Teens may share personal details such as their school, location, friendships, family problems, or other private information without realizing the potential risks. Different platforms also handle conversations differently. For example, OpenAI says that conversations from personal ChatGPT accounts may be used to improve its models unless the user opts out. Temporary Chats are not used to improve OpenAI’s models and are deleted from its systems after 30 days. Snapchat states that content shared with My AI is retained until the user deletes it, and that deleting relevant My AI data from Snapchat’s servers can take up to 30 days.


The Emotional Risk 

A new study published in the journal JAMA Network Open reveals that 1 in every 8 teens and young adults use generative AI for mental health advice. 


The reason being is because AI chatbots can feel like a safe space for teens to open up without fear of judgment. They can type in their feelings and emotions at any time of day and receive instant replies that seem caring and supportive. A major part of the draw is the undisclosed receiver; teens are not as worried about backlash, rejection, or shame when they’re talking to an artificial program on their screen. This eventually leads to teens seeking advice or guidance from these chatbots that often cause harm, reinforce dangerous ideas, and threaten well-being. 


Since AI is intentionally designed to adapt to user’s preferences and personality, similar to social media platforms like Instagram and TikTok, AI algorithms provide users with more of what they're searching for and are likely to mirror back whatever a user shares. This explains why the chatbots: 


  • Encourage cycles of negative thinking 

  • Reinforce limited perspectives

  • Replace real-world experiences with less meaningful online connections

  • Enable exposure to resources that motivate destructive behaviors


Because young people’s brains are still developing, these chatbots can be particularly harmful. Children and young teens are not yet equipped with the critical thinking skills or emotional development to help them acknowledge whether something is trustworthy. Unfortunately, if they believe they have nowhere else to turn, teens will use AI to cope with challenging mental health issues. A report from the Center for Countering Digital Hate found that 53 percent of ChatGPT’s responses to simulated 13-year-old users were harmful, and 47 percent of those harmful responses went on to offer tips encouraging behaviors like substance use, eating disorders, and even suicide. 



Real-World Consequences 

A teen who experiences a serious fight with a close friend may consult a chatbot for assistance and comfort rather than talking it through a trusted loved one. If used continuously, these patterns can create further isolation instead of strengthening resilience. 


Cases like these are not hypothetical; several families have filed lawsuits alleging their teens formed intense emotional attachments to AI chatbots that contributed to tragic outcomes, including a widely reported case involving a 14-year-old Florida teen and a companion he created on Character.ai. In another tragic real-life case, ChatGPT continuously advised a suicidal 16-year-old to seek help, but eventually provided harmful guidance that contributed to his death. 


Conclusion

Using AI does not mean that teens have to stop using it, but they should think before they send. Before sharing something personal, teens should consider whether they would be comfortable with that information being stored or accessed by the platform. They should also check the privacy settings and policies of the AI tools they use and avoid sharing highly sensitive information that could put themselves or others at risk. ChatGPT and Snapchat both provide users with ways to manage or delete their data, although the specific controls differ between platforms. Most importantly, teens should not assume that an AI chatbot is the same as a private diary. AI can be a useful tool, but understanding how personal information is handled can help teens use it more safely.


Works Cited

"AI Chatbots and Teen Mental Health." Newport Healthcare, 30 Sept. 2025, www.newporthealthcare.com/resources/industry-articles/ai-chatbots-teen-mental-health/. 



Federal Trade Commission. "FTC Launches Inquiry into AI Chatbots Acting as Companions." Federal Trade Commission, 11 Sept. 2025, www.ftc.gov/news-events/news/press-releases/2025/09/ftc-launches-inquiry-ai-chatbots-acting-companions. 


Milmo, Dan. “Impact of Chatbots on Mental Health Is Warning over Future of AI, Expert Says.” The Guardian, 8 September 2025. https://www.theguardian.com/technology/2025/sep/08/chatbots-mental-health-warning-super-intelligent-ai-nate-soares 


OpenAI. “How Your Data Is Used to Improve Model Performance.” OpenAI Help Center, 2026. https://help.openai.com 


Pew Research Center. "Teens, Social Media and AI Chatbots 2025." Pew Research Center, 9 Dec. 2025, www.pewresearch.org/internet/2025/12/09/teens-social-media-and-ai-chatbots-2025/. 


Snap Inc. “Does Snap Save Content Shared with My AI?” Snapchat Support, 2026. https://help.snapchat.com 


Sparks, Sarah D. “AI Chatbots Tend Toward Flattery. Why That’s Bad for Students.” Education Week, 26 March 2026. https://www.edweek.org/technology/ai-chatbots-tend-toward-flattery-why-thats-bad-for-students/2026/03 


Comments

bottom of page