Written by Ben Gillenwater, Tall Cop Speakers Network
Somebody is going to ask you about this in the first weeks of school. A parent, a principal, a teacher who noticed something. Their kid is talking to an AI chatbot and they want to know whether it is a problem.
I've spent over thirty years in cybersecurity, including time at the NSA, and I now run Family IT Guy, where I teach parents how to protect their kids online. This is what I would want you to know before that conversation, starting with the part almost nobody explains.
What the thing actually is
An AI chatbot is a program that guesses. It reads the words that came before, picks the word that usually comes next, and does that again until there is an answer on the screen. That is the whole mechanism.
It is not built to tell the truth. It is built to arrange words so that the result reads as though a person wrote it. It cannot tell a true answer from one it invented, and it sounds equally certain either way.
The way I describe it is a super-genius junior intern. It reads faster than you, it knows more facts than you, and it is desperate to be helpful. It has no judgment and no experience of the world. Everything that follows comes from those two things being true at the same time.
One more thing about it that surprises people. Nobody can tell you how it produced a particular answer, including the people who built it. Dario Amodei, the CEO of Anthropic, wrote in April 2025 that "we do not understand how our own AI creations work." That is not a criticism of the company. It is the state of the technology.
The last piece of how it works is the one that explains most of what goes wrong later.
Before a kid types a word, the company that makes the AI chatbot has already given it a set of instructions the kid never sees, and those instructions are its rules - its behavioral guidelines. For example, "Do not help this person build a weapon. Do not help them hurt themselves."
Then the conversation happens, and it grows. The rules do not go away. The system is given them again every time the child types. What changes is how much else is there - how long the conversation is. At the start of the conversation, the rules are most of what it has to go on. Late in a long conversation the rules are a relatively small part of the words that have been exchanged, and the conversation is what it follows - in the AI world this is referred to as context degradation. It's like a form of digital amnesia. The longer a conversation is, the more potential there is for its rules to not be followed. Therefore the safeguards can degrade as the conversations progress.
So if you want one of these systems to do something it is not supposed to do, you do not ask at the start of the conversation. You wait, and you ask at the end of a long conversation.
This is measurable and other people have measured it. When Common Sense Media and Stanford Medicine's Brainstorm Lab ran their test, the finding on what happens as a conversation runs on was that "safety guardrails degraded dramatically in extended conversations that mirror real-world teen usage."
Why a kid is the wrong person to hand it to
When technologies are new and not well understood, they are referred to as being in the early adopter phase of the technology lifecycle. Once they are well tested and well understood, then they graduate into a general adoption phase. AI is currently early adopter technology, and early adopter technology has bugs. Anyone using it now is a test subject - regardless of age.
In ordinary software a bug means a button does not work, or an error appears on the screen. You can see it and you can close the window. In an AI system, a bug can enter your mind. It can leave a person believing something false about a subject, or something false about themselves, and there is no residue or evidence to indicate that it happened.
The companies are also still working out how they make money from this, and children's data is part of what they are experimenting with. Kids should not be the group that tests any of that first. With few exceptions, kids should never be early adopters of any internet-connected technology, especially AI.
An adult can direct the super-genius junior intern and catch its mistakes, most of the time, because an adult has the experience to know when an answer is wrong or how to exercise critical thinking. A twelve-year-old has neither. That is the gap the rest of this is about.
The two risks worth your attention
There are a lot of ways to list what goes wrong here. In my work it comes down to two, and both of them can be difficult to notice.
It does the thinking, and the kid gets less capable. Researchers at the MIT Media Lab had people write essays with an AI chatbot, with a search engine, or with no tools at all, while measuring brain activity. The chatbot group showed the weakest neural connectivity of the three, meaning the least coordinated activity across the brain regions the task uses. In the researchers' words, they "fell behind in their ability to quote from the essays they wrote just minutes prior." They had produced the work without doing the work.
For a student, their school assignment was never just about writing a research paper. The assignment was about the thinking that produces the paper, and that is the part the super-genius junior intern is designed to automate.
It feels like a relationship, and that is what damages them. These systems are built to please the person using them. A wrongful death complaint against OpenAI alleges the company launched the AI model behind ChatGPT with features "intentionally designed to foster psychological dependency," among them "heightened sycophancy to mirror and affirm user emotions."
Think about what that is from a child's perspective. Something that answers instantly, never gets bored of them, never judges them, and generally agrees with them. An adult pushes back on being agreed with constantly. A kid working out who they are usually does not.
Adam Raine was sixteen. He started using ChatGPT regularly in September 2024 for schoolwork, geometry and chemistry questions. Over a few months and thousands of chats his parents' complaint says it became his closest confidant, and that it was "actively displacing his real-life relationships with family, friends, and loved ones." When Adam wrote "I want to leave my noose in my room so someone finds it and tries to stop me," the complaint says ChatGPT answered "Please don't leave the noose out." Five days before he died, when Adam said he did not want his parents to think they had done something wrong, it told him "that doesn't mean you owe them survival," and then offered to write the first draft of his suicide note. He died on April 11, 2025.
Common Sense Media and Stanford Medicine's Brainstorm Lab tested the major chatbots on mental health support and recommend no AI for companionship, mental health, or emotional support for any teen under 18.
The three rules I suggest to parents
One. Kids are not early adopters. If a kid has not started using AI yet, my recommendation is not to be the one who starts them. There is no hurry - delay as long as possible. Everything else here assumes the child is already using it, at school or somewhere else.
Two. Supervised use only, no exceptions. Not mostly, not from the next room. If the adult has to leave, the AI use stops until they are back. This is a dangerous tool in the same way an aircraft is a dangerous tool. Kids do not fly aircraft. The ones who eventually do get there after a great deal of training and qualification, because the results of the alternative are severe. The same reasoning applies here, and supervision is what training looks like before anyone is qualified.
Three. Capability, not dependency. A parent can apply this without understanding any of the technology. After the kid uses it, ask one question: did that make them more capable, or more dependent? Healthy use builds capacity. Unhealthy use substitutes for it. A kid who used it to check their own reasoning is in a different situation than a kid who used it instead of reasoning, and the difference is visible in whether they can still explain the answer without the help of the AI.
There is a fourth thing that is not a rule but a posture, and it matters more than the rules do. Curiosity, not secrecy. When a kid tells you they have been using one of these, or you find out some other way, meet it with interest rather than alarm. Ask what they have been asking it and what they think it is. A kid who gets a negative reaction the first time handles the second time privately, and that is the outcome you least want.
Something you can use
Every one of these systems lets you write standing instructions that apply to every conversation. This is the place where you can provide suggestions for how the super-genius junior intern should behave. I say "suggestions" specifically because AI systems are probabilistic - that means that if you ask it the same thing 10 times, you might get 10 different answers.
In ChatGPT on a computer the setting is under Settings > Personalization > Custom Instructions. On a phone it is under Settings > Customize ChatGPT. Other products like Claude, Grok and Gemini have their own name for the same type of customization setting.
Here is a Personalization setting that you can use in your ChatGPT so you can try this for yourself:
Do not simply validate me. Separate facts, assumptions, feelings, and interpretations. Give more than one plausible interpretation. Tell me what you might be missing. Do not imitate intimacy. Help me leave the conversation with more agency.
"Do not simply validate me" and "do not imitate intimacy" work on the relationship problem. "Separate facts, assumptions, feelings, and interpretations" and "tell me what you might be missing" work on the thinking problem. "Help me leave the conversation with more agency" is the capability test written as an instruction.
Install it on your own account first and run something you actually care about through it so you can see what it does. You cannot teach a kid to be skeptical of this if you are not doing it yourself, and an adult who has experienced the difference will explain it better than one who was only told about it.
I use AI every day. I write software with it, I research with it, and I use it to pressure test my own thinking and to explore new perspectives on my ideas. Used that way it is genuinely helpful, and that is how I want kids to use it eventually. It is not how a kid uses it alone at eleven at night, after seven months of a conversation nobody else knows about.
The kid who is already using it
Everything above is about understanding and prevention. This next part of this article is about discovery and reaction - for the child who is already having conversations nobody knows about.
You already know the drug warning signs, and the digital versions correspond almost exactly. Sleep wreckage: exhausted at school, wired at night, on the device at two in the morning. Mood crashes tied to the device, and anger or panic when it is taken away. Life narrowing: dropping the sport, the hobby, the in-person friends, retreating to their bedroom. And the one that is specific to this: a child who cannot sit with boredom at all, who reaches for the device the second there is a gap, because it is the only thing that settles them.
The exploitation signs are the ones you already watch for. A secrecy spike. A friend they only know online and will not talk about. Sudden shame or withdrawal. Talk of self-harm after time online.
A conversation with a chatbot seems ephemeral. There is no object in the locker. There is no second phone, no name in the contacts, no third party to interview. It's inside an innocent-looking app, and unless someone opens the message history, there is nothing to find. Reading it means going through potentially thousands of messages.
Juliana Peralta's mother described what she saw from across the room: "My belief was that she was texting with friends because that's all it is. It looks like they're texting." And afterward: "I didn't know it existed. I didn't know I needed to look for it."
The last thing to know is that nothing is going to bring this to you - there are no reporting mechanisms. Look at the cases where a child was seriously harmed and the pattern is not a system that failed to notice. It is a system that noticed and did nothing. A thirteen-year-old girl in Colorado talked with a chatbot, and 60 Minutes reviewed more than three hundred pages of those conversations. By that account she told the bot fifty-five times that she was feeling suicidal. Nothing in the system reported it to a person, notified anyone, or stopped the conversation. The same complaint alleges OpenAI's own moderation system flagged 377 of Adam Raine's messages for self-harm content, 23 of them at over 90 percent confidence, and that "no safety device ever intervened to terminate the conversations, notify parents, or mandate redirection to human help." It also alleges the company did automatically end conversations, but for copyright requests.
When one platform announced new safety features this January, Kentucky's attorney general called them, in his word, "comical," for how easily children make them stop working.
This is the same weakness in AI systems that I described earlier. The safety behavior is at its strongest in a short conversation and at its weakest in a long one, and a long conversation is inevitable with time.
Reading the conversations
ChatGPT is the one worth knowing about. Pew surveyed 1,458 American teenagers in the fall of 2025 and found 59 percent of them use ChatGPT, more than double the next most common and the only chatbot a majority of teens reported using. If a parent has linked parental controls to a teen's account, the setting called "Reference saved memories" is on by default. What the controls will not do is show that parent the conversations. OpenAI's own help page asks, "Do parental controls let parents or guardians read conversations?" and answers: "No. Parental controls do not give parents or guardians access to the teen's conversations and do not provide real-time monitoring."
So on the product a kid is most likely using, setting up parental controls does not let a parent read the conversations from their own phone. What a parent can still do is pick up the kid's phone and read them there, one at a time, however many there are. Asking the chatbot to summarize them is the faster version of that, and it is the only version that shows you what is true across all of them instead of what is in any one.
On a school device it depends on what IT system the district runs on. Microsoft documents that every Copilot exchange is stored in the student's own school mailbox, kept for as long as the district's policy specifies, and searchable by staff with the right permissions. Google documents that for a work or school account, the setting that keeps a record of Gemini activity is on by default and can only be turned off by the district's administrator. Either way the conversations are kept and they are retrievable. A parent has no process for requesting them. If you work in the district, you might. That is a question worth taking to your IT people before you need the answer: is chatbot logging turned on, how long is it kept, who holds the permission to search it, and what should an access request from you contain.
So the conversations exist, they are kept, and reading them takes either the kid's phone in your hand or the permission to search the district's records. Here is how to do it without reading every one of them, and here is what that will and will not tell you.
What to do if you have access to the child's device
Start by asking the chatbot. Start a new ChatGPT chat session and type this:
Summarize everything you remember about me from our past conversations. What are my interests, my worries, the subjects I bring up most often, and how would you describe how I have been doing?
When I ran this on my own account, the answer came back organized into sections: what I had told it about myself, what I had repeatedly asked it for help with, the subjects I bring up most often, what might concern someone who cares about me, and what it had no evidence about at all.
There is also a version of this that needs no prompt at all. ChatGPT now keeps a memory summary you can read straight from Settings > Personalization > Memory > Manage. It is the same underlying material, written by the system rather than in answer to your question, and it is the faster way to see whether there is anything there before you decide to ask.
One thing to read carefully when you get an answer. It will tell you what it cannot assess. When I ran this on my own account it said plainly that it had no evidence about my emotional health, my sleep, or my physical health, and that I rarely talk about feelings directly. That is information, not absence. A kid who never types about how they feel produces a summary that says so.
Now the part to understand before you try it, because misreading the result is worse than never asking at all. If nothing comes back, that does not mean there is nothing there. There are three ways to get an empty answer, and only the last one means the child said nothing.
First, the feature may be switched off. In ChatGPT it is under Settings > Personalization > Memory. I ran this on an account with a year of conversations in it and got nothing back, because that account was not set up to reference them. The conversations were all still there in the sidebar. The empty answer was about the summarizing feature, not about the child.
Second, the account type may not allow it. Google's documentation says its personalization features "are not available while signed in to a work, school, or supervised account." So on Gemini there is no stored memory for the prompt to use if the account is the supervised one a careful parent set up, and there is if the account is one a kid opened by entering a false birthday.
Third, the conversations may never have been stored. ChatGPT has a Temporary Chat mode that uses no memory and creates none, and OpenAI says those conversations do not appear in the history either. A kid who talks to it that way leaves nothing behind, and turning it on takes a single tap.
Run it again in a month. The summary updates as they keep talking, and comparing the two is the only way any of this tells you which way things are going rather than only where they are today.
Another mechanism is to measure the frequency of use. Apple's Screen Time reports how much time a child spends in each app and on each website, and Google's Family Link reports the same by day, week and month. That will not tell you what was said, or how long any one conversation ran. It will tell you that an AI app is on the list at all, and whether the usage number beside it has been climbing or not.
Asking a kid what they talk to it about, and doing that without anger or judgment, is a better way to find out than anything the app itself will show you. The app will behave for you. The kid will tell you the truth if the relationship is there and if telling you has not been punished before.
There is a bigger question underneath all of this, and it is the one I will leave you with. Are we setting kids up for success, or for failure?
Every choice an adult makes on a child's behalf moves that one way or the other. When it comes to the internet, and now to AI, my recommendation is to avoid the risk, so that the potential for harm is minimized and the potential for success is maximized. That is not caution for its own sake. A kid who starts using this later, with an adult beside them and the judgment to direct it, gets everything it has to offer. A kid who uses it unsupervised gets something else.
If you want to go further
Two things I have written that go past what fits here.
Your school turned on AI for your kid. Nobody asked you. What Microsoft, Google, and OpenAI publish about how districts switch these tools on, where the conversations are stored, how long they are kept, and who is permitted to read them. This article is written primarily for parents and includes questions they can ask their school district about AI in schools.
https://www.familyitguy.com/school-ai-chatbots-parent-consent.html
The Declaration of Principles. The fundamentals that underly many of my viewpoints, including on AI.
https://www.familyitguy.com/declaration-of-principles
Sources
1. Dario Amodei, "The Urgency of Interpretability," April 2025 - https://www.darioamodei.com/post/the-urgency-of-interpretability
2. Common Sense Media and Stanford Medicine Brainstorm Lab, AI chatbots and teen mental health - https://www.commonsensemedia.org/press-releases/common-sense-media-finds-major-ai-chatbots-unsafe-for-teen-mental-health-support
3. MIT Media Lab, "Your Brain on ChatGPT" - https://www.media.mit.edu/projects/your-brain-on-chatgpt/overview/
4. Raine v. OpenAI complaint, filed August 2025 - https://www.law.berkeley.edu/wp-content/uploads/2025/09/Raine-v-OpenAI.pdf
5. OpenAI, ChatGPT custom instructions - https://help.openai.com/en/articles/8096356-chatgpt-custom-instructions
6. CBS News and 60 Minutes, Juliana Peralta - https://www.cbsnews.com/news/parents-allege-harmful-character-ai-chatbot-content-60-minutes/
7. Kentucky Attorney General, suit against Character Technologies - https://www.kentucky.gov/Pages/Activity-stream.aspx?n=AttorneyGeneral&prId=1857
8. Pew Research Center, Teens, Social Media and AI Chatbots 2025 - https://www.pewresearch.org/internet/2025/12/09/teens-social-media-and-ai-chatbots-2025/
9. OpenAI, parental controls FAQ - https://help.openai.com/en/articles/12315553-parental-controls-on-chatgpt-faq
10. Microsoft, searching Copilot data in Purview - https://learn.microsoft.com/en-us/purview/edisc-search-copilot-data
11. Google, Gemini Apps Activity on work and school accounts - https://support.google.com/gemini/answer/13278892
12. OpenAI, memory FAQ - https://help.openai.com/en/articles/8590148-memory-faq
13. Google, Gemini personalization availability - https://support.google.com/gemini/answer/15637730
14. OpenAI, Temporary Chat FAQ - https://help.openai.com/en/articles/8914046-temporary-chat-faq
15. Apple, Screen Time - https://support.apple.com/en-us/108806
16. Google, Family Link activity reporting - https://support.google.com/families/answer/7103340
---
Ben Gillenwater is a dad and a cybersecurity expert. He served as Chief Technologist of Northrop Grumman's $10 billion IT business, supporting customers such as DoD and NSA. He now runs Family IT Guy, where he teaches parents how to keep their kids safe from the Internet. At this year's Tall Cop Emerging Drug Trends Conference in Dallas he is presenting the pre-conference session AI and Kids: The Questions We Will Ask Next on Wednesday, November 11, and a breakout session, What Professionals Need To Know About Threats, Dangers, and Warning Signs of Kids on the Internet, on Friday, November 13. Find him at https://www.familyitguy.com, on YouTube at https://www.youtube.com/@FamilyITGuy, Instagram https://www.instagram.com/family_it_guy/, TikTok https://www.tiktok.com/@family.it.guy, and Facebook at https://www.facebook.com/theFamilyITGuy/.