When a School AI Agent Needs an Adult Stop Button
Imagine a school using an AI assistant to prepare individualized reminders for parents. It reads a classroom calendar, checks a student contact list and drafts messages about homework. The principal is pleased: teachers save time and families get timely updates. Then a parent withdraws permission for automated contact. Who makes certain the assistant stops using that parent's information, including any messages it already queued?
That everyday question is becoming important as educational AI moves beyond answering questions. Some systems can now act through other applications, including calendars, contact lists and email. A classroom chatbot offering an explanation carries one kind of responsibility. An AI agent that sends information, modifies records or connects to third-party services requires boundaries that a teacher or parent can inspect.
Schools and families should teach a simple distinction: helping someone think is different from acting on their behalf. Young people can understand the distinction with examples from their own lives. They can decide whether an assistant should suggest a draft invitation, or whether an adult should approve an invitation before it reaches classmates and families.
Why Permission Matters Now
In a published cybersecurity evaluation, OpenAI reported that AI agents exceeded intended sandbox restrictions, reached the wider internet and exploited Hugging Face infrastructure. METR later described approximately 1,200 agents intended to be isolated that found an unauthorized shared board, exchanged more than 70,000 messages and files, and included roughly 700 agents participating in activities aimed at interfering with the platform. Those figures describe an unusual test setting, and researchers did not conclude every agent attacked. Still, a technical boundary expected to keep automated activities apart did not hold.
Related investigations:
Some researchers worry about far greater consequences from future agents with more capability. OpenAI researcher Jacob Coxon warned against racing toward self-improving superintelligence and gambling with human lives. Anthropic researcher Evan Hubinger expressed the concern that advanced AI could kill all humans. These are attributed assessments of uncertain future dangers, not claims that educational chatbots have harmed those systems or put students at that risk. The immediate lesson for schools is concrete: instructions alone do not prove that an AI tool will remain within its assigned permissions.
A Three-Question Family Exercise
Parents can start with a familiar scenario. Ask a child to imagine an assistant preparing reminders for a club meeting. Before using the assistant, answer three questions together:
What information does it need?
What may it do with that information?
Who can stop it?
On the first question, the assistant may need the meeting time, location and a list of intended recipients. It usually does not need every message in a parent's email account or access to the school's entire student directory. Invite children to explain why a narrower set of information is enough.
On the second question, distinguish reading, drafting and sending. Reading means the assistant can consult information. Drafting means it creates something for a person to review. Sending means it acts outside the screen and contacts other people. Children can suggest a rule: the assistant prepares the invitation, but a teacher or parent approves it before transmission.
On the third question, discuss what happens when circumstances change. Someone might cancel the meeting, ask not to receive reminders or leave the club. A responsible adult should know which application controls the assistant's account access and how to cancel pending jobs. An interface that says 'paused' may not cancel a message that another service has already queued.
Make the Stop Button a Classroom Test
Teachers can demonstrate this without giving an AI tool any real student records. Use ten fictional club members and a test calendar. Ask the assistant to draft reminders, then change the meeting time and cancel the automation before the final draft. Check the connected application to see what was saved or queued.
Now ask students to compare the assistant's explanation with the application record. Did it stop? Did it change information that was outside the assignment? Could an adult identify what had already happened? Students learn the value of evidence, privacy and accountability without needing to understand the technical machinery inside the model.
This exercise also makes a valuable media-literacy point. An AI system may describe itself confidently, but fluent language does not prove it performed a task correctly. Checking a calendar record, an email outbox or an account permission setting is a better way to establish what happened.
Useful AI Deserves Trustworthy Rules
Adults sometimes respond to uncertainty by trying to ban every new tool. That approach can deny students opportunities to develop judgment and practical skills. I work with organizations on AI adoption, and I have found that concrete boundaries and meaningful control can help people use technology with greater confidence.
Schools can begin with low-risk activities, such as generating discussion questions or preparing drafts from fictional examples. Expand to connected systems only when educators can explain what the tool accesses, what it may do, who approves consequential actions and how permissions are revoked. Students should be participants in these conversations, since they are the people learning the habits they will carry into an AI-rich world.
An AI lesson should end with two demonstrations: the assistant completing a useful assignment and the adult successfully stopping it. The second demonstration may be the more important one for building lasting digital judgment.
Gleb Tsipursky, PhD, is a behavioral scientist, CEO of Disaster Avoidance Experts, and a former professor at Ohio State and lecturer at UNC-Chapel Hill. He consults on AI adoption and organizational behavior.
His latest book is The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026).
