Microsoft Agrees to New AI Privacy Rules for Schools

Schools are adopting AI faster than regulation is catching up.

That has created a simple but uncomfortable question:

What happens to student data when AI tools are used in class?

This week, Microsoft, the American Federation of Teachers and the United Federation of Teachers announced a new National AI Safety & Privacy Standard for schools.

The agreement introduces legally enforceable protections for student and educator data when Microsoft AI tools are used in U.S. schools.

And some of the rules are unusually clear.

Student data cannot be used to train AI

Under the agreement, student and educator data cannot be:

• used to train AI models
• sold
• repurposed for unrelated uses

Schools retain control over how data is collected, stored, retained and deleted.

That matters because one of the biggest concerns around classroom AI is not only what students type into a chatbot.

It is what happens to that information afterwards.

Read Microsoft's announcement

Schools keep control over AI use

The agreement also says schools should retain control over when and how AI tools are used.

AI systems must include human oversight and cannot independently make important decisions about students.

Families and educators must also receive clear information about:

• what data is collected
• how AI systems use it
• what safeguards are in place
• what changes are made to the product

That transparency requirement may be just as important as the privacy rules themselves.

Schools should not have to guess what an AI product is doing behind the scenes.

Manipulative AI experiences are also restricted

The standard goes beyond data protection.

Microsoft says AI systems used in schools must avoid harmful or manipulative experiences and include protections for sensitive information.

This comes at a time when schools are becoming increasingly cautious about AI companions and systems designed to create emotional relationships with users.

The broader principle is simple:

AI used for learning should support students, not manipulate them.

Why this agreement is unusual

The rules are not just recommendations.

U.S. school districts can incorporate the protections directly into their Microsoft customer agreements.

That makes them contractually enforceable.

If the terms are broken, districts can potentially end agreements or seek remedies.

That is very different from a company simply publishing a set of voluntary AI principles.

The American Federation of Teachers describes the agreement as an attempt to create enforceable protections in the absence of broader federal regulation.

It could become a model for other companies

Microsoft is only one major provider of AI tools used in education.

Google, OpenAI, Anthropic and others are also expanding their presence in schools.

If similar standards spread across the industry, schools may begin to expect certain protections by default:

No training AI on student data.

No selling student data.

Human oversight for important decisions.

Clear information for families and educators.

School control over how AI is deployed.

Those expectations could eventually become more important than individual product features.

The bigger shift

For the past few years, much of the conversation around AI in education has focused on what the technology can do.

Write.

Explain.

Tutor.

Summarize.

Generate.

The next phase may focus much more on the conditions under which schools are willing to use it.

Who controls the data?

Who makes the decisions?

What can the AI system do with student information?

Can a school disable it?

Can families understand what is happening?

Those are less exciting questions than generating an essay in five seconds.

But they may determine which AI tools schools actually trust.

Sources

Microsoft: AFT, UFT and Microsoft announce National AI Safety & Privacy Standard for schools

American Federation of Teachers: National AI Safety & Privacy Standard

Microsoft: Safe Participation Framework for young people and AI