FIVE KEY QUESTIONS TO ASK
Richard Human looks at how school leaders can take the initiative to monitor AI use in their school effectively . . . before they face a major problem.
The implementation gap
Ask a school leader which AI tools their school has chosen to use, and an answer normally comes back quite quickly. Ask which AI tools staff and students are actually using, and the confidence usually disappears. That gap between sanctioned AI and AI really in use is no longer just a teaching and learning question. It is a legal one.
Two legal frameworks
Countries around the world are developing their own approach. Two general frameworks, however, now sit over this gap and are leading the way. Both are daunting and define general principles which require universal compliance:
- The EU’s strict GDPR framework has applied to schools for some years, and its accountability principle asks a simple question: can you demonstrate what happens to personal data in school and on what lawful basis?
- More recently, the EU AI Act asks something close to it from the other direction. Schools that use AI rather than build it are classed as “deployers” under Article 29, and Article 4 requires staff to have “a sufficient level of AI literacy.” From August 2026, AI used in admissions or assessment becomes high-risk, with obligations requiring transparency and human oversight.
The legal net is wider than it looks: a school offering remote exams may already be using behaviour-monitoring AI through its invigilation provider, without anyone having assessed it as such. The Act is particularly far-reaching in certain respects, banning AI that infers emotion from a pupil’s face or voice since February 2025, signalling how broadly “AI in education” is now defined, even where the specific breaches may be rare.
Neither framework cares whether a school has chosen to use a specific tool. Both care whether the school knows it is being used, and the tool most likely to cause a problem is rarely the one a school carefully vetted – it is the one a teacher or pupil quietly adopted because it saved time. A marking assistant that retains pupil scripts to improve its model, a homework chatbot that stores a child’s name alongside their queries, a free transcription tool used in a parents’ evening: none of these arrive through procurement. All of them create the same exposure as personal data are processed without a documented basis, by a system nobody assessed.
A five-question audit, before you read on
Before we continue, take a moment to answer these questions honestly. They will only take a minute and it will tell you more about your compliance position than any policy document will.
- Could you list every AI tool staff use to draft emails, reports, or letters to parents?
- Has any member of staff or pupil installed an AI tool, extension, or agent on a school-owned device or system (laptop, iPad, email, MIS) without it going through a permissive protocol supervised by competent people?
- Do you know which AI tools your pupils use for homework or coursework, including the ones the school hasn’t sanctioned?
- For your most-used tools, do you know where that data is stored, and whether the vendor can use it to train their own models?
- Is there one named colleague who could answer all four of these questions today, if asked?
A starting point
If you could not answer any one of these with confidence, that is not necessarily a reason for alarm! It is the normal starting position for most schools right now. But it is a gap that needs closing quickly, because it is precisely the gap GDPR and the EU AI Act are both built to test.
There is a related blind spot also worth naming: not just which tools are running, but whether staff are open about using them. A teacher who drafts a parent letter, a report comment, or a worksheet with AI and says nothing, is a transparency gap in miniature and it sits awkwardly next to a school that expects pupils to declare their own AI use. Schools that ask pupils to be honest about AI need staff modelling the same standard, not because the regulation specifically demands a disclosure line on every email, but because the credibility of the policy depends on it applying both ways.
Closing the gap
Closing it does not require a data protection officer on retainer. It requires one trained, supported colleague and someone experienced enough to back them when things get complicated. Rather than handing a school’s AI Leader a syllabus and leaving them to it, what is needed is a proactive support programme operating to do just that – support them. Ideally there will also be assistance for parents and students as well, helping build a complete Governance Toolkit audit, risk register, practice matrix, and risk assessment methodology managed by the AI lead who will soon be much better informed about what is happening and where gaps have formed.
None of this removes risk entirely, and none of it replaces the need for legal advice about a school’s specific obligations. What it removes is the uncertainty of facing the future in isolation and before an inspector or regulator starts asking one of the ‘five questions’ or a data breach occurs. Better to find the gaps in your own audit, with someone experienced beside you, than to have a regulator find them first.
Richard Human delivers the AI Safe Schools Programme through Consilium Education. He is a Founding Fellow of the Chartered College of Teaching, an adviser to Cambridge University Press & Assessment, and a member of the OECD Future of Education Network.
