Back to all articles
GovernmentJanuary 22, 20267 min

The Risks of ChatGPT in Government — What Can Go Wrong with Personal Data

Government employees use AI tools in their daily work. That makes sense — it saves time. But when it comes to citizens' personal data, there are serious risks that many municipalities are insufficiently aware of.

Government computer — risks of ChatGPT in municipalities

Government employees use AI. That is a fact — and it is understandable. ChatGPT saves time when summarizing documents, drafting letters, and answering questions. But when those documents contain citizens' personal data, the situation changes fundamentally.

Not because AI is bad. But because public AI tools were not built for the kind of information that government agencies handle.

What makes government information so sensitive?

Municipalities process the most sensitive information there is: youth protection files, care trajectories, social support applications with diagnoses and home situations. Information that citizens share because they have to — not because they want to.

Entering that information into a public AI system is not a minor risk. It is a violation of the trust that citizens must be able to place in their government.

What can go wrong: the Eindhoven case

In October 2025, a data breach was discovered at the municipality of Eindhoven. An internal audit covering the September-October period revealed that employees had uploaded files containing personal data to public AI websites, including ChatGPT.

This was not harmless data. Research by legal consultancy Hooghiemstra & Partners showed that it included:

  • Youth Act documents with information about the mental and physical health of minors, including citizen service numbers and sometimes photographs
  • Social Support Act documents with diagnoses, addictions, and debts — including name, address, and citizen service number

The municipality reported the data breach to the Dutch Data Protection Authority on October 23. ChatGPT was immediately blocked for all employees. OpenAI was asked to delete the files.

According to experts, that request is virtually hopeless. Once data has been entered more than 24 hours ago, it is likely already incorporated into training models. And data that has been processed into a training model is, in practice, impossible to remove.

What makes it even more painful: at the time of the incident, the municipality of Eindhoven had been under enhanced supervision by the Dutch Data Protection Authority for two years, due to earlier data breaches that had been reported too late.

This is not just an Eindhoven problem

Eindhoven made the news. But the pattern is not unique.

The use of generative AI by Dutch government organizations has increased tenfold over the past year — from 8 applications in 2024 to 81 in 2025. And more than half of municipalities have no visibility into how employees use ChatGPT.

Employees reach for the easiest tool available. If there is no approved alternative, that tool is ChatGPT. Not out of malice — but due to a lack of awareness and a lack of alternatives.

Why 'within the municipal environment' is not enough

After the incident, Eindhoven switched to Microsoft Copilot within the secured municipal environment. Understandable as a first step. But Copilot is also a product of an American company that falls under the Cloud Act.

The Cloud Act gives American law enforcement agencies the ability to request access to data from American companies — even when that data is physically located in Europe. This means that 'within the Microsoft environment' is not the same as 'fully under Dutch law'.

For government information of the type that was leaked in Eindhoven, that distinction is crucial.

What government organizations can do now

The problem is not that employees use AI. The problem is that there is no safe alternative available, which causes them to reach for tools that were not built for sensitive government information.

The solution is not to ban AI — that does not work. The solution is to provide a good alternative:

  • An AI platform that runs on Dutch infrastructure
  • That does not fall under foreign legislation
  • That does not use data for model training
  • That provides a complete audit trail
  • And that gives employees the same productivity gains as ChatGPT

That is exactly what soev.ai offers. Knowledge retrieval, summarization, writing assistance — but fully within Dutch infrastructure, without data leaving the municipal environment.

Frequently asked questions

Is ChatGPT banned in government?

There is no general ban, but the use of ChatGPT for sensitive government information violates the GDPR. The Dutch Data Protection Authority has warned about the risks on multiple occasions.

Does soev.ai comply with GDPR and BIO 2.0?

Yes. soev.ai processes all data in Dutch data centers, does not fall under the Cloud Act, and provides a full data processing agreement in accordance with the GDPR. The platform is designed to comply with BIO 2.0.

What if employees keep using ChatGPT anyway?

As long as there is no good alternative available, employees will find workarounds. The most effective measure is to offer an approved platform that integrates with the workflows of government employees.

Learn more

Schedule a demo and discover how the AI Knowledge Base works for your municipality.

Want to experience what soev.ai can do for your organization?

Book a demo and discover how sovereign AI works in your own environment.