AI and Data Privacy: What the Numbers Say, and What Organizations Must Do
By Nassir Al Rumhy | Head of Data Science & AI
Imagine this.
An employee is preparing a board paper late in the evening. Tomorrow’s meeting is important, time is short, and the presentation still does not look right. Wanting to improve it quickly, they open an AI tool and type:
“Please improve this presentation and make it more executive level.”
Without thinking much about it, they paste internal financial numbers, project details, names of suppliers, perhaps even commercially sensitive information.
Within seconds, the AI produces a polished result. Cleaner wording. Better structure. Stronger messaging.
The employee closes their laptop feeling productive. But something important just happened.
The information they passed did not stay inside their laptop.
Depending on the platform, settings, and organizational controls in place, that data may have travelled through external cloud environments, temporary processing systems, third-party infrastructure, or storage mechanisms completely outside the organization’s visibility. In some cases, inputs may be retained for monitoring, model improvement, security review, troubleshooting, or future service optimization, particularly when personal or free accounts are used.
This does not mean every AI tool is unsafe. Many enterprise-grade platforms offer private environments, contractual protection, data isolation, and guarantee that customer information is not used to train public models.
The issue is that most employees are unaware of the difference.
To them, it is simply “an AI tool.”
To the organization, it could mean confidential business information leaving controlled environments without anyone realizing it.
And this is exactly where the conversation about AI and data privacy begins.
As I write this article with the help of an AI tool, billions of AI-driven decisions are being made across industries and geographies. Adoption is accelerating at a pace most governance functions have not kept up with. The numbers tell a clear story.
SC Media reports that 38% of employees who use AI tools at work admit to submitting sensitive information without their employer’s knowledge. IBM’s 2024 Cost of a Data Breach Report puts the average incident cost at $5.2 million — 28% higher than conventional breaches. A Kiteworks survey of cybersecurity and risk management professionals goes further, finding that 83% of organizations lack automated controls to prevent sensitive data from entering public AI tools, and 86% have no visibility into their AI data flows at all.
Locally, the Royal Oman Police reported a 50% surge in cybercrime and fraud cases in the first half of 2025 alone, with investigators linking the rise directly to the rapid growth of digital platforms and the use of AI-powered tools by attackers.
What these numbers collectively reveal is not a technology problem. Behind every statistic is an employee who did not know better, a policy that did not exist, or a governance gap that no one had closed yet. The risk is human in origin, which means it is also human in solution.
Most AI risk inside organizations does not come from sophisticated attacks. It comes from well-meaning people using the wrong tools, or the right tools in the wrong way. That is where governance needs to focus.
So what should organizations actually do? Six principles matter most.
- Treat AI Inputs Like Third-Party Disclosures
When data enters an AI platform, it leaves your direct control. If you would not send that information to an external party, it should not enter an AI tool.
- Establish and Enforce an Approved Tools List
Personal accounts on public AI platforms sit entirely outside organizational governance. Without a clear approved list, policy cannot be enforced and liability cannot be managed.
- Anchor Decisions in Existing Data Classification Practices
As highlighted in Point 1, employees should treat information entered into AI systems in the same way they would treat information shared with any external third party. In most cases, subject matter experts already understand the sensitivity of the information they work with daily and can reasonably determine whether content is public, internal, confidential, commercially sensitive, or restricted.
The key principle is simple: if the information would not normally be shared externally, it should not be entered into external AI platforms unless approved controls, governance requirements, and security measures are in place.
- Use AI for Structure and Process, Not Raw Sensitive Data
AI does not need real names, real financials, or real client details to deliver value. Anonymizing inputs wherever possible reduces risk without meaningfully affecting output quality.
- Create a Culture Where Gaps Are Raised, Not Worked Around
Employees who quietly navigate unclear policies do not reduce risk. They accumulate it. Organizations that want to use AI safely need to make it easy for people to raise uncertainty rather than avoid it.
- Review Tools on a Regular Cycle
Data handling terms change. A platform considered safe six months ago may operate under different conditions today. Periodic review is not optional. It is governance.
The organizations that will use AI well are not those that move the fastest. They are the ones that build clarity into how they adopt it, with policies that are understood and followed at every level.
AI offers genuine and transformative value. That value is only sustainable when trust, governance, and data responsibility are built alongside it.
And perhaps that brings us back to the employee sitting late in the evening, trying to finish tomorrow’s presentation faster with the help of AI.
The presentation may have improved.
The productivity gain may have been real.
But if sensitive information quietly left organizational boundaries without anyone understanding where it went, what it could be retained for, or who ultimately remained responsible for it, then the real cost may only become visible much later.
The real question is not whether employees will use AI. They already are.
The question is whether organizations have created the clarity, guardrails, and trust needed for people to use it safely.
