An AI Governance Policy for Mid-Market Companies
An AI governance policy for mid-market companies works when it fits on two pages and people actually follow it, not when it runs forty pages and satisfies an auditor nobody has met. The enterprise frameworks most teams copy were written to survive review, not to guide the person deciding whether to paste a customer list into a chatbot at 4pm. This is the skeleton of a policy that settles real decisions: approved tools, data that never goes in, when a human must check the output, what gets logged, who to call when something breaks, and who owns it. It is not legal advice, and the final version should be reviewed by qualified counsel.
I lead governance, and I have watched more AI policies die on a shared drive than get followed. The failure is almost never a missing clause. It is that the document was written for the wrong reader. So before the skeleton, it is worth being honest about why the long ones fail.
Why do most AI governance policies fail?
Most AI policies are written for auditors, not employees. That single choice explains the rest.
An auditor-first policy is exhaustive, defensive, and structured to prove that every conceivable risk was considered. It reads like a contract because, in effect, it is one, written to be produced in a review rather than consulted at a desk. The person it is supposed to guide, the marketer with a deadline, the analyst with a spreadsheet, the developer with a support ticket, opens it once, cannot find the one line that answers their question, and closes it forever. From then on they decide alone.
A policy that gets followed is written for that person. It answers the questions they actually have, in the order they have them, in language that does not require a compliance translator. It is shorter not because it is lazier but because it is disciplined about what a policy is for. The long framework and the two-page policy can hold the same rules. Only one of them changes behavior.
How big is the gap between AI adoption and AI governance?
Wide, and widening, which is the reason this matters now rather than later. The published numbers tell a consistent story of intent outrunning control.
On maturity, 90% of organizations have funded AI governance and 74% believe they could pass an AI compliance audit, yet only 27% describe their programs as fully mature (Schellman, State of AI Governance 2026). The confidence is running ahead of the capability.
On proof, 55% of organizations have core AI policies and training in place, but fewer than one in five (19%) have the logging and retention controls to prove what happened (Arctera, State of AI Governance 2026). A policy you cannot evidence is a policy you cannot defend.
On pace, 55% of enterprises are actively deploying AI while only 26% report governance frameworks aligned with that pace (Smarsh and FTI Consulting, 2026 Enterprise AI Trends Study). Deployment is not waiting for governance to catch up.
And on what is coming next, 74% of organizations plan to adopt agentic AI within two years, but only 21% have a mature governance model for agents (Deloitte, cited 2026). The systems that act on their own are arriving before the rules that would hold them.
Read together, these say the same thing a governance lead sees on the ground: the tools are in the building, and the policy that governs them is either missing, unread, or unprovable. The fix is not a bigger framework. It is a smaller one people obey.
What goes in a two-page AI policy skeleton?
Here is the skeleton. Six sections, and for each, the single decision it exists to settle. If a section does not settle a decision an employee actually faces, it does not belong on the two pages.
| Policy section | The one decision it settles |
|---|---|
| Approved tools | Which AI tools may be used for work, and how to request one that is not on the list |
| Data that never enters a prompt | What information is off limits to paste in, named plainly, so there is no judgment call |
| Human review triggers | When a person must check AI output before it is used, sent, shipped, or decided on |
| Logging | What gets recorded about AI use, so what happened can be reconstructed later |
| Incident path | Who to contact and what to do when AI produces something harmful, wrong, or leaked |
| Ownership | Who owns this policy, keeps it current, and answers questions about it |
Notice what is not here: aspirational principles, a glossary, a maturity model. Those can live in an appendix. The two pages are only the decisions. The rule of thumb is that a new employee should be able to read the policy in one sitting and correctly answer can I paste this in, do I need a human to check this, and who do I tell if it goes wrong. If they can, it will be followed. If they cannot, its length will not save it.
How should a policy handle shadow AI?
Govern it, do not ban it. Shadow AI, the tools people use without approval, is a symptom, and banning treats the symptom while feeding the disease.
People reach for unapproved AI because the approved path is slower, narrower, or does not exist. Forbidding it does not remove the demand; it removes your visibility into how the demand is being met. The productive move is to make the sanctioned route fast and genuinely useful, give people an easy way to ask for a tool they need, and draw bright lines on the data that is never allowed regardless of tool. Do that and usage moves into the open, where you can see it, support it, and govern it. A ban gives you a clean-looking policy and a dirty reality. Governing shadow AI gives you the reverse, which is the one that matters.
How do you prove what your AI actually did?
With an evidence layer, and this is the part most policies skip until an auditor or an incident forces the question. A rule you cannot evidence is a rule you are only asserting.
The evidence layer is logging and audit trails: a record of which tool was used, on what input, by whom, and what came back, retained long enough to reconstruct events after the fact. The gap here is real and measured, and it is why the section deserves the same weight as the rules themselves. Decide what you log, where it lives, who can see it, and how long you keep it, and write that into the policy rather than leaving it to whatever each tool happens to record by default.
This matters most as the systems start acting on their own. When AI drafts a document, a human can catch a bad output before it goes anywhere. When an agent takes actions across your systems, the log is often the only place the decision is visible at all. Before you let agents act, decide how you will evaluate and record what they do; our guide to how to evaluate LLM outputs covers the checking, and our work on AI agents and Agentforce development is built around keeping that record so an agent's actions stay reviewable rather than opaque.
How do you roll out an AI policy people actually follow?
You roll it out the way you wrote it: for the people, not the file. A policy that is technically live but practically unread has not been rolled out, only published.
Three things carry it. First, training that is short and concrete, built around the decisions people actually face rather than a recitation of the document. Second, defaults over rules: wherever you can, make the safe path the easy one, so following the policy is the route of least resistance rather than an act of discipline. The approved tool that is one click away beats the rule that asks people to remember a restriction. Third, a review cadence, because AI tooling moves and a policy frozen at launch is out of date within a quarter. Put a named owner on it, revisit it on a schedule, and update the appendices without reissuing the whole thing.
None of this is legal advice, and I will say it once more plainly: draft the operational policy first, then have qualified counsel review it against your sector, contracts, and jurisdictions before you rely on it. The skeleton settles the practical decisions. Counsel confirms it holds where the law is specific to you.
The companies that get this right are not the ones with the longest frameworks. They are the ones whose two pages are actually read, whose approved tools are actually usable, and whose logs can actually show what happened. If you want help turning a policy into an evidence layer that holds, including for agents that act on their own, that is work we do. See how we think about proposals and red flags when vendors overclaim, or talk to us about governance you can prove rather than just publish.
