There is a reflex in software engineering that serves us well most of the time: if something is slow, expensive, or error-prone when done manually, automate it. Ship it faster. Remove the bottleneck. The human in the loop is usually the problem.
The reflex is right often enough that it has become a default. CI/CD pipelines exist because the human who manually ran the deploy was the bottleneck. Auto-scaling exists because the human who watched the dashboards and provisioned servers was the bottleneck. Observability tooling is automated because the human who read logs was the bottleneck.
Now companies are deploying AI agents, and they are discovering - some gradually, some after an incident - that the governance layer around those agents has the same properties as everything else they have automated. It is slow. It involves humans. It is a bottleneck. The reflex fires, and someone proposes automating it.
This is the wrong call. And the reason is worth understanding clearly, because the pressure to make it will only increase as agent fleets grow.
What makes governance different
Every other system you have automated does work that has a correct answer. A deploy either succeeds or fails. A server either has enough capacity or it doesn't. A log either contains the error string or it doesn't. Automation on top of these systems is deterministic: it reads state, it compares against a threshold, it takes action.
Policy decisions - which of this agent's actions are permitted, which require human review, which are blocked - are not like this. They are value judgments about organizational risk. A rule that says "all contract modifications above $50,000 require legal approval" is not a fact about the world. It is a decision someone made about what level of risk the organization will accept without a human checkpoint.
That decision creates liability. If an agent routes around it, or if it is applied incorrectly, or if it fails to catch something it was supposed to catch, someone has to explain that. Not a monitoring dashboard. Not an automated runbook. A person, to another person, usually in writing, sometimes under oath.
Automating the decision does not eliminate the liability. It just removes the person from the chain that leads to it. And that makes the liability harder to discharge, not easier.
The speed argument turns out to be weak
The case for automating governance usually comes down to speed: human review is slow, agents need to act faster, HITL is a bottleneck.
This argument is weaker than it looks for a few reasons.
First, the actions that most benefit from speed - routine, low-value, low-risk operations - are usually the ones that don't need governance at all. A well-designed governance layer lets the vast majority of agent actions through automatically, because a well-designed policy engine can approve the 95% that are clearly fine without any human involvement. The bottleneck is not "governance is slow." It is "governance is not well-designed enough to distinguish the 5% from the 95%."
Second, the actions that are genuinely risky - the ones that actually warrant a human checkpoint - are usually the ones where speed matters least relative to accuracy. A contract signing or a large payment or a data export can usually wait an hour for a human to review it. The business cost of getting those wrong dwarfs the business cost of the delay.
Third, and most importantly, the organizations under the most regulatory pressure - financial services, healthcare, government contractors - do not have the option of trading away auditability for speed. For them, the question is not whether to have human-reviewed governance, but how to make it less painful. The bottleneck to solve is human review quality, not human review presence.
What AI can actually do here
None of this means AI is useless in the governance layer. It just means the role is different from what the automation reflex suggests.
The problem governance teams actually have is not that they are too slow to approve individual actions. It is that they are too slow to design and maintain the rules that determine which actions need approval in the first place. Writing a good policy requires understanding your organization's risk profile, analyzing historical behavior, knowing where the gaps are, and anticipating the edge cases. Most governance teams are doing this with raw data, a spreadsheet, and intuition honed over time.
AI can help dramatically with this problem - not by making governance decisions autonomously, but by giving the humans who make those decisions much better information. Surface the patterns they are missing. Draft the rules they do not have time to author. Quantify the impact of a rule before it goes live. Flag the rules that have drifted from reality.
This is a different product from autonomous governance automation. It is a product that makes human governance decisions faster to make, better grounded in evidence, and easier to explain after the fact.
The constraint - that a human must be in the loop on the actual policy decision - is not a limitation to be engineered around. It is a requirement that protects the organization, and the tooling should be designed to honor it while making it as easy as possible to exercise well.
Formael's view is that the right question is not how much of the human's role we can remove, but how much better we can make the decisions the human has to make. The audit trail, the evidence, the simulation - these exist to serve that question. The governance responsibility stays where it belongs.