Guardrails Are Not Governance¶
A guardrail might hold. Governance does not negotiate. Most of what ships as AI safety is the first thing wearing the second thing's name.
Two newsrooms went looking for experts in the same week. One asked who to blame when AI goes astray. The other asked how autonomous agents really are. Neither knew they were asking the same question.
AI is increasingly used in business today. That is a good thing. Productivity gains are real, and so are the other benefits — this is not an argument against using it.
It is a double-edged sword, and the second edge is this: the way it is being deployed is reckless. Nobody scopes it. Nobody audits it. Nobody sets the rules inline at the moment of implementation. The tool arrives, people are told to use it harder, and the question of what it is actually allowed to do never gets asked.
Larger companies think they have solved this by building their own in-house AI. They skip the same steps. Same missing scope, same missing audit, same missing governance — in a building they own. "We built it ourselves" sounds like control. It is the same gap wearing a bigger badge.
And even if a company does all of it — and I would hedge that bet, most probably are not — there is no tool to measure against. No way to know whether it was done properly. That is the inherent problem with operator self-attestation, and with an industry proposing to govern itself when what it needs is third-party verification. The auto industry figured this out decades ago. Third-party attestation is why anyone trusts a safety record.
Who is to blame¶
A gun owner with children in the house is responsible for securing the weapon. If a child gets hold of it and shoots someone, nobody asks what the child was thinking. The duty belonged to the person who granted access and failed to secure it.
AI is the same question. Whoever granted the agent its access is the one to blame.
Which is why blame is the wrong tool. You reach for it after something has gone wrong, and the moment that actually counted is long past. It was the moment access got handed over. Who signed off on this agent doing this kind of thing, how far was it allowed to go, and who was it acting for? Most companies cannot tell you that about the agents they built on purpose. They certainly cannot tell you about the tools already open in browser tabs across the office, which nobody set up and nobody scoped.
So the person holding the bag is an employee who was never given the authority in the first place.
Guardrails are not governance¶
Coming down a winding mountain road, if you hit a guardrail it may retain your vehicle from going over the cliff. Or it breaks through, you go over, and your goose is cooked.
That is what a guardrail is. It might hold.
If that guardrail were more like a brick wall you can't get through — that is closer to governance. You do not negotiate with it.
And if you go over the cliff and then put up the brick wall, that isn't protection. That's a report, written after the fact, too late to matter.
Most of what ships today as AI safety is guardrails wearing governance's name.
No proof of life¶
Here is the part nobody wants to sit with. Even where guardrails or governance do exist, there is no way to measure them by an independent third party. No proof of life. It is all self-reported.
We do not trust a car because the automaker says it is safe. Not because automakers lie — because a company grading its own work is not evidence, even when the grade is accurate. Independent testing is what produces the trust. That is what the IIHS does in the US and Euro NCAP does in Europe: they crash-test cars nobody asked them to test and publish what they find.
Agentic AI has no equivalent. You cannot measure without a ruler.
What to do before the ruler exists¶
AI: governed, attested, measured.
Three things have to be true at once, and they have to agree with each other:
- Governed. Somebody decided what it is allowed to do, before and not after the fact.
- Attested. That decision was written down in a form that cannot be quietly rewritten after the fact.
- Measured. An outside party can check.
Miss one and you do not have governance of AI. You have an intention.
Most companies are missing all three. The fix starts with the one you control.
When you put AI into your business, document it. What it was approved to do, where the limits sit, who signed off. Not as a compliance chore — as your provenance record and the baseline you improve from. You cannot iterate against nothing, and you cannot show anyone what changed if you never wrote down where you started.
This is not self-attestation dressed up. I have just spent several paragraphs arguing that a company grading its own work is not evidence, and that still holds. Documentation is not the ruler and it does not produce trust on its own. It is the raw material the ruler will eventually read. An independent party can crash-test a car because the car exists and has a specification. Nobody can assess governance that was never written down.
The standard is not here yet. Start keeping the record anyway. When the ruler shows up, the companies with no history will be measured from zero.
The part nobody is looking at¶
Everyone is focused on the digital. What the model says. Whether the output is biased, wrong, offensive, hallucinated. That conversation is loud and it is almost entirely about text.
But AI and AI agents are not the same thing, and the difference is the whole point. AI makes a decision and hands you output. You look at it and decide whether to act. An agent acts on its own, and it can throw physical switches with real-world consequences.
Meanwhile agents are being handed control of things that move.
Put an agent in charge of a power grid. It reads a fault signal wrong and drops power nobody needed dropped. As mistakes go, the reasoning slip is unremarkable — models do that daily and we shrug and retype the prompt. The outage is not without consequence. It rolls straight into everything that was counting on the lights staying on.
This is where the physical and the digital converge, and the AI governance conversation is not looking at it. A guardrail on a mountain road and a prompt filter fail the same way: probabilistically, at the worst possible moment, with the consequence already in motion. The difference is that when the guardrail fails, something physical is already over the edge.
Guardrails and governance are not the same thing, and the consequences of confusing them are larger than people realize.
The chair for an independent scorekeeper exists. Nobody is sitting in it.
Updated October 2026. The chair has now been named. The White House Accord on Super Intelligence, signed September 29, asks every frontier lab to partner with an "independent external auditor or evaluator." Naming the role is not the same as setting the standard: the accord defines no accreditation, no criteria, and no qualification for who may sit there. That is the guardrail-versus-wall distinction arriving in a presidential document: the accord requires an external assessment of whether controls are "operating as intended," and leaves intended by whom unanswered. The auditor may be external. The measuring stick is still internal. Governed, attested, measured still holds, and measured is still the leg nobody has built. The Accord Created an Auditor's Chair. It Didn't Create the Measuring Stick.