Who Verifies the Verifier¶

Q: What is independent measurement in AI governance?

An assessment of whether a system's governance actually held, produced by a party that did not produce the thing being measured and is structurally separate from the party that did. Independence in this sense is not a claim about the assessor's sincerity, employment, or technical competence. It is a property of the assessor's position in relation to the thing being measured, the standard it is measured against, the criteria applied, the evidence preserved, and the audience served.

Q: What is the difference between independent staff and an independent institution?

Independent staff means the people performing an assessment do not work for the company being assessed. That is a hiring policy. An independent institution additionally locates the funding, the assessment criteria, and the authority to appoint and remove the assessor outside the party being measured. That is a structure. A body can be staffed entirely by people of unimpeachable integrity, none of whom works for the measured party, and still occupy a position from which an independent measurement cannot be produced.

Q: Why does motive not resolve whether a verification result can be trusted?

Because motive and position are different questions. A sincerely motivated industry-funded evaluator and a cynically motivated one occupy the same structural position: both are funded by the party being measured, and both test against criteria that party helped set. Sincerity can't be proved, can change tomorrow, and is irrelevant to the structural question. Position can be checked, holds still, and decides whether the measurement means anything. A field that debates motive will produce better-intentioned institutions; only a field that debates position will produce measurable ones.

Q: Why is the issuer-pays problem relevant to AI auditing?

Because it is a documented failure mode of the self-regulatory and ratings models that AI governance proposals are copying. When the party being rated funds the rater, rating standards face a persistent downward pressure that no individual participant has to intend. A related failure is composition drift, in which a board designed with a public-interest majority gradually reorients toward the priorities of the members who sustain it. Neither requires anyone to act in bad faith, which is why the structure rather than the sincerity of its architects is the thing to examine.

Q: Why is post-deployment assessment the hard case for AI governance?

A pre-deployment evaluation characterizes a model. A deployed system that takes consequential actions raises a different question, in two parts: whether the controls described are the controls operating, and whether they held while the system acted. Neither is established by testing a model before release. A system that acts continuously cannot be characterized by a single assessment at a single moment, so a measurement performed once and not repeated describes a system that no longer exists.

Q: How do the five conditions in this paper differ from the Five Laws of AI Governance?

They govern different objects and are easily confused. The Five Laws of AI Governance, authored by Pharns Genece (2026), are five structural prohibitions on the system being measured: Independent Measure, Binding at Dispatch, External Witness, Separated Authority, and Revocable Authority. Stated as questions, that set asks whether a system can grade itself, whether it is the sole witness to its own conduct, whether it certifies its own safety, whether it can be stopped, and whether its limits were set before it acted or explained afterward. Those are questions about the subject. The five conditions in this paper are questions about the assessor: position, criteria, reproducibility, audience, and standing. The two sets are independent of each other. A system can satisfy every condition of good governance and still have that governance measured by a party in no position to measure it, and a rigorous, structurally independent evaluator can assess a system whose governance is incoherent. The first is established by how the system is built; the second by where the assessor stands. The Five Laws are published as a citable reference under DOI 10.5281/zenodo.21271749.

Q: How do the five conditions relate to the governance of the system itself?

They address different objects, and a reader can hold both without conflating them. One set of questions is about the system being measured: whether it can grade itself, whether it is the sole witness to its own conduct, whether it certifies its own safety, whether it can be stopped, and whether its limits were set before it acted or explained afterward. The five conditions in this paper are about the assessor instead: position, criteria, reproducibility, audience, and standing. The two sets are independent of each other. A system can satisfy every condition of good governance and still have that governance measured by a party in no position to measure it, and a rigorous, structurally independent evaluator can assess a system whose governance is incoherent. The first is established by how the system is built; the second by where the assessor stands. Most existing work addresses the governance of the system. This paper is about the measurement of it.

Through 2026, six institutions and one public figure proposed answers to the question of who should check the companies building frontier AI systems: a state licensing regime, an industry-funded standards body modeled on FINRA, a frontier lab embedding external evaluators with employee-like access, a federal bill licensing independent verification organizations, a federal testing and evaluation track, and an accord signed at the White House asking each signatory to partner with an independent external auditor or evaluator. Each proposal supplies independent staff, meaning technical people who do not work for the company being assessed. None of them locates the funding, the assessment criteria, and the appointment authority outside the party being measured. Independent staff is a hiring policy. An independent institution is a structure. Verification independence is not a property of the verifier's sincerity, employment, or technical competence; it is a property of the verifier's position.

On evaluator independence, result transparency, and the vocabulary AI governance has not yet agreed on.

1. The question the 2026 AI safety proposals did not center¶

A guardrail might hold. Hit it at the wrong angle and you go through it. A bollard in front of a storefront does not have that property: it does not negotiate with the vehicle, and nobody has to trust it to work. One is a probability. The other is a constraint. That is the whole distinction between guidance and governance, and a great deal of what is currently presented as AI safety is built to the first standard while being described in the language of the second.

The distinction matters more now because the systems are being given more to do. A control that might hold has to be tested by someone, and in every field where the consequence is physical, somebody outside the manufacturer does that testing. A car is not trusted because its maker says it is safe. It is trusted because an independent body crash-tests cars nobody asked it to test and publishes what it finds. The manufacturer builds. Someone else says whether it was manufactured safely.

No such body exists for AI systems that act. That phrase is doing real work: a model that answers a question produces output a person can read and judge before acting on it. An agent executes. It books, files, transfers, dispatches, configures, and shuts down, and the consequence lands before anyone reviews it. What governance those systems have is asserted by the party that built them, measured against criteria that party selected, and reported by that party to itself. There is no outside reading to compare it against.

Through 2026 a number of serious institutions began proposing ways to fill that role. This paper is about what their proposals have in common, and about the vocabulary a reader would need in order to tell a credible verification claim from a confident one.

Verification independence is not a property of the verifier's sincerity, employment, or technical competence. It is a property of the verifier's position: in relation to the thing being measured, the standard it is measured against, the criteria applied, the evidence preserved, and the audience served.

That sentence is the whole argument. Everything below is either evidence that the field has not yet absorbed, or an attempt to state precisely what follows from it.

The occasion for writing this was a cluster of proposals that appeared in 2026, which is worth describing because the pattern across them is more instructive than any one of them individually.

Seven institutional actions addressed AI governance, and an eighth voice answered the same question in public. Each is described briefly below.

The chief executive of a frontier AI laboratory published a framework proposing a national standards body for frontier models, modeled explicitly on a financial-industry self-regulatory organization, funded largely by the industry and empowered to test models before deployment. Three days later, the Treasury Secretary was reported to have helped develop a proposal along the same lines, for an independent frontier-AI regulator reporting to a federal financial regulator. A state legislature enacted a framework for licensed third-party AI auditors, creating a registry and a designation process. A second laboratory chief executive proposed embedding external evaluators inside his own company, with desks, access badges, and company laptops, committing to it unilaterally. A bipartisan bill in Congress would require developers of the most capable systems to maintain the technical ability to throttle, suspend, or shut them down. A federal standards body opened a standardization track on how AI systems should be tested, evaluated, verified, and validated. And in September a voluntary accord signed at the head of government by the largest frontier developers asked each signatory for four layers of control. Three sit inside the firm: its own controls, its own team, and a committee of its own directors. The fourth asks for an external assessment of whether the other three are working, and leaves every qualifying term for that assessor blank.

One of those actions went further than the rest in a respect that deserves to be credited precisely. The embedded-evaluator proposal gave its reviewers the right to publish their findings, including the access they did or did not receive, without editorial control by the company, reserving only a narrow redaction for security-sensitive, legally privileged, or commercially sensitive material and not for findings that are merely unfavorable. Among the actions described here, that is the strongest concession an operator has made on this question, and it closes the failure mode that makes most voluntary review worthless. It specifies, in operational detail, what an evaluator may see and say. It does not specify who is qualified to be one, against what criteria, or who they answer to.

An eighth voice, the president of an AI-safety organization, was asked on national television what would actually constrain the frontier and answered with liability law and hardware verification. Asked again, the answer did not change. Both are serious remedies. Neither is an assessment of whether a developer's governance held, which is the question the other seven were circling.

Seven actions. Seven framings. One recurring omission.

Each of the seven actions specifies, in real operational detail, who builds the safeguard, who enforces it, or who governs the body that oversees it. The framework specifies the board composition. The legislature specifies who may call themselves an auditor. The laboratory specifies what access an evaluator receives, down to the laptop. The bill specifies the capability a developer must maintain. The standards body specifies the vocabulary that evaluation will use.

Each of them treated independent measurement as an implementation detail. They specified authority, participation, funding, or governance before specifying the accreditation, conflict-control, evidence-access, reproducibility, transparency and reporting conditions under which a verification claim would be credible.

This is not a complaint about any of these efforts. Each is a serious response to a real problem, and each is better than the absence it replaces. It is an observation about a shape. When seven institutional actions approach the same problem from seven directions and arrive at seven versions of the same omission, the omission is probably structural rather than accidental.

Some of these gaps will be filled. Accreditation layers, appeals processes, evaluator qualification criteria, and public reporting requirements can be added to any of these frameworks, and in some cases probably will be. The argument here is not that they never will. It is that the order in which these things were specified tells you something about which questions the field treats as foundational and which it treats as details to be worked out later. Measurement was in the second category.

There is a second pattern worth naming, because it explains why the first one persists.

Much of the political and press commentary that followed these proposals treated motive as the underlying question. Asked whether the industry's call to be regulated was genuine, a senator answered that the executives were legitimately nervous and then observed that their political spending pointed the other way. A vice president said the spectacle of companies asking to be regulated felt like a Trojan horse. A cabinet official called the timing an election-season maneuver. A congressional leader argued the industry understands its own frontier better than legislators do. Each of these is a position about whether the people proposing governance mean it.

None of them is a position about whether the proposed structure would produce a reliable measurement.

Those answers addressed a different question, and the swap is easy to miss. Motive cannot cure positional dependence. Good intentions cannot make a dependent structure independent. A sincerely motivated industry-funded evaluator and a cynically motivated one occupy the same structural position: both are funded by the party being measured, and both test against criteria that party helped set. Whichever way a motive argument is settled, it settles nothing about whether the resulting number can be relied upon. Sincerity can't be proved, can change tomorrow, and is irrelevant to the structural question. Position is none of those things: it can be checked, it holds still, and it decides whether the measurement means anything.

This is why the distinction is worth stating carefully rather than assuming. A field that debates motive will produce better-intentioned institutions. Only a field that debates position will produce measurable ones.

The omission matters in proportion to how much is being deployed against it. Industry surveys through this period consistently found a wide gap between the share of organizations deploying autonomous systems and the share reporting a mature governance model for them, with adoption expected to accelerate faster than governance maturity. The specific figures will date quickly and are not the point. The structural observation will outlast them: systems are being deployed faster than the means of assessing them are being built, and the field has not settled on a shared way to describe which side of that gap a given system sits on.

This paper is about that gap in vocabulary. It does not describe a way to close it, and it does not propose one. It asks what a reader should want to see before treating any verification result as independent, whoever produces it and by whatever method.

A version of this argument was submitted to NIST's AI Standards Zero Drafts process in September 2026.

On prior work, including my own. The observation that this role cannot be occupied from inside is not new here. I argued it publicly in May 2026, before the proposals described above existed, using the analogy that recurs throughout this paper: credit scoring, bond rating, and product safety certification each ended up in the hands of an outsider, and none of them got there by the incumbents volunteering. That essay, The Referee Seat Nobody Can Sit In, made the case that the position exists and cannot be legitimately held by a participant. A separate piece of prior work of mine, The Five Laws of AI Governance (Genece 2026, DOI 10.5281/zenodo.21271749, CC BY 4.0), sets out conditions on the system being governed rather than on the party measuring it; the distinction between the two is taken up in Section 5. This paper takes the next step: it sets out what a measurement produced from that position would actually have to satisfy, and examines the structures now being proposed to fill it.

Others have reached these conclusions independently, at greater length and with more institutional weight, and the honest thing to do is say so at the outset. A coalition of more than thirty researchers and governance practitioners published a detailed framework for frontier-AI auditing in early 2026 which sets out tiered assurance levels, extensive independence safeguards, and explicit recommendations for an accreditation program and an auditor-of-auditors body. Practitioners in the assurance field have written on professional codes, accreditation, and the structural difficulty of remaining independent inside an immature ecosystem. The questions this paper raises are live questions that serious people are already working on, in public, with more resources than any individual brings.

This paper does not claim to have found a gap nobody noticed. It makes a narrower contribution. The existing work is largely addressed to institution-builders: what the ecosystem should construct, who should fund it, what authority it should hold. Less attention has gone to the reader's side of the transaction, which is the question of how someone handed a verification result should decide how much weight to put on it, before any of those institutions exist and regardless of whether they ever do. That is the vocabulary problem this paper takes up.

2. Why this is structural, not moral¶

The most common misreading of this argument is that it accuses operators of dishonesty. It does not, and the distinction matters enough to state plainly before anything else.

A perfectly honest operator, acting in complete good faith, with excellent engineers and a genuine commitment to safety, still cannot produce an independent measurement of its own system. Not because it would lie. Because independence is a property of position, not of character. The party that designed a system, operates it, and sets the criteria it is judged against occupies every seat at the table. There is no seat left for a party whose interest is only in whether the answer is correct and verifiable.

This is not a new problem, and the fields that solved it did not solve it by asking builders to try harder.

Public companies do not audit their own financial statements. Not because accountants at those companies are dishonest, but because a statement that a company verifies about itself carries no information beyond the company's own assertion. The audit profession exists to supply a second position, not a second opinion.

Bridges are not certified by the firms that build them. The structural engineer who designed a span may be the most qualified person alive to assess it, and is still the wrong person to sign the inspection. Competence and independence are different qualifications, and a bridge needs both from different people.

Pharmaceutical efficacy is not established by the manufacturer scoring its own trial. The trial protocol is registered in advance, the endpoints are fixed before the data arrive, and the analysis is reviewable by parties with no stake in the result. The manufacturer runs the trial. It does not get to grade it.

Airworthiness determinations are made under public authority against published criteria the manufacturer did not write, rather than by ordinary manufacturer self-attestation. An aviation authority, or a designated representative operating under that authority's delegation, makes the determination. Every commercial flight rests on a judgment made by someone who does not work for the company that built the aircraft.

In each case, the same structure recurs. The builder builds. A separate party, structurally independent of the party that built the thing, determines whether it meets a standard the builder did not author, and reports that determination to someone other than the builder. None of these regimes was constructed on the premise that builders are untrustworthy. They were constructed on the recognition that the builder's seat and the judge's seat are different seats, and that a claim made from the first seat does not do the work of a claim made from the second.

These are not merely analogies. The problem has a formal literature, and the AI governance discussion has largely proceeded without reference to it.

Conformity assessment is the established discipline concerned with exactly this question: how a claim that something meets a requirement can be made credible to a party who was not present when it was tested. It distinguishes, as separate roles with separate requirements, the body that sets the standard, the body that tests against it, the body that certifies the result, and the body that accredits the testers and certifiers. International standards govern each layer, including the impartiality, competence, and consistency of the accreditation bodies themselves, which is to say that the question "who verifies the verifier" already has a documented institutional answer in other fields. Public-law implementations exist as well; in food safety, for instance, a federal program deliberately separates accreditation bodies from certification bodies, and requires records access, monitoring, and conflict safeguards at both layers. This machinery is already being pointed at AI. European law routes high-risk systems through notified bodies operating under national accreditation, and a management-system standard for AI is being certified by accredited registrars today. Those regimes are real, they are structured, and they answer part of the question this paper asks: they establish who may assess, under whose accreditation, against a published standard. What they assess is the management system and the pre-market conformity of a system against documented requirements. That is a different object from whether a deployed system's governance held while it was acting, which is the measurement this paper is about. The point is not that the infrastructure is missing. It is that the infrastructure exists for one object and has not yet been pointed at the other.

The vocabulary that discipline supplies is precise in ways the current AI debate is not. It distinguishes scope from competence, surveillance from initial assessment, and impartiality from independence. It specifies what happens when a finding is contested, when a certification must be suspended, and when it must be withdrawn. None of this had to be invented for AI, and very little of it has been imported.

AI governance has, so far, built almost entirely for the operator. It has largely not built for the assessor, and it has done without the conceptual apparatus other fields developed for exactly that role.

3. The two properties, separated¶

Independence is usually treated as a single thing. It is at least two, and the two are separable in ways that matter operationally.

Evaluator independence is the degree to which the party conducting an assessment is free from relationships, incentives, control, or dependencies that could reasonably impair impartial judgment. The relevant questions are concrete. Did the evaluating party, or an affiliate of it, design, develop, deploy, operate, finance, insure, or otherwise materially benefit from the system under evaluation? Does the evaluator's compensation, continued access, or future work depend on the result? Does the evaluator select the criteria against which the system is assessed, or apply criteria established by someone else? Are these relationships disclosed to the people who rely on the result?

Result transparency is the degree to which the evaluation itself is visible to the parties relying on it. Its scope. Its limitations. Any material access the evaluator requested and did not receive. And, critically, whether the evaluated party holds editorial control over what may be reported.

These are separable, and the failure modes run in both directions.

An evaluator can be structurally independent in every respect and still produce a finding that only the evaluated party ever sees, or one the evaluated party may edit before release. In that case the independence is real but unobservable. A person deciding whether to rely on the system learns nothing from it, because they cannot see the finding, cannot see what was withheld, and cannot tell whether the version they received is the version that was written.

Conversely, an evaluation can be fully published, richly detailed, and completely transparent, and still be produced by the party whose conduct is in question. Total transparency about a self-assessment yields a very clear picture of what an organization thinks of itself.

Independence without transparency is unobservable. Transparency without independence remains self-assessment. Both properties have to be present, and they have to be present in different parties, before an evaluation result means more than the assertion of the party who commissioned it.

The standards vocabulary in circulation as this was written tends not to distinguish them explicitly. A representative federal definition of evaluation describes it as a determination made against a defined and documented set of criteria, using a clear and documented methodology. Every word of that is about the rigor of the method. Nothing in it addresses the relationship between the evaluator and the evaluated. Two evaluations can satisfy that definition identically. One is performed by a system's operator on its own system. The other is performed by a party with nothing at stake. The definition does not distinguish them, which means any regime that references it inherits the ambiguity.

4. Five ways the role gets filled without the measurement being independent¶

The most useful thing to observe about the current landscape is that it is not empty. A great deal of serious work is being done, by capable people, on assessing AI systems. The problem is subtler than absence. Five patterns recur, each of which produces something that resembles independent measurement, and each of which fails at a different point.

None of these patterns is described here to disparage any organization practicing it. Each represents a genuine improvement over doing nothing, and several are meaningfully better than what the field had a year ago.

The self-attested badge. An organization adopts a published framework, assesses its own system against that framework, and publishes the resulting score or badge. The framework may be excellent. The assessment may be diligent. But the party running the assessment is the party being assessed, and the number is one it can set. In some cases the publishers of these schemes state this limitation explicitly in their own documentation, describing the output as a builder attestation rather than a verified control. The honest ones say so. The failure point is at the beginning: the assessment never leaves the first seat.

The embedded evaluator. An external team is invited inside the organization it assesses. It receives real access, sometimes extraordinary access, and it may be staffed by genuinely skilled and genuinely independent-minded people. The most developed version of this pattern proposed in 2026 includes desks in the assessed company's offices, access badges, company laptops, and permissions comparable to internal risk assessment teams. It also includes a meaningful anti-capture provision: the right to publish findings without editorial control by the assessed party, including the right to say publicly what access was denied.

That provision is a real advance and deserves to be recognized as one. The failure point is elsewhere. The evaluator is invited. The scope is set by the inviting party. And in the most detailed public proposal of this kind, there is no selection mechanism, no stated qualification, no accreditation, and no published criteria for who is eligible to hold the badge. The proposal specifies the access in operational detail and leaves the appointment authority unspecified. An evaluator whose authority derives entirely from the invitation of the party it evaluates is in a structurally different position from a bank examiner, even when the analogy to bank supervision is the one being invoked. Bank examiners are not invited. They are appointed, qualified against published criteria, and they report outward.

The tamper-evident receipt. A system produces cryptographically verifiable records of its own decisions. Anyone can check the signature. Anyone can confirm the record has not been altered. This is genuinely valuable and solves a real problem, which is the problem of after-the-fact tampering with logs.

It does not solve the problem this paper is about. A receipt establishes what happened according to the party that issued it, in a form that cannot later be edited. It does not establish that the policy being enforced was adequate, that the criteria were the right criteria, or that the governance was sound. Checkable is not the same as independently measured. An unforgeable record of a self-set standard is an unforgeable record of a self-set standard.

The licensed auditor with no instrument. A jurisdiction creates a profession. It establishes who may conduct assessments, sets independence and conflict-of-interest requirements, and builds a registry. This is real infrastructure and a significant step.

And then it asks each licensed auditor to declare the benchmarks, metrics, and methodologies it proposes to use, and supplies none itself. The result is a profession without a shared instrument. Two licensed, genuinely independent auditors can assess two comparable systems, apply entirely different criteria, and produce results that cannot be compared with each other. The independence problem is solved. The measurement problem is untouched. A regime of this kind creates demand for a common reference and does not create the reference.

The industry-funded standards body. A real institution is created, with real authority and government supervision. Its board is designed with independent technical experts, and in the more careful proposals those independent members outnumber industry representatives. It develops assessment protocols, conducts testing before deployment, and can eventually be made mandatory for market access. Membership can be defined by an objective threshold. Governors can be given staggered terms and protection from removal.

This is the most substantial of the five patterns and the hardest to argue with, and it deserves to be treated as the serious proposal it is. The self-regulatory model it draws on is not a failure. In the domains where it works, it works because of two features that are easy to overlook: the standard the body tests against is set independently of the members, and the supervising authority is not funded by them. Where both hold, industry funding of the testing apparatus is an efficiency rather than a conflict. The industry pays for the laboratory. It does not write the specification or grade the result.

The failure point in the AI proposals is that the funding is specified and those two features are not yet. The testing capacity is to be funded largely by the laboratories being tested. The criteria, at least initially, are developed in close collaboration with them. In the most detailed public formulation, the body would only eventually develop the technical capacity to create its own held-out tests independent of the laboratories, which is a candid acknowledgment that at inception the tests are not independent of them. Participation begins voluntarily. Each of these is a reasonable transitional compromise. Each is also a respect in which the arrangement, at inception, does not yet have the two features that make the model work where it works. On the proposals' own account, the output at that point is a well-supervised industry attestation.

Two documented failure modes of the model being copied are worth naming, because they are not hypothetical. The first is the issuer-pays problem: when the party being rated funds the rater, rating standards face a persistent downward pressure that no individual participant has to intend. The second is composition drift, in which a board designed with a public-interest majority gradually reorients toward the priorities of the members who sustain it. Neither requires anyone to act in bad faith. Both are why the structure, rather than the sincerity of its architects, is the thing that has to be examined.

There is also a vocabulary problem running through all five patterns, and it is the clearest single thing to fix.

The word independent is being used to describe two different properties. One is independent staffing: the people conducting the assessment are technically capable and do not draw a salary from the party being assessed. The other is institutional independence: the funding, the criteria, and the authority to appoint the assessor all originate outside the party being assessed. The first is a hiring policy. The second is a structure. The current proposals supply the first in considerable detail and describe it using the language of the second.

An evaluator can be staffed entirely by people of unimpeachable integrity, none of whom works for the measured party, and still occupy a position from which an independent measurement cannot be produced. That is not a statement about the people. It is a statement about where the money, the criteria, and the appointment come from.

Each of these patterns represents progress. Together they describe a field that has correctly identified that something is missing and has repeatedly built adjacent to it.

5. A screen for reading verification claims¶

If the five patterns above each fall short at a different point, the useful question is what a reader might look for in a claim that avoids all five failure points. What follows is a set of properties, not a design, and not a specification. It describes what tends to make a verification result worth relying on, and deliberately says nothing about how any particular instrument might achieve it.

The field currently lacks a generally accepted way to describe when a verification result is independent, reproducible, and reportable. What follows is a vocabulary proposal and a screen for reading such claims. It is not a design for an instrument, and it is not offered as a necessary-and-sufficient definition of every assurance method that could serve this purpose.

Stated compactly:

A verification claim is strongest, for the purposes of this paper, when it can show:

Position: the measuring party did not produce the thing being measured and is structurally separate from the party that did.

Criteria: the measured party did not author or control the standard applied.

Reproducibility: a competent third party can reproduce the result from preserved evidence.

Audience: the result is reported to someone other than the measured party.

Standing: the measurement is repeatable over time, including after deployment.

A claim missing one or more of these factors may still be useful and may still be the best available evidence. What it should do is disclose the limitation and use a narrower label for itself.

Other screens are possible, and a reader who finds a sixth factor worth adding, or an argument that one of these is unnecessary, is engaging the paper on exactly the terms intended. The proposal here is that the field should have such a vocabulary and argue about it in the open, not that this particular five-factor formulation is the final word.

A note on what these conditions are not. They are easily confused with a separate set of conditions I have described elsewhere as the Five Laws of AI Governance, which govern the system being measured rather than the measurement of it. That other set asks whether a system can grade itself, whether it is the sole witness to its own conduct, whether it certifies its own safety, whether it can be stopped, and whether its limits were set before it acted or explained afterward. Those are questions about the subject. The five conditions above are questions about the assessor.

The distinction matters because the two are independent. A system can satisfy every condition of good governance and still have that governance measured by a party in no position to measure it. A rigorous, structurally independent evaluator can assess a system whose governance is incoherent. Both layers have to hold, and they are established by different means: the first by how the system is built, the second by where the assessor stands.

Most existing work addresses the first layer. This paper is about the second.

The five factors in full:

Produced by a party that did not produce the thing being measured. This is the positional requirement from Section 2, stated as a test. It is not satisfied by good intentions, by contractual assurances of objectivity, or by internal separation of duties within a single organization. The question is whether the measuring party built, operated, configured, or otherwise holds a material interest in the system under measurement. If it did, or if it materially benefits from the system being measured, the measurement carries that relationship regardless of anyone's conduct.

Assessed against criteria the measured party did not author. An organization that selects the standard it is judged against has substantially determined the outcome before the assessment begins. This is the property most often lost in otherwise rigorous programs, because criteria selection looks like a technical decision rather than a governance one. It is a governance one.

Reproducible by a third party from the same evidence. If a measurement can only be reproduced by the party that performed it, it is a finding rather than a measurement. Reproducibility across parties is what distinguishes a measurement from an opinion held carefully. It is also what makes disagreement productive: two parties applying the same criteria to the same evidence should be able to locate precisely where they diverge.

Reported to someone other than the measured party. A measurement delivered only to its subject is advice. It may be excellent advice. It does not inform the regulator, the procurer, the insurer, the downstream deployer, or the public, none of whom can act on a result they cannot see. This is the transparency property from Section 3, expressed as a requirement rather than a description.

Standing and repeatable rather than a one-time engagement. A system that acts continuously cannot be characterized by a single assessment at a single moment. A measurement that is performed once, by arrangement, and not repeated on a defined cadence describes a system that no longer exists by the time the result is read. Standing measurement is what allows change to be detected rather than reconstructed.

A reader may notice that funding is not among the five, given how much of Section 4 turns on who pays. That is deliberate, and the reasoning is worth stating rather than leaving implicit. Funding is not an independent property; it is one of the mechanisms by which the first two are lost. A body funded by the party it measures can still satisfy the screen if the standard is set elsewhere and the authority to appoint and remove the assessor sits elsewhere, which is the condition under which the financial-industry analogy actually works. Conversely, a body funded entirely from outside fails the screen anyway if the measured party wrote the criteria. Funding is where the failure usually enters, which is why Section 4 spends time on it, but Position and Criteria are what it damages. A reader applying the screen should treat the funding arrangement as the first place to look, not as a sixth box to tick. These properties are stated as conditions, not as a specification. Several instruments could satisfy them, and they would differ in method, in scope, and in what they are able to say. That variation is healthy. What is not healthy, and what the current landscape reflects, is a field in which the conditions themselves have not been named, so that each participant satisfies whichever subset its position permits and describes the result using the same words.

The strongest objection¶

The most serious argument against everything above runs as follows. Self-regulatory bodies with industry funding, public-interest board majorities, and a government supervisor already exist and function. Several have operated for decades. If the standard is set independently of the members and the supervising authority is not funded by them, then industry funding of the testing apparatus is an efficiency rather than a conflict, the industry pays for the laboratory without writing the specification or grading the result. On this view, the proposals under discussion are not structurally deficient; they are conventional, and the objection in this paper describes a solved problem.

That argument is substantially correct, and it identifies the conditions under which the model works. The disagreement is narrower than it appears.

Where an independently-set standard and a non-industry supervisor are both present, an industry-funded body can produce measurements that satisfy the test above. The paper does not dispute this. The observation is that in the AI proposals as drafted, the funding is specified and those two conditions are not yet, the criteria are developed in collaboration with the members, independent held-out testing is deferred to a later phase, and the accreditation of evaluators is unaddressed. A structure that would work if those conditions held is not the same as a structure in which they hold.

The objection is therefore best read as a specification of what the proposals still need rather than as a refutation. If a future body sets its standard independently of its members, is supervised by an authority those members do not fund, and accredits its evaluators against published criteria, this paper's argument is satisfied. Nothing here requires that such a body be publicly funded, adversarial to industry, or staffed by outsiders.

What this paper does not claim¶

It does not claim that operators are dishonest, that industry expertise is unwelcome in standard-setting, or that self-regulatory models are inherently illegitimate.

It does not claim that any existing evaluator is incompetent. The positional argument applies to the most capable evaluator in the world and says nothing about skill.

It does not propose a mechanism, a scale, a set of dimensions, an architecture, or an instrument, and nothing here should be read as describing how any such thing would be built.

It does not offer the five factors as a definition of independent measurement, as necessary conditions, or as the boundary of a category. They are a reading aid: a way to ask better questions of a verification claim someone hands you. Other formulations could serve the same purpose, and a formulation that displaced this one by being clearer would be a good outcome.

It does not claim that no adequate assurance approach exists anywhere, or that the author's own work is the answer to these gaps. The gaps described here are properties of a set of proposals at a moment in their development, and the useful test of this paper is whether the conditions it names turn out to be the ones that mattered.

6. Why post-deployment is the hard case¶

Most evaluation attention, and most of the maturity in existing evaluation practice, is pre-deployment. This is reasonable. Pre-deployment testing is tractable, repeatable, and it happens at a moment when the organization has both the motive and the opportunity to look carefully.

It is also, for systems that act, not the governing question.

A pre-deployment evaluation characterizes a model. The question that matters for a deployed system that takes consequential actions is different, and it has two parts that do not have pre-deployment answers.

First: were the controls the system is claimed to operate under actually in effect at the time it acted, rather than documented as intended? A control described in a policy, approved in a design review, and tested before release is not the same thing as a control that was operating at the moment a specific action was taken. The gap between the two is where most operational failure lives, and nothing established before deployment closes it.

Second: would a departure from those controls have been detected at the time, rather than reconstructed afterward? Reconstruction after an incident is forensics, and forensics is valuable. But a system whose control failures can only be found by investigating the consequences is a system whose governance is measured by its casualties.

Both of these are measurable properties of a deployed system. Neither is established by testing a model before release. And a standard that treats evaluation as concluding at deployment leaves the party relying on the result with no defined way to ask either question.

This becomes considerably less abstract when the system can actuate something physical.

I came to AI governance from that direction. From 2018 to 2025 I founded and operated a company in unmanned aircraft systems, where I authored the risk-tiering, escalation, and approval frameworks governing high-consequence flight operations as they moved from automated to autonomous. I am also a named inventor on a published international patent application for an unmanned-system cargo delivery mechanism.

Aviation is instructive here because it solved the post-deployment problem a long time ago, and did not solve it with better pre-deployment testing. An aircraft is certified before it flies, and it is also inspected on a schedule, its components tracked against service life, its incidents reported to an authority that does not work for the operator, and its continued airworthiness treated as a property that must be maintained and demonstrated rather than established once. The certificate is the beginning of the obligation, not the end of it.

The question of who verified a safety-relevant claim, under what authority, and against whose criteria is settled infrastructure in aviation rather than a matter of preference. When an AI system can dispatch a vehicle, adjust a physical process, or authorize a transaction, the identity of the evaluator stops being an administrative footnote and becomes part of what the claim means.

7. The profession has been created. The reference scale has not.¶

It would be wrong to say that nobody is positioned to perform independent measurement. As of September 2026 that is no longer true, and the change happened quickly.

A state has now created a licensed profession of independent AI verifiers, with a registry, a designation process, and statutory independence requirements. A frontier laboratory has committed to embedding external reviewers with publication rights its own contracts cannot override. A national standards body for frontier models has been proposed by industry and taken up inside the executive branch. A federal standards body is actively drafting the vocabulary that all of this will eventually reference. These are not small things, and most of them are recent.

What has not been settled is the reference the profession would cite.

The licensing regime asks each verifier to declare the benchmarks and methodologies it will use, and supplies none. The laboratory specifies the evaluator's access in operational detail and leaves the appointment authority and the qualifying criteria unstated. The standards vocabulary, as currently outlined, defines evaluation by the rigor of its method and is silent on the evaluator's relationship to the evaluated.

The design literature is further along than the legislative and industry proposals it runs alongside, and the distance between them is itself informative.

The most developed assurance frameworks do take up the hard questions. They propose accreditation programs for auditors, an auditor-of-auditors body with authority to revoke accreditation, mandatory disclosure of financial relationships, standardized engagement terms to prevent shopping for favorable conclusions, cooling-off periods in both directions, and explicit examination of whether an auditee-pays model can survive its own incentives. These are the right questions, asked by people who have thought carefully about them.

What has not happened is adoption. The frameworks proposing accreditation are proposals. The bodies they describe do not yet exist. Meanwhile the legislative and industry structures moving fastest, the licensing regimes and the standards bodies now being drafted into existence, have largely not incorporated that thinking. The gap is not that nobody has specified how to accredit an evaluator. It is that the institutions being built are not, so far, being built to those specifications.

The profession has been created. The measuring stick it would cite has not.

This is a solvable problem, and it is solvable in the ordinary way that measurement problems have been solved in other fields: by naming the properties that make a measurement worth relying on, arguing about them in the open, and letting instruments compete on how well they meet them. The naming is the part that has to happen first, and it is the part that is currently missing.

If AI governance follows the path that financial reporting, structural engineering, pharmaceutical testing, and aviation safety each followed, the sequence is fairly predictable. The profession forms. The shared reference follows. The criteria get argued about in public for several years, badly at first. And eventually a result produced by an independent party comes to mean something that a result produced by the operator never could.

The useful question for anyone building or buying AI governance right now is a narrow one, and it does not require settling any of the larger arguments.

When you are told that a system's governance held, ask who is telling you, what they measured it against, whether anyone else could reproduce the finding, and whether you would have heard about it if the answer had been different.

Cite this paper¶

Genece, P. (2026). Who Verifies the Verifier: On evaluator independence, result transparency, and the vocabulary AI governance has not yet agreed on. AAM Cyber. https://aamcyber.com/insights/who-verifies-the-verifier/

ORCID: 0009-0009-0791-4124 · Working paper · Version 1.0 · Published 3 October 2026

Other citation formats

Chicago (author-date)

Genece, Pharns. 2026. "Who Verifies the Verifier: On Evaluator Independence, Result Transparency, and the Vocabulary AI Governance Has Not Yet Agreed On." Working paper. AAM Cyber. https://aamcyber.com/insights/who-verifies-the-verifier/.

BibTeX

@techreport{genece2026whoverifies,
  author      = {Genece, Pharns},
  title       = {Who Verifies the Verifier: On Evaluator Independence,
                 Result Transparency, and the Vocabulary AI Governance
                 Has Not Yet Agreed On},
  institution = {AAM Cyber},
  type        = {Working paper},
  year        = {2026},
  month       = {10},
  url         = {https://aamcyber.com/insights/who-verifies-the-verifier/},
  note        = {Version 1.0},
  orcid       = {0009-0009-0791-4124}
}

Disclosure¶

I work on the independent measurement of AI governance quality and hold pending patent applications in that area. I therefore have an interest in the subject of this paper, and I note it so that its arguments can be weighed accordingly. They are offered because I believe the distinction is correct on the merits. I would regard a definition of independence that excluded my own work as an equally good outcome for the field, and I have tried to write one that could.

I have authored three Concept Crosswalks that appear in NIST's OLIR catalog as Final informative references, mapping a runtime AI-governance control set to the AI Risk Management Framework 1.0, Cybersecurity Framework 2.0, and SP 800-53 Rev. 5.2.0. OLIR cataloging is the publication of mapping data. It is not NIST endorsement, validation, or adoption of the underlying framework, and nothing in this paper should be read as implying otherwise.

A version of this argument was submitted to NIST's AI Standards Zero Drafts process in September 2026 as a public comment on the proposed standard for AI Testing, Evaluation, Verification, and Validation.

Pharns Genece is the founder of AAM Cyber, where his work focuses on the independent measurement of AI governance quality. He previously founded and operated a company in unmanned aircraft systems.