Last week, half a dozen of the most prominent artificial intelligence enterprises worldwide committed to the White House to permit external parties to review their operations.
On September 29, Nvidia, OpenAI, Anthropic, Google, Meta, and xAI executed an agreement regarding cutting-edge AI systems which U.S. President Donald Trump described as “morally binding.” Every single business pledged that it will “collaborate with an independent external auditor or evaluator” and have independent board members assess the auditors’ findings to ensure issues are resolved.
External reviews represent a positive concept, and I am pleased to witness their inclusion. Nevertheless, the arrangement omits details concerning who selects the inspectors, the scope of their examination, or whether conclusions become public; furthermore, no penalties exist for disregarding it. Presently, individual corporations designate their own auditor and define the boundaries of what that auditor is permitted to inspect.
Back in 2000, Enron compensated the accounting firm Arthur Andersen—previously part of international accounting’s “Big Five,” now reduced to the “Big Four” following Arthur Andersen’s collapse—$25 million for bookkeeping assessments alongside an additional $27 million for taxation and advisory services. Within two years, both entities had failed. Legislators reacted via the Sarbanes-Oxley Act, which removed auditor selection rights from corporate leadership, prohibited inspectors from offering most supplementary services to their audit clients, and established a public oversight board to evaluate the auditors themselves.
Andersen failed not because of technical incompetence, but owing to a conflict of interest driven by a client providing $52 million across a single 12-month period. Unenforced pledges within the AI sector possess an equally underwhelming track record. During 2023, leading AI developers established a series of voluntary commitments with the White House, which included granting external specialists permission to evaluate their systems prior to deployment. When investigators from Stanford, Harvard, and Brown subsequently assessed 16 corporations against those pledges, the average score reached 53%. Regarding the promise to assist third parties in discovering and disclosing vulnerabilities, eight out of the 16 organizations earned a score of zero.
Even when independent testing does occur, the creator dictates the parameters. METR, a non-governmental organization specializing in AI evaluation, documented that its assessment of OpenAI’s o3 system prior to release transpired “over a mere three-week span,” featuring “restricted access to data concerning the system” coupled with zero visibility into the system’s sequential reasoning process.
To be fair to OpenAI, it permitted the evaluation in the first instance. Still, this underscores how heavily outcomes depend upon what an enterprise decides to disclose.
A trained artificial intelligence model essentially functions as a massive dataset of numerical parameters, frequently requiring investments exceeding $100 million to build, meaning whoever holds a duplicate effectively possesses the model. Transferring duplicates to outside entities creates a security vulnerability. Until recently, this dynamic forced evaluators into an awkward dilemma, clearly articulated by Google DeepMind: either the testers surrender their inquiries, granting the enterprise advance sight of the test, or the enterprise surrenders its model.
During August, DeepMind unveiled a method to bypass this limitation. Cooperating alongside Singapore’s AI Safety Institute alongside three distinct external organizations, it executed what it termed the globe’s inaugural double-blind assessment of an AI model, mirroring the methodology of a double-blind pharmacological trial. The evaluation operated on isolated computing architecture, ensuring external evaluators never observed the model while Google never viewed their test questions. Google asserts that both participants can verify mathematically that their respective data remained confidential.
This approach additionally shifts the burden of trust onto the hardware itself. Last year, investigators successfully penetrated this category of isolated hardware using under $1,000 worth of apparatus, though physical contact with the server was necessary to achieve it. That reality deserves serious attention. Yet, it represents a significantly smaller challenge than the argument that “we cannot permit anyone close to the model,” and it remains manageable by exercising vigilance over who maintains physical control of the infrastructure.
Within our study, “Alternatives to Concentrated AI,” released this week by the Decentralization Research Center, my co-author Kyle Bligen and I consulted 36 individuals and cataloged nearly 30 functional alternatives to consolidated oversight of artificial intelligence. Our primary conclusion was straightforward: external parties can currently analyze advanced AI systems without acquiring possession of either the model or its underlying data.
If the evaluations mandated by the accord are to hold any genuine value, inspectors ought to dictate the scope of their tests, especially now that they can accomplish this without acquiring a duplicate of the system. Corporations should additionally disclose the identities of their auditors alongside the parameters of what the reviews covered, even if granular findings remain confidential. Publicly traded companies have identified their auditors for decades without forfeiting proprietary trade secrets. Moreover, the board committees analyzing these evaluations must report to an entity external to the organization. Sarbanes-Oxley established an oversight body for the auditors. The current accord lacks any equivalent mechanism, and lawmakers should determine the appropriate measure to address that deficiency.
Note: The perspectives articulated in this opinion piece belong solely to the writer and do not necessarily mirror those of CoinDesk, Inc. or its parent companies and affiliates.
Originally published at https://www.coindesk.com/opinion/2026/10/16/big-tech-s-ai-auditors-will-only-see-what-they-re-shown.
