Beyond the Threshold: Designing Compute Governance That Actually Works

The Wrong Debate

The debate about compute governance is currently structured as a binary: either compute thresholds are the right tool for managing frontier AI risk, or they are not. Advocates point to the measurability of FLOP counts and the tractability of semiconductor supply chain control. Critics point to the DeepSeek result frontier-equivalent capabilities at dramatically lower compute than assumed and conclude that hardware-based regulation is obsolete before it has been implemented.
Both positions are wrong, and their mutual opposition is preventing a more productive conversation.
Compute governance is neither sufficient nor obsolete. It is a necessary first layer of a governance architecture that requires two additional layers to function. The problem is not that policymakers chose the wrong tool. The problem is that they stopped after choosing the first one.
This essay examines what compute governance gets right, where it structurally fails, and what a more robust architecture combining compute controls with capability evaluations, interpretability requirements, and threshold-triggered escalation would look like. The institutional models already exist. The IAEA safeguards system, the EU AI Act's risk-tiering logic, and the UK DSIT's pro-innovation framework each contribute components of the answer. What is missing is a synthesis.

1. What Compute Governance Gets Right

The intellectual foundation of compute governance is sound. Three properties make hardware-based regulation genuinely attractive as a policy tool.

Measurability.

Floating point operations per second are physically defined quantities. A training run of 10^26 FLOP either occurred or it did not. This is not true of capability-based thresholds, which require ongoing evaluation against contested benchmark standards, or intent-based regulation, which requires inference about developer goals. The EU AI Act's adoption of a 10^25 FLOP threshold for GPAI models with systemic risk, whatever its specific calibration problems, reflects a genuine insight: observable quantities make better regulatory triggers than subjective assessments.

Supply chain enforceability.

The global market for AI training accelerators is structurally concentrated. TSMC fabricates the overwhelming majority of advanced AI chips. ASML holds a near-monopoly on the extreme ultraviolet lithography machines required to produce them. Applied Materials and Lam Research dominate the deposition and etch equipment markets. This concentration creates natural enforcement points that do not require pervasive domestic surveillance of AI development activities a significant political advantage in liberal democracies.

Historical precedent.

The Bureau of Industry and Security's export control architecture, built over decades to manage dual-use technologies, provides ready institutional infrastructure for compute governance. The October 2023 and subsequent 2024 export control rules applied this infrastructure to AI chips with relatively low implementation friction precisely because the legal and administrative machinery already existed.

The Hiroshima Process G7 AI Principles, adopted in October 2023, implicitly endorse compute governance by including advanced AI systems defined in capability-adjacent terms that map onto compute thresholds. The international coordination architecture, while weak, exists.
These are real strengths. The critique of compute governance should not dismiss them.


2. Where Compute Governance Structurally Fails

Compute governance fails at three structural points that cannot be addressed by adjusting thresholds.

The efficiency substitution problem.

The DeepSeek-R1 result demonstrated that frontier-equivalent capabilities can be achieved at significantly lower compute than incumbent models through algorithmic innovation. This is not an isolated event it is the expected direction of progress. The history of computing is a history of efficiency improvements that expand access to capabilities previously requiring expensive hardware. Compute thresholds set at the current frontier will, by definition, become increasingly permissive as the efficiency frontier advances. A governance framework built on fixed thresholds against a moving capability target will systematically fall behind.
The EU AI Act's implementing regulations, still being drafted by the EU AI Office, face this problem directly. The 10^25 FLOP threshold that defines GPAI systemic risk is already being challenged by models that may approach systemic risk thresholds through inference-time compute and architectural efficiency rather than raw training compute. The implementing regulations have no mechanism to address this without legislative amendment a process measured in years, not months.

The post-training capability amplification gap.

Compute governance addresses training runs. It has no mechanism to address the capability amplification achievable through targeted fine-tuning, reinforcement learning from human feedback applied to a base model, or retrieval-augmented generation architectures that dramatically extend effective capability without large training compute expenditure. A model trained at 10^24 FLOP below most proposed thresholds can be fine-tuned with targeted datasets to exhibit highly capable and potentially dangerous behaviors in specific domains at a fine-tuning cost orders of magnitude lower than the base training cost.
The UK DSIT's 2023 consultation on AI regulation acknowledged this gap but did not propose a mechanism to close it. The pro-innovation framework's emphasis on sector-specific regulation downstream of development means that fine-tuning-based capability amplification falls into a regulatory gap between the compute-focused upstream controls and the application-focused downstream regulation.

The weights proliferation problem.

Once a frontier model is trained and its weights are released or leaked, the compute required to produce it is irrelevant to its subsequent spread. The Meta LLaMA series demonstrated that weights, once public, cannot be meaningfully recalled. Open-weight models eliminate training compute as a controlling variable for deployment risk entirely. Compute governance, by design, operates upstream of this point. It has nothing to say about the governance of weights after training is complete.
The IAEA safeguards model is instructive here. Nuclear material can be physically controlled after production enriched uranium and plutonium are fungible but physically constrained quantities. Model weights are information. They can be copied, compressed, and transmitted at zero marginal cost. The asymmetry between physical material control and information control means that the nuclear nonproliferation analogy, while useful for the upstream chokepoint argument, breaks down precisely at the point where compute governance ends and deployment risk begins.


3. The Three-Layer Architecture

A governance framework adequate to the actual risk surface requires three layers operating in sequence. Compute governance is Layer 1. It needs Layer 2 (capability evaluation) and Layer 3 (deployment verification) to function as a coherent system.

Layer 1 — Compute governance (existing, requires reform)

Retain compute thresholds as the trigger for enhanced regulatory scrutiny, but reform them in two ways. First, index thresholds to the capability frontier rather than fixing them at absolute FLOP values. A threshold defined as "within two orders of magnitude of the most capable publicly known model" automatically tracks the frontier without requiring legislative amendment. Second, extend the regulatory perimeter to cover fine-tuning runs that meet capability-equivalent thresholds, not just base training runs.

Layer 2 — Capability evaluation (required, largely absent)

The IAEA safeguards model provides the institutional template. IAEA inspectors evaluate declared nuclear facilities against independently verified standards, with the authority to conduct unannounced inspections and report findings to member states without prior approval from the inspected party. The AI equivalent requires: mandatory pre-deployment capability evaluations conducted by bodies with institutional independence from developers; standardized evaluation protocols that are public and contestable; and mandatory disclosure of evaluation results to the relevant national oversight body.

The UK AISI's evaluation work with frontier labs represents a prototype, but it lacks two critical properties of the IAEA model: compulsory participation and independent publication. Voluntary cooperation from labs is valuable but structurally insufficient it creates systematic incentives to make systems appear less capable during evaluation than they perform in deployment.
A mandatory evaluation framework would operate as follows. Any training run meeting Layer 1 compute thresholds triggers a mandatory capability evaluation before deployment. The evaluation covers a pre-specified set of dangerous capability domains autonomous cyberoffense, biological synthesis assistance, autonomous replication, and deception under evaluation conditions. Results are disclosed to the national oversight body. Deployment is conditional on evaluation findings falling below pre-specified thresholds in each domain.

Layer 3 — Deployment verification (required, does not yet exist)

Layers 1 and 2 address training. Layer 3 addresses the post-deployment period, where weights proliferation and fine-tuning-based capability amplification create risks that compute governance cannot reach.
The mechanism is interpretability-based auditing with defined reporting obligations. As mechanistic interpretability methods mature the Sparse autoencoder research at Anthropic and circuit analysis work at DeepMind are advancing toward practical tools they create the possibility of runtime verification that a deployed model is not exhibiting capability profiles that diverge from its pre-deployment evaluation results.

This is not yet possible at frontier scale. The research is approximately 2–4 years from practical deployment-scale application based on current trajectory. But the governance framework should be designed now for the tools that will exist within its operational lifetime, not only for the tools that exist today.


4. The Institutional Design Problem

The three-layer architecture requires institutions that do not currently exist in the required form.
The central design problem is independence. Each layer requires an oversight body with genuine authority to compel compliance and publish findings without approval from the regulated entities. This is the property that distinguishes the FDA from a voluntary industry certification body, the PCAOB from a self-regulatory accounting organization, and the IAEA from a gentlemen's agreement among nuclear states.
Current AI oversight institutions lack this property. The US AISI operates through voluntary cooperation agreements. The UK AISI has published capability evaluations of frontier models but cannot compel participation or require disclosure. The EU AI Office has regulatory authority under the AI Act but is still developing the technical capacity to exercise it meaningfully.
The path to genuine independence runs through two mechanisms that existing bodies have used successfully. First, mandatory disclosure requirements backed by civil liability the Sarbanes-Oxley model, where failure to disclose material information to the oversight body creates personal liability for senior executives. Second, pre-market approval authority the FDA model, where no product can be deployed without affirmative clearance from the oversight body.


5. Why This Architecture Is Politically Achievable

The strongest objection to the three-layer architecture is not technical it is political. The response is threefold.
First, the political economy of AI governance is changing faster than most analysts anticipated. Mandatory capability evaluation framed as national security screening is politically viable in a way that AI safety regulation is not.
Second, the institutional model is already accepted in adjacent domains. CFIUS conducts mandatory national security reviews of foreign investments in US technology companies. A capability evaluation framework modeled on CFIUS screening for dangerous capabilities would face a different political environment than broad AI safety regulation.
Third, the alternative is reactive regulation after a major incident. The history of technology governance consistently shows that proactive governance frameworks are politically easier to implement before a crisis than reactive frameworks after one.


6. Implementation Sequencing

2025–2026 (Layer 1 reform): Reform existing compute threshold frameworks to use adaptive rather than fixed thresholds. Add fine-tuning provisions to EU AI Act implementing regulations.

2026–2028 (Layer 2 deployment): Establish mandatory capability evaluation requirements for models meeting Layer 1 thresholds. Build evaluation capacity at US AISI and UK AISI with compulsory participation authority.

2028–2030 (Layer 3 development): As interpretability tools mature toward frontier-scale applicability, integrate deployment verification requirements into the framework.


Conclusion

Compute governance is necessary. It is not sufficient. The gap between what it covers training compute and what it needs to cover fine-tuning amplification, weights proliferation, and post-deployment capability drift is precisely where the most tractable near-term risks accumulate.
The three-layer architecture proposed here assembles institutional components that already exist IAEA-style independent evaluation, FDA-style conditional deployment authority, CFIUS-style national security framing into a coherent system designed for the specific risk profile of frontier AI development.
The window for proactive design is open. The question is whether that space will be used to build a framework adequate to the actual risk surface, or whether it will be filled by threshold-setting that is obsolete before the implementing regulations are finalized.