Executive Summary
This RAND report analyzes the conditions under which rival states can credibly and sustainably restrain their development of Artificial General Intelligence (AGI) through a game-theoretic model. It identifies the strategic requirements for mutual restraint, focusing on verification, domestic oversight, and the challenges of coordination. The analysis explains why rivals might fail to build necessary oversight capacity even when both prefer restraint.
Why It Matters
This document is crucial for defense and geopolitical analysts as it models the strategic challenges and conditions for international cooperation or competition in the development of AGI, a technology with profound national security implications.
Key Takeaways
- Expected catastrophic harm from AGI can reduce the need for formal verification in restraint agreements, as the risk itself acts as a self-deterrent.
Source evidence · PDF page 12
Expected catastrophic harm can lower the amount of formal verification required to sustain restraint. If a state expects to bear some of the risk that it creates through a defection, the defection is partly self-deterring.
- Shorter timelines for AGI arrival weaken the effectiveness of deterrence based on future cooperation benefits, necessitating punishments tied to post-AGI outcomes.
Source evidence · PDF page 12
Short timelines weaken enforcement unless punishment occurs after AGI’s arrival. In the model, the punishment for defection is the loss of the future benefit of cooperation. A state that cheats today loses the cooperative relationship tomorrow.
- Mutual doubt between rival states can prevent the establishment of necessary domestic oversight infrastructure for AGI restraint, even when both sides would prefer it.
Source evidence · PDF page 21
The coordination problem arises only when the increment that states need to add is actually required for the veri-fication regime to hold. If unilateral collection and the oversight that states maintain for domestic purposes already deter the entire covered menu of defections, the required additional oversight is zero, and there is nothing on which states must coordinate.
Strategic Relevance
The report provides a framework for understanding the strategic dynamics of AGI development among rival states, highlighting the critical role of verification, domestic oversight, and the timing of AGI arrival in achieving mutual restraint. It informs policy discussions on arms control and strategic stability in the age of advanced AI.