Executive Summary
This paper investigates the inconsistency of Language Models (LMs) in free-form decision-making within military crisis simulations, similar to wargames conducted by the US military. It quantifies inconsistency using a BERTScore-based metric, revealing that all five tested LMs exhibit significant semantic differences in their responses, even under varying conditions. The study highlights that prompt sensitivity can induce greater inconsistency than temperature sampling, urging caution for LM deployment in high-stakes military contexts.
Why It Matters
This document is crucial for defense and geopolitical analysts as it directly assesses the reliability and consistency of AI decision-making in high-stakes military scenarios, informing policy on AI integration into national security.
Key Takeaways
- Language Models exhibit significant inconsistency in free-form military crisis simulations, generating semantically different responses.
- Inconsistency due to subtle prompt variations (prompt sensitivity) can be more pronounced than that caused by temperature sampling.
- The study recommends extreme caution before deploying LMs for military decision-making due to the high volatility and unpredictability of their responses.
- Anonymizing country names did not significantly impact LM inconsistency, suggesting the issue is inherent to the models rather than specific geopolitical biases.
Strategic Relevance
The strategic relevance lies in evaluating the trustworthiness and predictability of AI systems for military applications. Inconsistent AI decision-making could lead to unpredictable and potentially escalatory outcomes in real-world conflicts, impacting national security and international stability. Understanding these limitations is critical for developing robust AI governance and deployment strategies in defense.