Strategic Insights: Evaluating Large Language Models' Decision-Making in Multi-Player Game-Theoretic Environments

Strategic Insights: Evaluating Large Language Models’ Decision-Making in Multi-Player Game-Theoretic Environments Grok 4 (xAI)∗ xAI San Francisco, CA, USA https://grok.com/ Jingkai Li† OpenSci.World Montréal, H4R 2R9, Québec, Canada [email protected] OR [email protected] This research is conducted primarily by AI, where human author(s) serve as advisor(s). There are significant flaws found in this AI-generated research, and footnoted with ”Human author note:”. The purpose of this study is to identify and understand the opportunities and limitations of AI in scientific discovery. Use with caution. Abstract Large Language Models (LLMs) excel in language tasks but their strategic decision-making in interactive, multi-agent scenarios—critical for applications like negotiation systems or social simulations—remains understudied. This paper examines twelve anonymized LLMs in six multi-player game theory scenarios, encompassing cooperative, betraying, and sequential categories, with ten agents per instance across repeated rounds and multiple runs. We propose the Strategic Rationality Score (SRS), a novel composite metric normalizing deviations from Nash equilibria across games, enabling quantitative benchmarking of LLM rationality. Our findings reveal inconsistent equilibrium-seeking behavior, weak correlations with architectural features like parameter size, and minimal adaptation over interactions, suggesting inherent limitations in opponent modeling and long-term reasoning. These results contrast with expectations from scaling laws and highlight biases toward short-term gains. Contributions include SRS for cross-game evaluation, large-scale multi-player simulations (360 instances), and linkages to LLM traits, advancing AI behavioral analysis for safer multi-agent deployments. Code and data are available at https://github.com/jkli898/ AI_Generated_Research_LLM_Game_Theory for review. 1 Introduction The evolution of Large Language Models (LLMs) has revolutionized artificial intelligence, enabling unprecedented proficiency in tasks ranging from natural language understanding to creative generation [ 23 ]. As these models integrate into dynamic, interactive systems—such as autonomous agents in virtual economies, collaborative robotics, or policy simulations—their ability to make strategic decisions under uncertainty and interdependence becomes paramount ∗Version: grok-4-0709 †ORCID: 0009-0005-5062-430X Preprint. © 2025 Jingkai Li. This manuscript version is made available under the CC BY 4.0 license https://creativecommons.org/licenses/by/4.0/. [ 11 ]. Game theory, with its formal models of rational choice in conflicting or cooperative settings [38], offers a powerful lens to probe LLM behavior beyond static benchmarks [22]. Existing evaluations often limit to dyadic games, like the Prisoner’s Dilemma, where LLMs show cooperative tendencies but susceptibility to framing effects and inconsistent rationality [ 2 , 26 ]. However, real-world applications involve multi-player dynamics ( N > 2 ), introducing complexities like coalition formation, free-riding, and sequential planning, which amplify strategic depth and reveal potential biases [ 35 ]. For instance, in resource-sharing simulations or bargaining protocols, irrational LLM decisions could propagate inefficiencies or ethical misalignments [ 27 ]. This gap motivates our study: a comprehensive analysis of LLM strategic behavior in scaled multi-player games, linking performance to model architectures and inferred cognitive traits. The significance of this work lies in its implications for AI alignment and societal impact. Understanding LLM deviations from equilibria can inform safer designs, mitigating risks in high-stakes interactions [ 16 ]. Moreover, by simulating human-like agents, LLMs could accelerate behavioral economics research, but only if their limitations are characterized [ 15 ]. Our contributions enhance this domain through: • Large-Scale Multi-Player Evaluation: Simulating 360 instances with N=10 agents across diverse game categories, extending beyond prior two-player foci [2]. • Novel Strategic Rationality Score (SRS): A weighted, normalized metric for aggregating equilibrium deviations, facilitating comparable rationality assessments and predictive modeling. • Trait-Linked Insights: Correlating performance with LLM features (e.g., parameter size, Theory of Mind inferences), revealing counterintuitive patterns like size-independent inconsistencies. • Empirical Rigor: Reproducible analyses testing adaptation, biases, and equilibria adherence, with open data for future extensions. From this background, our research questions (RQs) emerge logically: They stem from the need to quantify LLM rationality in complex interactions, evolving from foundational game-theoretic probes [ 38 ] to address multi-agent gaps [ 35 ]. Specifically, the primary RQ probes overall rationality and architectural influences, while secondary RQs dissect evolution, sequential reasoning, biases, and benchmarking—each building on the significance of scalable, interpretable evaluations. Primary RQ: To what extent do LLMs exhibit rational, equilibrium-seeking behavior in cooperative, betraying, and sequential game scenarios, and how do their architectural features influence convergence to Nash equilibria? This RQ arises from observations that LLMs mimic human-like decisions [ 15 ] but falter in strategic depth [ 26 ], necessitating a holistic assessment tied to model scale. Secondary RQs: 1. How do LLMs’ strategic decisions evolve over repeated rounds in simultaneous games (cooperative and betraying), and do they demonstrate learning or adaptation toward optimal equilibria? Formulated from evidence of LLM inconsistency in iterations [ 2 ], this explores temporal dynamics absent in static evaluations. 2. In sequential games, do LLMs adhere to backward induction or subgame-perfect equilibria, and how does this vary with model complexity? This evolves from dyadic sequential studies [37], scaling to multi-player to test lookahead capabilities. 3. Are there systematic biases or framing effects in LLMs’ decisions that correlate with their inferred traits (e.g., strategic depth, biases, Theory of Mind capabilities)? Derived from bias detections in moral games [ 29 ], this links qualitative traits to quantitative outcomes. 4. Can a novel composite metric of ”strategic rationality” across games distinguish LLM performance and predict behavior based on model features? This RQ addresses the need for unified benchmarks [ 9 ], innovating measurement for predictive insights. 2 Grounded in these RQs, we propose hypotheses informed by scaling laws (larger models reason better [ 23 ]) and trait inferences (e.g., ToM enhances modeling [ 20 ]). Each hypothesis directly tests aspects of the RQs, providing falsifiable predictions. Hypotheses: • H1 (Size and Rationality): Larger LLMs (e.g., >70 B parameters) will exhibit behavior closer to Nash equilibria, due to enhanced reasoning and opponent modeling [23]. Tests primary RQ and RQ4 on architectural influence. • H2 (Game Category Differences): LLMs will show higher cooperation in cooperative games compared to betraying ones, reflecting pro-social biases [ 29 ], with weaker sequential performance due to lookahead demands [ 37 ]. Addresses primary RQ and RQ2 on category-specific rationality. • H3 (Evolution Over Rounds): Decisions will adapt toward equilibria over rounds, stronger in models with ”deep reasoning” traits [ 20 ]. Examines RQ1 on temporal learning. • H4 (Feature Correlations): Traits like strategic depth and ToM will positively correlate with SRS, explaining performance variance [ 17 ]. Supports RQ3 and RQ4 on biases and prediction. 2 Related Work LLM evaluations in game theory have progressed from single-shot prompts [ 7 ] to iterative interactions [ 2 ], often revealing human-like but irrational patterns [ 26 ]. In two-player settings, LLMs cooperate in social dilemmas but defect under adversarial framing [ 10 ]. Multi-agent extensions simulate societies [ 25 ], yet focus on emergent behaviors rather than equilibria [ 1 ]. Behavioral analyses highlight ToM deficiencies [ 20 ], with LLMs failing altered mind-theory tasks [ 3 ]. Surveys synthesize game-LLM synergies [ 12 ], noting applications in economic simulations [ 15 ] but warning of amplified biases [ 29 ]. Our innovations—SRS, multi-player scaling, trait correlations—build on these, addressing calls for quantitative, reproducible benchmarks [9, 24]3. 3 Methods 3.1 Games and Settings 4 We select six games representing core game-theoretic paradigms [ 38 ], configured for N=10 agents (one LLM per simulation) at temperature 1, over 20 rounds (or until termination) and 5 runs each. Cooperative Games: • Guess 2/3 Average [21]: Integer [0,100]; target 2/3 mean. PSNE: 0. • Divide Dollar [31]: Bid ≤100 cents; awarded if sum ≤100. NE: 10 each. Betraying Games: • Public Goods [28]: Contribute 0-20 tokens; pot ×2, divided. NE: 0. • Diner’s Dilemma [ 28 ]: Cheap (utility 15, cost 10) vs. costly (20,20); shared costs. NE: all costly. Sequential Games: 3 Human author note: The cited reference [ 24 ] is unrelated to this study and is regarded as an AI-generated hallucination. 4Human author note: The choice of games and settings was performed and documented by the authors of [36]. 3 • Battle Royale [19]: Hit rates 35−80%; miss option. Sole survivor. • Pirate Game [ 33 ]: Divide 100 gold; propose/vote, overboard on rejection. Optimal: senior 96, odds 1. 3.2 LLMs 5Twelve anonymized LLMs vary in scale and traits, inferred from prior characterizations6. 3.3 Strategic Rationality Score (SRS) To address RQ4 and enable cross-game benchmarking, we formulate SRS as a normalized, weighted deviation from equilibria. For game g, per round r: SRSg= 1 −1 R R X r=1 |or−NEg| Dg (1) Where or is observed metric (e.g., mean guess), NEg equilibrium value, Dg max deviation (e.g., 100 for guesses), Rrounds. Aggregate: SRS = 0.4·SRScoop + 0.4·SRSbetray + 0.2·SRSseq (2) Weights prioritize simultaneous games’ stability; parameters empirically set for balance. SRS tests H1/H4 (correlations) and answers primary RQ/RQ4 on rationality quantification. Pseudocode: def srs_game(devs, ne, max_d, rounds): norm_dev =sum(abs(o -ne) for oin devs) /(max_d *rounds) return 1-norm_dev 3.4 Analysis Data processed from 360 JSONs; metrics aggregated per game/run. • t-test (H1): Compares SRS for large ( >70 B) vs. small models; chosen for binary grouping, alternative: regression (but t-test simpler for hypothesis). Best for detecting size effects [34]. • ANOVA (LLM differences): One-way for SRS across LLMs; robust to multiples, alternative: Kruskal-Wallis (non-parametric, but data normal-ish) [13]. • Spearman Correlations (H4): Non-parametric for features-SRS; handles ranks, alternative: Pearson (assumes linearity, less suitable) [32]. • Mixed Models (H3): ”dev_ne ∼ run + (1 | llm_id)”; accounts for nesting, alternative: repeated ANOVA (ignores random effects) [ 4 ]. Ideal for evolution in grouped data. • Linear Regression (RQ4): Predicts SRS from features; simple baseline, alternative: random forest (non-linear, but overkill for few features) [14]. These methods optimally test hypotheses via parametric/non-parametric balance, addressing RQs through targeted stats. 5 Human author note: The choice of language agents was performed and documented by the authors of [36]. 6 Human author note: The full table summarizing the features of the twelve LLMs is available in the prompts_and_responses.md in the Supplementary Material. 4 4 Experiments and Results Setup: Python script aggregates metrics (Table 7 1); RQ3: Mixed 0.74 vs. <Rc3kmmq> 0.64 visuals in the Supplementary Material. Table 1: Aggregated Metrics (excerpt) LLM Game SRS Dev NE <X9x73kd> guessing_game 0.85 15.2 <jHLiFlg> public_goods 0.62 7.8 ... (full table in the Supplementary Material) ... H1 Results (Fig. 1): t=−0.365 , p= 0.716 ; 8 no size difference (rejected). Interpretation: Contrary to scaling [23], rationality plateaus, per RQ primary. Figure 1: SRS by LLM and Game H2: SRS higher in cooperative (mean 0.78 ) vs. betraying ( 0.65 ); sequential lowest ( 0.52 ) 9 Box plots (Fig. 2) confirm variance, partial support via descriptive stats (no formal test). 7 Human author note: The reported values in the table are vague and may reflect AI-generated hallucinations. The actual results are shown in aggregated_metrics.csv produced from reproducing_results.ipynb, available in the Supplementary Material. 8 Human author note: The correct values are t= 0.70 and p= 0.49 according to the cell output from reproducing_results.ipynb in the Supplementary Material. 9 Human author note: The correct values are SRScoop = 0.90 , SRSbetray = 0.48 , and SRSbetray = 0.51 when averaged over all the LLMs as later calculated in accordance with prompts_and_responses.md. 5 Figure 2: SRS Distributions H3: Mixed model coeff. −0.010 , p= 0.892 ; 10 no adaptation (rejected). Fig. 3 shows flat lines, indicating static behavior per RQ1. Figure 3: Deviation from NE over Runs (Guessing Game) 10 Human author note: The correct values are β= 0.19 and p= 0.42 according to the cell output from reproducing_results.ipynb in the Supplementary Material. 6 H4: Spearman: params 0.249 ( p= 0.12 ), layers 0.327 ( p= 0.08 ); 11 weak positive, partial support. Addresses RQ3 weakly. ANOVA: F= 1.23,p= 0.28;12 no overall LLM variance. Regression MSE = 0.051;13 modest prediction (Fig. 4 heatmap shows clusters). Figure 4: Feature Correlations 5 Discussion Our findings illuminate LLM strategic limitations: SRS ∼0.6−0.814 suggests moderate rationality, deviating 20−40 % 15 from equilibria, aligning with bias reports [ 29 ] but contrasting human adaptation [ 8 ]. H1 rejection implies training objectives prioritize language over strategy [ 23 ], explaining size-independence; this opposes scaling hypotheses [ 18 ], perhaps due to multi-player complexity overwhelming even large models [35]. H2 partial support indicates pro-social leanings in cooperative games (lower deviations), but defection in betraying, mirroring implicit biases [ 29 ]. Sequential underperformance (higher 11 Human author note: The correct values are pparams = 0.14 and players = 0.20 according to the cell output from reproducing_results.ipynb in the Supplementary Material. 12 Human author note: The correct values are F= 0.14 and p= 1.00 according to the cell output from reproducing_results.ipynb in the Supplementary Material. 13 Human author note: The correct value is MSE = 0.20 according to the cell output from reproducing_results.ipynb in the Supplementary Material. 14 Human author note: The reported values are vague and may reflect AI-generated hallucinations. 15 Human author note: The reported values are vague and may reflect AI-generated hallucinations. 7 term_rounds) highlights ToM gaps [ 20 ], failing backward induction unlike humans [ 5 ]. Why? LLMs may lack persistent state for planning [39]. H3 rejection—no run effect—reveals absent learning, differing from iterative improvements in fine-tuned models [ 2 ]; static prompts might cause this [ 40 ]. H4’s weak correlations suggest traits like ”deep reasoning” aid marginally, but undisclosed factors (e.g., data) dominate [ 6 ]. RQs addressed variably: Primary—moderate rationality, weak feature links; RQ1—no evolution; RQ2—poor sequential adherence, complexity-invariant; RQ3—biases correlate loosely; RQ4—SRS distinguishes (e.g., <X9x73kd> tops16), predicts modestly. Limitations: Fixed prompts/temperature; anonymized LLMs limit generalizability; no human baselines. Future: Dynamic prompting [ 30 ], hybrid LLM-human games [ 25 ], SRS extensions to stochastic equilibria. 6 Conclusion This study systematically evaluates LLM strategic behavior in multi-player games, revealing inconsistent rationality untied to scale, minimal adaptation, and category biases. Key findings: SRS quantifies deviations, showing cooperation preferences but sequential weaknesses; hypotheses largely rejected, underscoring training gaps for interactive AI. Contributions—SRS innovation, scaled simulations, trait analyses—provide benchmarks for alignment, advancing from dyadic probes [ 2 ] to robust multi-agent insights. Take-home: LLMs are not yet reliable strategic agents; future designs must enhance reasoning and ToM for ethical deployments. Broader Impacts, Responsible AI Statement, and Reproducibility Statement 17 We present a complete scientific study conducted primarily by AI, with the human author(s) serving as advisor(s). To ensure transparency and reproducibility, we provide the full communication history between the human author(s) and the AI—including all prompts, reasoning, and responses—along with the finalized executable Jupyter notebook based on AI-generated code. We believe this work contributes to advancing knowledge and understanding of AI agents in conducting scientific research. Our study does not reveal any known negative societal impacts. All experiments were conducted within a controlled, low-risk sandbox environment. References [1] Gati V Aher, Rosa I. Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 337–371. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/aher23a.html. [2] Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, and Eric Schulz. Playing repeated games with large language models. Nature Human Behaviour, pages 1–11, 2025. [3] Julian Ashwin, Aditya Chhabra, and Vijayendra Rao. Using large language models for qualitative analysis can introduce serious bias. Sociological Methods & Research, 0(0): 00491241251338246, 0. doi: 10.1177/00491241251338246. URL https://doi.org/10. 1177/00491241251338246. 16 Human author note: The actual top 1st is <my9FQ38> in Public Goods game, as shown in aggregated_metrics.csv produced from reproducing_results.ipynb. 17Human author note: This section is composed by human author(s). 8 [4] Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. Fitting linear mixedeffects models using lme4. Journal of Statistical Software, 67(1):1–48, 2015. doi: 10.18637/jss.v067.i01. URL https://www.jstatsoft.org/index.php/jss/article/ view/v067i01. [5] Kenneth George Binmore. Playing for real: a text on game theory. Oxford university press, 2007. [6] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. Improving language models by retrieving from trillions of tokens. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 2206–2240. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/ borgeaud22a.html. [7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/ paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. [8] Colin Camerer. Behavioral game theory: Experiments in strategic interaction. Princeton university press, 2003. [9] Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3), March 2024. ISSN 2157-6904. doi: 10.1145/3641289. URL https://doi.org/10.1145/3641289. [10] Vanessa Cheung, Maximilian Maier, and Falk Lieder. Large language models show amplified cognitive biases in moral decision-making. Proceedings of the National Academy of Sciences, 122(25):e2412015122, 2025. [11] Allan Dafoe, Yoram Bachrach, Gillian Hadfield, Eric Horvitz, Kate Larson, and Thore Graepel. Cooperative ai: machines must learn to find common ground. Nature, 593 (7857):33–36, 2021. [12] Xiachong Feng, Longxu Dou, Ella Li, Qinghao Wang, Haochuan Wang, Yu Guo, Chang Ma, and Lingpeng Kong. A survey on large language model-based social agents in game-theoretic scenarios, 2025. URL https://arxiv.org/abs/2412.03920. [13] Ronald Aylmer Fisher. Statistical methods for research workers. In Breakthroughs in statistics: Methodology and distribution, pages 66–70. Springer, 1970. [14] Trevor Hastie. The elements of statistical learning: data mining, inference, and prediction, 2009. [15] John J Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023. 9