Date of Award
Spring 6-4-2025
Document Type
Thesis (Undergraduate)
Department
Computer Science
First Advisor
Soroush Vosoughi
Abstract
Strategic communication capabilities in artificial
intelligence systems remain poorly understood, yet they carry
profound implications for AI safety and human-AI interaction
as these systems become more prevalent in social environments.
This research investigates how large language models respond to
strategic instruction by developing a comprehensive simulation
framework where GPT-4 agents play Mafia across five experi-
mental variants. We introduce novel automated evaluation met-
rics to quantify AI deception and persuasion capabilities—two
fundamental dimensions of strategic communication that have
lacked systematic measurement in AI systems.
Our findings reveal that explicit strategic guidance transforms
AI agent behavior dramatically. Mafia win rates increased from
60.0% to 80.0% when agents received strategic instruction,
demonstrating that LLMs exhibit substantial responsiveness
to strategic direction. Through systematic analysis of differ-
ent strategic configurations, we discovered that asymmetric
guidance—where only one team member receives strategic in-
struction—produces optimal results by balancing coordinated
manipulation with natural communication patterns. Qualitative
analysis of agent communications identified distinct linguistic
markers of successful strategic behavior, including increased
emotional framing, unity-building language, and meta-strategic
commentary.
These results provide a systematic quantitative framework
for evaluating AI strategic communication and demonstrate that
current LLMs can effectively simulate complex social deception
when provided with appropriate guidance. Our findings offer
important insights into AI agent capabilities in competitive
environments and suggest the need to consider safeguards as
these capabilities are further developed and deployed.
Recommended Citation
Veeramachaneni, Goutham; Ma, Weicheng; and Vosoughi, Soroush, "Strategy in AI-Simulated Social Deduction Games: An Analysis of GPT-4 Agents in Mafia" (2025). Computer Science Senior Theses. 87.
https://digitalcommons.dartmouth.edu/cs_senior_theses/87
