A Survey on Large Language Model-Based Game Agents

🔥 Must-read papers for LLM-based Game agents.

💫 Continuously update on a weekly basis. (last update: 2024/04/03)

Content

• A Survey on Large Language Model-Based Game Agents
  • Text Adventure Games
  • Crafting & Exploration Games
  • Simulation Games
  • Competition Games
  • Cooperation Games
  • Communication Games (Conversational) Games
  • Action Games

Text Adventure Games

• [2019/09] Interactive Fiction Games: A Colossal Adventure AAAI 2020 [paper] [code]
• [2020/10] ALFWorld: Aligning Text and Embodied Environments for Interactive Learning ICLR 2021 [paper][code]
• [2022/03] ScienceWorld: Is your Agent Smarter than a 5th Grader? EMNLP 2022 [paper] [code]
• [2022/10] ReAct: Synergizing Reasoning and Acting in Language Models ICLR 2023 [paper] [code]
• [2023/02] Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning ICML 2023 [paper][code]
• [2023/03] Reflexion: Language Agents with Verbal Reinforcement Learning NeurIPS 2023 [paper] [code]
• [2023/04] Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions arXiv [paper]
• [2023/05] SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks NeurIPS 2023 [paper] [code]
• [2023/11] ADaPT: As-Needed Decomposition and Planning with Language Models arXiv [paper][code]
• [2024/02] Soft Self-Consistency Improves Language Model Agents arXiv [paper][code]
• [2024/02] Empowering Large Language Model Agents through Action Learning arXiv [paper][code]
• [2024/03] KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents arXiv [paper][code]
• [2024/03] Language Guided Exploration for RL Agents in Text Environments arXiv [paper][code]

Crafting & Exploration Games

MineCraft

• [2023/02] Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents NeurIPS 2023 [paper][code]
• [2023/03] Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks FMDM@NeurIPS2023 [paper][code]
• [2023/05] Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory arXiv [paper]
• [2023/05] VOYAGER: An Open-Ended Embodied Agent with Large Language Models FMDM@NeurIPS2023 [paper][code]
• [2023/10] LLaMA Rider: Spurring Large Language Models to Explore the Open World arXiv [paper][code]
• [2023/10] Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds ICLR 2024 [paper]
• [2023/11] JARVIS-1: Open-world Multi-task Agents with Memory-Augmented Multimodal Language Models arXiv [paper][code]
• [2023/12] Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft arXiv [paper]
• [2023/12] Creative Agents: Empowering Agents with Imagination for Creative Tasks arXiv [paper][code]
• [2024/02] RL-GPT: Integrating Reinforcement Learning and Code-as-policy arXiv [paper]

Crafter

• [2023/02] Guiding Pretraining in Reinforcement Learning with Large Language Models ICML 2023 [paper]
• [2023/05] SPRING: Studying Papers and Reasoning to play Games NeurIPS 2023 [paper]
• [2023/06] OMNI: Open-endedness via Models of human Notions of Interestingness arXiv [paper][code]
• [2023/09] AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback arXiv [paper]
• [2024/03] EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents arXiv [paper]

Simulation Games

• [2022/01] Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents ICML 2022 [paper][code]
• [2022/12] LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models ICCV 2023 [paper]
• [2023/04] Generative Agents: Interactive Simulacra of Human Behavior UIST 2023 [paper][code]
• [2023/05] Language Models Meet World Models: Embodied Experiences Enhance Language Models NeurIPS 2023 [paper][code]
• [2023/08] AgentSims: An Open-Source Sandbox for Large Language Model Evaluation arXiv [paper]
• [2023/10] Humanoid Agents: Platform for Simulating Human-like Generative Agents arXiv [paper]
• [2023/10] Lyfe Agents: Generative agents for low-cost real-time social interactions arXiv [paper]
• [2023/10] Octopus: Embodied Vision-Language Programmer from Environmental Feedback arXiv [paper] [code]
• [2024/01] CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents ICLR 2024 [paper][code]
• [2024/01] True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning arXiv[paper][code]

Competition Games

• [2022/10] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task ICLR 2023 [paper]
• [2023/06] ChessGPT: Bridging Policy Learning and Language Modeling NeurIPS 2023 [paper][code]
• [2023/08] Are ChatGPT and GPT-4 Good Poker Players?--A Pre-Flop Analysis arXiv [paper]
• [2023/09] Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4 arXiv [paper]
• [2023/12] Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach arXiv [paper][code]
• [2024/01] PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model arXiv [paper]
• [2024/01] SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models arXiv [paper]
• [2024/02] PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models arXiv [paper][code]
• [2024/02] Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization arXiv [paper][code]

Cooperation Games

• [2023/07] Building Cooperative Embodied Agents Modularly with Large Language Models ICLR 2024 [paper][code]
• [2023/09] MindAgent: Emergent Gaming Interaction arXiv [paper]
• [2023/10] Evaluating Multi-agent Coordination Abilities in Large Language Models arXiv [paper]
• [2023/12] LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination arXiv [paper]
• [2024/02] S-Agents: Self-organizing Agents in Open-ended Environments arXiv [paper]
• [2024/03] ProAgent: Building Proactive Cooperative Agents with Large Language Models AAAI 2024 [paper]
• [2024/03] Can LLM-Augmented Autonomous Agents Cooperate?, An Evaluation of Their Cooperative Capabilities through Melting Pot arXiv [paper]
• [2024/03] Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation arXiv[paper]

Communication (Conversational) Games

• [2022/12] Human-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning Science [paper]
• [2023/08] GameEval: Evaluating LLMs on Conversational Games arXiv [paper][code]
• [2023/09] Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf arXiv [paper]
• [2023/10] Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game arXiv [paper]
• [2023/10] Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation arXiv [paper]
• [2023/10] AvalonBench: Evaluating LLMs Playing the Game of Avalon FMDM@NeurIPS2023 [paper][code]
• [2023/10] LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay arXiv [paper]
• [2023/10] Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models arXiv [paper][code]
• [2023/11] War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars arXiv [paper][code]
• [2023/12] Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game arXiv [paper]
• [2023/12] Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games [paper]
• [2024/02] Enhance Reasoning for Large Language Models in the Game Werewolf arXiv [paper]
• [2024/02] What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents arXiv [paper]

Action Games

• [2024/03] Towards General Computer Control: A Multimodal Agent for Red Dead Redemption II as a Case Study arXiv [paper][code]
• [2024/03] Will GPT-4 Run DOOM? arXiv [paper][code]
• [2024/03] Evaluate LLMs in Real Time with Street Fighter III GitHub [code]

Citation

If you find this repository useful, please cite our paper:

@misc{hu2024survey,
      title={A Survey on Large Language Model-Based Game Agents}, 
      author={Sihao Hu and Tiansheng Huang and Fatih Ilhan and Selim Tekin and Gaowen Liu and Ramana Kompella and Ling Liu},
      year={2024},
      eprint={2404.02039},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

Contact

• Sihao Hu ([email protected])