Lifelong Learning of Large Language Model based Agents: A Roadmap

Welcome to the repository accompanying our survey paper on Lifelong Learning of Large Language Model based Agents: A Roadmap. This repository collects awesome paper for lifelong learning (also known as, continual learning and incremental learning) of LLM agent. We identify three key modules-Perception, Memory, and Action-that are integral to agent's ability to perform lifelong learning. Please refer to this survey for detailed introduction. Additionally, for other papers, surveys, and resources on lifelong learning (continual learning, incremental learning) of LLMs, you can refer to this repository. A chinese version of this README is provided in this file.

📢 News

• 2025.01: The interpretation of this survey is available on PaperWeekly and 知乎!

• 2025.01: We released a survey paper "Lifelong Learning of Large Language Model based Agents: A Roadmap". Feel free to cite or open pull requests.

📒 Table of Contents

Perception Module

Single-Modal Perception

Title	Venue	Date
AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agent	arXiv	2024-10
GPT-4V(ision) is a Generalist Web Agent, if Grounded	ICLR	2024-01
Webarena: A realistic web environment for building autonomous agents	ICLR	2023-07
Synapse: Trajectory-asexemplar prompting with memory for computer control	ICLR	2023-06
Multimodal web navigation with instruction-finetuned foundation models	ICLR	2023-05

Multi-Modal Perception

Title	Venue	Date
Llms can evolve continually on modality for x-modal reasoning	NeurIPS	2024-10
Modaverse: Efficiently transforming modalities with llms	CVPR	2024-01
Omnivore: A single model for many visual modalities	CVPR	2022-01
Perceiver: General perception with iterative attention	ICML	2021-07
Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text	NeurIPS	2021-04

Memory Module

Working Memory

Title	Venue	Date
Character-llm: A trainable agent for role-playing	EMNLP	2023-10
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers	ICLR	2023-09
Adapting Language Models to Compress Contexts	ACL	2023-05
Critic: Large language models can self-correct with tool-interactive critiquing}	NeurIPS Workshop	2023-05
Cogltx: Applying bert to long texts	NeurIPS	2020-12

Episodic Memory

Title	Venue	Date
DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory	arXiv	2024-10
MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation	arXiv	2023-08
RET-LLM: Towards a General Read-Write Memory for Large Language Models	ICLR	2023-05
Bring Evanescent Representations to Life in Lifelong Class Incremental Learning	CVPR	2022-06
iCaRL: Incremental Classifier and Representation Learning	CVPR	2017-04

Semantic Memory

Title	Venue	Date
Fast and Continual Knowledge Graph Embedding via Incremental LoRA	IJCAL	2024-07
PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval	arXiv	2024-06
CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks	arXiv	2024-02
Continual Multimodal Knowledge Graph Construction	IJCAI	2023-05
Lifelong embedding learning and transfer for growing knowledge graphs	AAAI	2022-11

Parametric Memory

Title	Venue	Date
ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA	arXiv	2024-08
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models	NeurIPS	2024-05
WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing	ACL	2024-02
Aging with GRACE: Lifelong Model Editing with Key-Value Adaptors	ICLR	2022-11
Plug-and-Play Adaptation for Continuously-updated QA	ACL	2022-04

Action Module

Grounding Actions

Tool Environment

Title	Venue	Date
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error	ACL	2024-03
EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction	ICLR Workshop	2024-01
Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum	AAAI	2023-08
GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution	EACL	2023-07
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs	ICLR	2023-07
Large Language Models as Tool Makers	ICLR	2023-05
Toolformer: Language Models Can Teach Themselves to Use Tools	NeurIPS	2023-05
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings	NeurIPS	2023-05
On the Tool Manipulation Capability of Open-source Large Language Models	arXiv	2023-05
ART: Automatic multi-step reasoning and tool-use for large language models	arXiv	2023-03

Web Environment

Title	Venue	Date
AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agent	arXiv	2024-10
WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration	arXiv	2024-08
SteP: Stacked LLM Policies for Web Actions	COLM	2024-07
LASER: LLM Agent with State-Space Exploration for Web Navigation	NeurIPS Workshop	2023-09
Large Language Models Are Semi-Parametric Reinforcement Learning Agent	NeurIPS	2023-06

Game Environment

Title	Venue	Date
VillagerAgent: A Graph-Based Multi-Agent Framework for Coordinating Complex Task Dependencies in Minecraft	ACL	2024-06
See and Think: Embodied Agent in Virtual Environment	ECCV	2023-11
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models	TPAMI	2023-11
Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	2023-05
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents	NeurIPS	2023-02

Retrieval Actions

Retrieval from Semantic memory

Title	Venue	Date
See and Think: Embodied Agent in Virtual Environment	ECCV	2023-11
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory	arXiv	2023-05
Planning with Large Language Models via Corrective Re-prompting	NeurIPS Workshop	2022-10
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents	ICML	2022-01

Retrieval from Episodic memory

Title	Venue	Date
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs	arXiv	2024-06
Large Language Models as Tool Makers	ICLR	2023-05
On the Tool Manipulation Capability of Open-source Large Language Models	arXiv	2023-05
Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	2023-05
ART: Automatic multi-step reasoning and tool-use for large language models	arXiv	2023-03

Reasoning Actions

Intra-Episodic Memory

Title	Venue	Date
Reasoning with Language Model is Planning with World Model	EMNLP	2023-05
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning	NeurIPS	2023-05
Tree of Thoughts: Deliberate Problem Solving with Large Language Models	NeurIPS	2023-05
SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks	NeurIPS2023	2023-05
Reflexion: Language Agents with Verbal Reinforcement Learning	NeurIPS	2023-03
ReAct: Synergizing Reasoning and Acting in Language Models	ICLR	2022-10

Inter-Episodic Memory

Title	Venue	Date
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs	arXiv	2024-06
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error	ACL	2024-03
See and Think: Embodied Agent in Virtual Environment	ECCV	2023-11
Large Language Models Are Semi-Parametric Reinforcement Learning Agent	NeurIPS	2023-06
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory	arXiv	2023-05
Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	2023-05

📚 Cite Our Work

@article{zheng2025lifelong,
      title={Lifelong Learning of Large Language Model based Agents: A Roadmap}, 
      author={Zheng, Junhao and Shi, Chengming and Cai, Xidi and Li, Qiuke and Zhang, Duzhen and Li, Chenxing and Yu, Dong and Ma, Qianli},
      journal={arXiv preprint arXiv:2501.07278},
      year={2025},
}

😍 Star History