Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution

Cheng Qian,Shihao Liang,Yujia Qin,Yining Ye,Xin Cong,Yankai Lin,Yesai Wu,Zhiyuan Liu,Maosong Sun

2024-01-25

Abstract:This paper introduces Investigate-Consolidate-Exploit (ICE), a novel strategy for enhancing the adaptability and flexibility of AI agents through inter-task self-evolution. Unlike existing methods focused on intra-task learning, ICE promotes the transfer of knowledge between tasks for genuine self-evolution, similar to human experience learning. The strategy dynamically investigates planning and execution trajectories, consolidates them into simplified workflows and pipelines, and exploits them for improved task execution. Our experiments on the XAgent framework demonstrate ICE's effectiveness, reducing API calls by as much as 80% and significantly decreasing the demand for the model's capability. Specifically, when combined with GPT-3.5, ICE's performance matches that of raw GPT-4 across various agent tasks. We argue that this self-evolution approach represents a paradigm shift in agent design, contributing to a more robust AI community and ecosystem, and moving a step closer to full autonomy.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to improve the adaptability and flexibility of AI agents when handling new tasks through inter - task self - evolution. Existing methods mainly focus on intra - task learning, while this paper proposes a new strategy - Investigate - Consolidate - Exploit (ICE), aiming to promote knowledge transfer between different tasks, thereby achieving true self - evolution, similar to the human experience - learning process. The ICE strategy dynamically investigates planning and execution trajectories, simplifies them into workflows and pipelines, and utilizes them to improve task execution efficiency.

Specifically, the paper proposes the following points to solve the problem:

1. **Cross - task learning**: Different from existing methods, the ICE strategy emphasizes cross - task knowledge transfer, which helps agents not only perform well on a single task but also apply their experience to new tasks, improving their overall performance.

2. **Dynamic investigation**: The ICE strategy identifies experiences worthy of learning and reference by dynamically tracking the planning and execution processes of tasks. This includes monitoring the generation of initial plans, any plan revisions, and the status of each goal or sub - goal.

3. **Experience consolidation**: The ICE strategy standardizes successful plans and execution trajectories to form workflows and pipelines that are easy to reuse in the future. This involves removing failed goals or sub - goals and converting successful goals into a linear structure for easy learning and reference.

4. **Experience exploitation**: The ICE strategy improves the planning and execution efficiency of new tasks by retrieving and exploiting past - consolidated experiences. This includes retrieving plans for similar goals during initial plan generation and plan revision, and directly applying relevant pipelines when executing new tasks.

Through these strategies, the paper demonstrates the effectiveness of the ICE method, especially in reducing the number of API calls, lowering the requirements for model capabilities, and increasing the task completion rate. Experimental results show that the ICE strategy can significantly improve the task execution efficiency and the rationality of plan formulation of agents, and can even achieve performance comparable to that of stronger models (such as GPT - 4) when using weaker models (such as GPT - 3.5).