Abstract:The AI community has been exploring a pathway to artificial general intelligence (AGI) by developing "language agents", which are complex large language models (LLMs) pipelines involving both prompting techniques and tool usage methods. While language agents have demonstrated impressive capabilities for many real-world tasks, a fundamental limitation of current language agents research is that they are model-centric, or engineering-centric. That's to say, the progress on prompts, tools, and pipelines of language agents requires substantial manual engineering efforts from human experts rather than automatically learning from data. We believe the transition from model-centric, or engineering-centric, to data-centric, i.e., the ability of language agents to autonomously learn and evolve in environments, is the key for them to possibly achieve AGI. In this work, we introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own in a data-centric way using symbolic optimizers. Specifically, we consider agents as symbolic networks where learnable weights are defined by prompts, tools, and the way they are stacked together. Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning: back-propagation and gradient descent. Instead of dealing with numeric weights, agent symbolic learning works with natural language simulacrums of weights, loss, and gradients. We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks and show that agent symbolic learning enables language agents to update themselves after being created and deployed in the wild, resulting in "self-evolving agents".

What problem does this paper attempt to address?

The problem that this paper attempts to solve is a fundamental limitation in the current development of language agents, that is, the construction and customization of these agent systems require a great deal of manual engineering efforts. Specifically, researchers and developers must manually decompose complex tasks into more manageable subtasks (called nodes), and then carefully design the prompts and tools for each node, including API functions, knowledge bases, memories, etc. The complexity of this process makes current language agent research tend to be model - centered or engineering - centered, which means that it is almost impossible for researchers to optimize or tune language agents in a data - centered way like training neural networks. This limits the robustness and universality of manually - coded language agents and requires a great deal of engineering effort to adapt language agents to new tasks or data distributions. The paper proposes that the shift from engineering - centered language agent development to data - centered learning is an important step in language agent research. To this end, the authors introduce a new framework - agent symbolic learning, aiming to enable language agents to self - optimize in a data - centered way. By imitating two basic algorithms in connectionist learning - back - propagation and gradient descent, the agent symbolic learning framework can achieve a similar optimization process in agents, but it deals with weights, losses and gradients in the form of natural language rather than numerical form. This framework can not only optimize individual components (such as the prompts of specific nodes), but also jointly optimize all symbolic components within the agent system, including prompts, tools and the way they are combined into the agent system, thus avoiding local optimal solutions and improving the performance of the entire agent system. Through a series of experiments, the paper demonstrates the effectiveness of the agent symbolic learning framework, especially in standard benchmark tests and complex real - world tasks, such as software development and creative writing, proving the potential of this framework in optimizing and designing prompts and tools, and in updating the entire agent pipeline through learning training data. This opens up the possibility for language agent research to shift from engineering - centered to data - centered, enabling language agents for complex real - world problems to effectively learn from data and achieve "self - evolving agents".

Symbolic Learning Enables Self-Evolving Agents

Learning to Model the World with Language

Multi-Agent Language Learning: Symbolic Mapping

Converging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Empowered Autonomous Agents

SymbolicAI: A framework for logic-based approaches combining generative models and solvers

Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization

Augmenting deep neural networks with symbolic knowledge: Towards trustworthy and interpretable AI for education

Synthesizing Evolving Symbolic Representations for Autonomous Systems

Neurosymbolic AI for Enhancing Instructability in Generative AI

Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

From Language Models to Practical Self-Improving Computer Agents

Training Language Model Agents without Modifying Language Models

Agents: An Open-source Framework for Autonomous Language Agents

Large Language Models Are Neurosymbolic Reasoners

Softened Symbol Grounding for Neuro-symbolic Systems

Self-evolving Agents with reflective and memory-augmented abilities

Understanding Early Word Learning in Situated Artificial Agents

Symbolic Learning to Optimize: Towards Interpretability and Scalability

DANLI: Deliberative Agent for Following Natural Language Instructions

Policy Learning with a Language Bottleneck