Input Conditioned Graph Generation for Language Agents

Lukas Vierling,Jie Fu,Kai Chen

2024-06-17

Abstract:Recent progress in Large Language Models (LLMs) and language agents has demonstrated significant promise for various future applications across multiple disciplines. While traditional approaches to language agents often rely on fixed, handcrafted designs, our research aims to develop both learnable and dynamic agents. Our method uses an existing framework that abstracts language agents as graphs. Within this graph framework, we aim to learn a model that can generate edges for every given input to the language agent. This allows us to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent. We learn to generate these edges using a pretrained LLM that is fine-tuned with reinforcement learning. This LLM can be fine-tuned on several datasets simultaneously, and we hypothesize that the model learns to adapt to these different domains during training, achieving good overall performance when encountering data from different domains during deployment. We demonstrate that our approach surpasses the previous static approach by nearly 6% accuracy on a combined dataset of MMLU and CMMLU, and by more than 10% when trained with a sparsity-inducing loss. It also performs superior in additional experiments conducted with the MMLU and Mini Crossword Puzzles datasets. The code is available at <a class="link-external link-https" href="https://github.com/lukasVierling/DynamicGPTSwarm" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

This paper presents a new method for optimizing the generation of edges in the graph framework of language agents to adapt to different inputs. Traditional language agents typically rely on fixed manual design, while this research aims to develop a learnable and dynamic agent. The authors utilize an existing framework that abstracts language agents as Directed Acyclic Graphs (DAGs) and learn a model to generate edges in the graph based on the given input, thereby adjusting the internal communication of the language agent. By using a pre-trained large-scale language model (LLM) and reinforcement learning fine-tuning, the model can learn from multiple datasets and adapt to tasks in different domains.

The main contributions of this paper include:
1. Proposing a method for optimizing edges based on input conditions, allowing the dynamic generation of graph structure based on the input.
2. Through experiments, it is demonstrated that this method achieves an accuracy improvement of nearly 6% over static methods on the MMLU and CMMLU comprehensive datasets and an improvement of over 10% when using sparsity-induced loss.
3. In additional experiments such as fill-in-the-blank games and datasets combining MMLU and CMMLU, the dynamic method outperforms static methods.

The paper also compares the static edge probabilities with input-conditioned edge probabilities, indicating that dynamically adjusting communication strategies can enhance the performance and adaptability of language agents. Furthermore, the paper explores the differences between role-based language agents and dynamic language agents and emphasizes the importance of adapting to input variations for the system to handle different types of data.

In summary, this paper addresses the problem of optimizing the internal communication of language agents by learning and adapting to inputs to improve their performance in various tasks.