Agents that Reason and Learn

J. Lloyd

DOI: https://doi.org/10.1007/978-3-540-39917-9_2

2003-09-29

Abstract:

What problem does this paper attempt to address?

AgentKit: Structured LLM Reasoning with Dynamic Graphs

Yue Wu,Yewen Fan,So Yeon Min,Shrimai Prabhumoye,Stephen McAleer,Yonatan Bisk,Ruslan Salakhutdinov,Yuanzhi Li,Tom Mitchell

2024-07-25

Abstract:We propose an intuitive LLM prompting framework (AgentKit) for multifunctional agents. AgentKit offers a unified framework for explicitly constructing a complex "thought process" from simple natural language prompts. The basic building block in AgentKit is a node, containing a natural language prompt for a specific subtask. The user then puts together chains of nodes, like stacking LEGO pieces. The chains of nodes can be designed to explicitly enforce a naturally structured "thought process". For example, for the task of writing a paper, one may start with the thought process of 1) identify a core message, 2) identify prior research gaps, etc. The nodes in AgentKit can be designed and combined in different ways to implement multiple advanced capabilities including on-the-fly hierarchical planning, reflection, and learning from interactions. In addition, due to the modular nature and the intuitive design to simulate explicit human thought process, a basic agent could be implemented as simple as a list of prompts for the subtasks and therefore could be designed and tuned by someone without any programming experience. Quantitatively, we show that agents designed through AgentKit achieve SOTA performance on WebShop and Crafter. These advances underscore AgentKit's potential in making LLM agents effective and accessible for a wider range of applications. <a class="link-external link-https" href="https://github.com/holmeswww/AgentKit" rel="external noopener nofollow">this https URL</a>

Artificial Intelligence,Machine Learning
Agents Thinking Fast and Slow: A Talker-Reasoner Architecture

Konstantina Christakopoulou,Shibl Mourad,Maja Matarić

2024-10-11

Abstract:Large language models have enabled agents of all kinds to interact with users through natural conversation. Consequently, agents now have two jobs: conversing and planning/reasoning. Their conversational responses must be informed by all available information, and their actions must help to achieve goals. This dichotomy between conversing with the user and doing multi-step reasoning and planning can be seen as analogous to the human systems of "thinking fast and slow" as introduced by Kahneman. Our approach is comprised of a "Talker" agent (System 1) that is fast and intuitive, and tasked with synthesizing the conversational response; and a "Reasoner" agent (System 2) that is slower, more deliberative, and more logical, and is tasked with multi-step reasoning and planning, calling tools, performing actions in the world, and thereby producing the new agent state. We describe the new Talker-Reasoner architecture and discuss its advantages, including modularity and decreased latency. We ground the discussion in the context of a sleep coaching agent, in order to demonstrate real-world relevance.

Artificial Intelligence,Computation and Language,Machine Learning
A Model for Learning and Acting Rationally in Multiagent System.

Yao-Hai Lin,Shan-Li Hu,Jian-Bin Luo

DOI: https://doi.org/10.1109/robio.2007.4522433

2007-01-01

Abstract:In order to characterize the agent, there is a presupposition that each agent has its set of propositions. The model is based upon our prior observation, and satisfies many properties. Furthermore, we describe concretely the desire of agent by twin-subset semantics of desire operator. If friendship for learning of two agents have been made, one agent has the authorization to increase his set of propositions rationally, namely, learning from the other one. The paper aims to make one agent collaborates rationally with the other one, and implement BDI logic in multi-agent system.
Agents: An Open-source Framework for Autonomous Language Agents

Wangchunshu Zhou,Yuchen Eleanor Jiang,Long Li,Jialong Wu,Tiannan Wang,Shi Qiu,Jintian Zhang,Jing Chen,Ruipu Wu,Shuai Wang,Shiding Zhu,Jiyu Chen,Wentao Zhang,Xiangru Tang,Ningyu Zhang,Huajun Chen,Peng Cui,Mrinmaya Sachan

2023-12-12

Abstract:Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces. We consider language agents as a promising direction towards artificial general intelligence and release Agents, an open-source library with the goal of opening up these advances to a wider non-specialist audience. Agents is carefully engineered to support important features including planning, memory, tool usage, multi-agent communication, and fine-grained symbolic control. Agents is user-friendly as it enables non-specialists to build, customize, test, tune, and deploy state-of-the-art autonomous language agents without much coding. The library is also research-friendly as its modularized design makes it easily extensible for researchers. Agents is available at <a class="link-external link-https" href="https://github.com/aiwaves-cn/agents" rel="external noopener nofollow">this https URL</a>.

Computation and Language
AGENT: A Benchmark for Core Psychological Reasoning

Tianmin Shu,Abhishek Bhandwaldar,Chuang Gan,Kevin A. Smith,Shari Liu,Dan Gutfreund,Elizabeth Spelke,Joshua B. Tenenbaum,Tomer D. Ullman

DOI: https://doi.org/10.48550/arXiv.2102.12321

IF: 14.4

2021-02-24

Artificial Intelligence

Abstract:For machine agents to successfully interact with humans in real-world settings, they will need to develop an understanding of human mental life. Intuitive psychology, the ability to reason about hidden mental variables that drive observable actions, comes naturally to people: even pre-verbal infants can tell agents from objects, expecting agents to act efficiently to achieve goals given constraints. Despite recent interest in machine agents that reason about other agents, it is not clear if such agents learn or hold the core psychology principles that drive human reasoning. Inspired by cognitive development studies on intuitive psychology, we present a benchmark consisting of a large dataset of procedurally generated 3D animations, AGENT (Action, Goal, Efficiency, coNstraint, uTility), structured around four scenarios (goal preferences, action efficiency, unobserved constraints, and cost-reward trade-offs) that probe key concepts of core intuitive psychology. We validate AGENT with human-ratings, propose an evaluation protocol emphasizing generalization, and compare two strong baselines built on Bayesian inverse planning and a Theory of Mind neural network. Our results suggest that to pass the designed tests of core intuitive psychology at human levels, a model must acquire or have built-in representations of how agents plan, combining utility computations and core knowledge of objects and physics.
Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

Pouya Pezeshkpour,Eser Kandogan,Nikita Bhutani,Sajjadur Rahman,Tom Mitchell,Estevam Hruschka

2024-02-02

Abstract:Remarkable performance of large language models (LLMs) in a variety of tasks brings forth many opportunities as well as challenges of utilizing them in production settings. Towards practical adoption of LLMs, multi-agent systems hold great promise to augment, integrate, and orchestrate LLMs in the larger context of enterprise platforms that use existing proprietary data and models to tackle complex real-world tasks. Despite the tremendous success of these systems, current approaches rely on narrow, single-focus objectives for optimization and evaluation, often overlooking potential constraints in real-world scenarios, including restricted budgets, resources and time. Furthermore, interpreting, analyzing, and debugging these systems requires different components to be evaluated in relation to one another. This demand is currently not feasible with existing methodologies. In this postion paper, we introduce the concept of reasoning capacity as a unifying criterion to enable integration of constraints during optimization and establish connections among different components within the system, which also enable a more holistic and comprehensive approach to evaluation. We present a formal definition of reasoning capacity and illustrate its utility in identifying limitations within each component of the system. We then argue how these limitations can be addressed with a self-reflective process wherein human-feedback is used to alleviate shortcomings in reasoning and enhance overall consistency of the system.

Computation and Language,Machine Learning
Practical Reasoning with Norms for Autonomous Software Agents (Full Edition)

Zohreh Shams,Marina De Vos,Julian Padget,Wamberto W. Vasconcelos

DOI: https://doi.org/10.48550/arXiv.1701.08306

2017-01-29

Abstract:Autonomous software agents operating in dynamic environments need to constantly reason about actions in pursuit of their goals, while taking into consideration norms which might be imposed on those actions. Normative practical reasoning supports agents making decisions about what is best for them to (not) do in a given situation. What makes practical reasoning challenging is the interplay between goals that agents are pursuing and the norms that the agents are trying to uphold. We offer a formalisation to allow agents to plan for multiple goals and norms in the presence of durative actions that can be executed concurrently. We compare plans based on decision-theoretic notions (i.e. utility) such that the utility gain of goals and utility loss of norm violations are the basis for this comparison. The set of optimal plans consists of plans that maximise the overall utility, each of which can be chosen by the agent to execute. We provide an implementation of our proposal in Answer Set Programming, thus allowing us to state the original problem in terms of a logic program that can be queried for solutions with specific properties. The implementation is proven to be sound and complete.

Artificial Intelligence
XAgents: A Framework for Interpretable Rule-Based Multi-Agents Cooperation

Hailong Yang,Mingxian Gu,Renhuo Zhao,Fuping Hu,Zhaohong Deng,Yitang Chen

2024-11-21

Abstract:Extracting implicit knowledge and logical reasoning abilities from large language models (LLMs) has consistently been a significant challenge. The advancement of multi-agent systems has further en-hanced the capabilities of LLMs. Inspired by the structure of multi-polar neurons (MNs), we propose the XAgents framework, an in-terpretable multi-agent cooperative framework based on the IF-THEN rule-based system. The IF-Parts of the rules are responsible for logical reasoning and domain membership calculation, while the THEN-Parts are comprised of domain expert agents that generate domain-specific contents. Following the calculation of the member-ship, XAgetns transmits the task to the disparate domain rules, which subsequently generate the various responses. These re-sponses are analogous to the answers provided by different experts to the same question. The final response is reached at by eliminat-ing the hallucinations and erroneous knowledge of the LLM through membership computation and semantic adversarial genera-tion of the various domain rules. The incorporation of rule-based interpretability serves to bolster user confidence in the XAgents framework. We evaluate the efficacy of XAgents through a com-parative analysis with the latest AutoAgents, in which XAgents demonstrated superior performance across three distinct datasets. We perform post-hoc interpretable studies with SHAP algorithm and case studies, proving the interpretability of XAgent in terms of input-output feature correlation and rule-based semantics.

Artificial Intelligence
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System

Zhiwei Liu,Weiran Yao,Jianguo Zhang,Liangwei Yang,Zuxin Liu,Juntao Tan,Prafulla K. Choubey,Tian Lan,Jason Wu,Huan Wang,Shelby Heinecke,Caiming Xiong,Silvio Savarese

2024-02-23

Abstract:The booming success of LLMs initiates rapid development in LLM agents. Though the foundation of an LLM agent is the generative model, it is critical to devise the optimal reasoning strategies and agent architectures. Accordingly, LLM agent research advances from the simple chain-of-thought prompting to more complex ReAct and Reflection reasoning strategy; agent architecture also evolves from single agent generation to multi-agent conversation, as well as multi-LLM multi-agent group chat. However, with the existing intricate frameworks and libraries, creating and evaluating new reasoning strategies and agent architectures has become a complex challenge, which hinders research investigation into LLM agents. Thus, we open-source a new AI agent library, AgentLite, which simplifies this process by offering a lightweight, user-friendly platform for innovating LLM agent reasoning, architectures, and applications with ease. AgentLite is a task-oriented framework designed to enhance the ability of agents to break down tasks and facilitate the development of multi-agent systems. Furthermore, we introduce multiple practical applications developed with AgentLite to demonstrate its convenience and flexibility. Get started now at: \url{https://github.com/SalesforceAIResearch/AgentLite}.

Artificial Intelligence,Multiagent Systems
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

Pranav Putta,Edmund Mills,Naman Garg,Sumeet Motwani,Chelsea Finn,Divyansh Garg,Rafael Rafailov

2024-08-14

Abstract:Large Language Models (LLMs) have shown remarkable capabilities in natural language tasks requiring complex reasoning, yet their application in agentic, multi-step reasoning within interactive environments remains a difficult challenge. Traditional supervised pre-training on static datasets falls short in enabling autonomous agent capabilities needed to perform complex decision-making in dynamic settings like web navigation. Previous attempts to bridge this ga-through supervised fine-tuning on curated expert demonstrations-often suffer from compounding errors and limited exploration data, resulting in sub-optimal policy outcomes. To overcome these challenges, we propose a framework that combines guided Monte Carlo Tree Search (MCTS) search with a self-critique mechanism and iterative fine-tuning on agent interactions using an off-policy variant of the Direct Preference Optimization (DPO) algorithm. Our method allows LLM agents to learn effectively from both successful and unsuccessful trajectories, thereby improving their generalization in complex, multi-step reasoning tasks. We validate our approach in the WebShop environment-a simulated e-commerce platform where it consistently outperforms behavior cloning and reinforced fine-tuning baseline, and beats average human performance when equipped with the capability to do online search. In real-world booking scenarios, our methodology boosts Llama-3 70B model's zero-shot performance from 18.6% to 81.7% success rate (a 340% relative increase) after a single day of data collection and further to 95.4% with online search. We believe this represents a substantial leap forward in the capabilities of autonomous agents, paving the way for more sophisticated and reliable decision-making in real-world settings.

Artificial Intelligence,Machine Learning
Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Filippos Christianos,Georgios Papoudakis,Matthieu Zimmer,Thomas Coste,Zhihao Wu,Jingxuan Chen,Khyati Khandelwal,James Doran,Xidong Feng,Jiacheng Liu,Zheng Xiong,Yicheng Luo,Jianye Hao,Kun Shao,Haitham Bou-Ammar,Jun Wang

DOI: https://doi.org/10.48550/arXiv.2312.14878

2023-12-23

Abstract:A key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL). However, constructing a standalone RL policy that maps perception to action directly encounters severe problems, chief among them being its lack of generality across multiple tasks and the need for a large amount of training data. The leading cause is that it cannot effectively integrate prior information into the perception-action cycle when devising the policy. Large language models (LLMs) emerged as a fundamental way to incorporate cross-domain knowledge into AI agents but lack crucial learning and adaptation toward specific decision problems. This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies. Our methodology is motivated by the modularity found in the human brain. The framework utilises the construction of intrinsic and extrinsic functions to add previous understandings of reasoning structures. It also provides the adaptive ability to learn models inside every module or function, consistent with the modular structure of cognitive processes. We describe the framework in-depth and compare it with other AI pipelines and existing frameworks. The paper explores practical applications, covering experiments that show the effectiveness of our method. Our results indicate that AI agents perform and adapt far better when organised reasoning and prior knowledge are embedded. This opens the door to more resilient and general AI agent systems.

Artificial Intelligence,Machine Learning
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents

Zhuosheng Zhang,Yao Yao,Aston Zhang,Xiangru Tang,Xinbei Ma,Zhiwei He,Yiming Wang,Mark Gerstein,Rui Wang,Gongshen Liu,Hai Zhao

2023-11-20

Abstract:Large language models (LLMs) have dramatically enhanced the field of language intelligence, as demonstrably evidenced by their formidable empirical performance across a spectrum of complex reasoning tasks. Additionally, theoretical proofs have illuminated their emergent reasoning capabilities, providing a compelling showcase of their advanced cognitive abilities in linguistic contexts. Critical to their remarkable efficacy in handling complex reasoning tasks, LLMs leverage the intriguing chain-of-thought (CoT) reasoning techniques, obliging them to formulate intermediate steps en route to deriving an answer. The CoT reasoning approach has not only exhibited proficiency in amplifying reasoning performance but also in enhancing interpretability, controllability, and flexibility. In light of these merits, recent research endeavors have extended CoT reasoning methodologies to nurture the development of autonomous language agents, which adeptly adhere to language instructions and execute actions within varied environments. This survey paper orchestrates a thorough discourse, penetrating vital research dimensions, encompassing: (i) the foundational mechanics of CoT techniques, with a focus on elucidating the circumstances and justification behind its efficacy; (ii) the paradigm shift in CoT; and (iii) the burgeoning of language agents fortified by CoT approaches. Prospective research avenues envelop explorations into generalization, efficiency, customization, scaling, and safety. This paper caters to a wide audience, including beginners seeking comprehensive knowledge of CoT reasoning and language agents, as well as experienced researchers interested in foundational mechanics and engaging in cutting-edge discussions on these topics. A repository for the related papers is available at <a class="link-external link-https" href="https://github.com/Zoeyyao27/CoT-Igniting-Agent" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence,Computer Vision and Pattern Recognition,Human-Computer Interaction,Multiagent Systems
ScienceWorld: Is your Agent Smarter than a 5th Grader?

Ruoyao Wang,Peter Jansen,Marc-Alexandre Côté,Prithviraj Ammanabrolu

DOI: https://doi.org/10.48550/arXiv.2203.07540

2022-03-14

Computation and Language

Abstract:We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum. Despite the transformer-based progress seen in question-answering and scientific text processing, we find that current models cannot reason about or explain learned science concepts in novel contexts. For instance, models can easily answer what the conductivity of a known material is but struggle when asked how they would conduct an experiment in a grounded environment to find the conductivity of an unknown material. This begs the question of whether current models are simply retrieving answers by way of seeing a large number of similar examples or if they have learned to reason about concepts in a reusable manner. We hypothesize that agents need to be grounded in interactive environments to achieve such reasoning capabilities. Our experiments provide empirical evidence supporting this hypothesis -- showing that a 1.5 million parameter agent trained interactively for 100k steps outperforms a 11 billion parameter model statically trained for scientific question-answering and reasoning from millions of expert demonstrations.
Reasoning About Action and Change

Florence Dupin de Saint-Cyr,Andreas Herzig,Jérôme Lang,Pierre Marquis

DOI: https://doi.org/10.1007/978-3-030-06164-7_15

2024-06-27

Abstract:The purpose of this book is to provide an overview of AI research, ranging from basic work to interfaces and applications, with as much emphasis on results as on current issues. It is aimed at an audience of master students and Ph.D. students, and can be of interest as well for researchers and engineers who want to know more about AI. The book is split into three volumes.

Artificial Intelligence,Discrete Mathematics,Logic in Computer Science,Symbolic Computation
Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents

Zhihan Liu,Hao Hu,Shenao Zhang,Hongyi Guo,Shuqi Ke,Boyi Liu,Zhaoran Wang

2024-01-01

Abstract:Large language models (LLMs) demonstrate impressive reasoning abilities, but translating reasoning into actions in the real world remains challenging. In particular, it is unclear how to complete a given task provably within a minimum number of interactions with the external environment, e.g., through an internal mechanism of reasoning. To this end, we propose the first framework with provable regret guarantees to orchestrate reasoning and acting, which we call *reason for future, act for now* (**RAFA**). Specifically, we design a prompt template for reasoning that learns from the memory buffer and plans a future trajectory over a long horizon (*reason for future*). At each step, the LLM agent takes the initial action of the planned trajectory (*act for now*), stores the collected feedback in the memory buffer, and reinvokes the reasoning routine to replan the future trajectory from the new state. The key idea is to cast reasoning in LLMs as learning and planning in Bayesian adaptive Markov decision processes (MDPs). Correspondingly, we prompt LLMs with the memory buffer to estimate the unknown environment (learning) and generate an optimal trajectory for multiple future steps that maximize a value function (planning). The learning and planning subroutines are performed in an in-context manner to emulate the actor-critic update for MDPs. Our theoretical analysis establishes a $\sqrt{T}$ regret, while our experimental validation demonstrates superior empirical performance.
CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents

Siyuan Qi,Shuo Chen,Yexin Li,Xiangyu Kong,Junqi Wang,Bangcheng Yang,Pring Wong,Yifan Zhong,Xiaoyuan Zhang,Zhaowei Zhang,Nian Liu,Wei Wang,Yaodong Yang,Song-Chun Zhu

2024-03-12

Abstract:The generalization of decision-making agents encompasses two fundamental elements: learning from past experiences and reasoning in novel contexts. However, the predominant emphasis in most interactive environments is on learning, often at the expense of complexity in reasoning. In this paper, we introduce CivRealm, an environment inspired by the Civilization game. Civilization's profound alignment with human history and society necessitates sophisticated learning, while its ever-changing situations demand strong reasoning to generalize. Particularly, CivRealm sets up an imperfect-information general-sum game with a changing number of players; it presents a plethora of complex features, challenging the agent to deal with open-ended stochastic environments that require diplomacy and negotiation skills. Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning. To catalyze further research, we present initial results for both paradigms. The canonical RL-based agents exhibit reasonable performance in mini-games, whereas both RL- and LLM-based agents struggle to make substantial progress in the full game. Overall, CivRealm stands as a unique learning and reasoning challenge for decision-making agents. The code is available at

Artificial Intelligence
Multi-Agent Cooperation Via Reasoning About The Behavior Of Others

Wenpin Jiao

DOI: https://doi.org/10.1111/j.1467-8640.2009.00352.x

2010-01-01

Computational Intelligence

Abstract:In multi-agent cooperation, the agents will cooperate efficiently if they can accurately anticipate the behavior of their partners. In this work, we put forward a framework (called as transpositional thinking principle) for reasoning about and predicting the behavior of others and propose an approach to planning the cooperation among agents based on the principle. By using the principle, agents can richen their understanding about the behavior patterns of their partners and then infer their partners' actions more and more accurately. The experiments show that the cooperation among agents will be performed more efficiently and at a low cost when agents can anticipate the behavior of others with a high enough accuracy.
Diversity of Logical Agents in Games

Johan van Benthem,Fenrong Liu

DOI: https://doi.org/10.4000/philosophiascientiae.571

2004-01-01

Philosophia Scientae

Abstract:Epistemic agents may have different powers of observation and reasoning, and we show how this diversity fits into dynamic update logics.
SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning

Alireza Ghafarollahi,Markus J. Buehler

2024-09-09

Abstract:A key challenge in artificial intelligence is the creation of systems capable of autonomously advancing scientific understanding by exploring novel domains, identifying complex patterns, and uncovering previously unseen connections in vast scientific data. In this work, we present SciAgents, an approach that leverages three core concepts: (1) the use of large-scale ontological knowledge graphs to organize and interconnect diverse scientific concepts, (2) a suite of large language models (LLMs) and data retrieval tools, and (3) multi-agent systems with in-situ learning capabilities. Applied to biologically inspired materials, SciAgents reveals hidden interdisciplinary relationships that were previously considered unrelated, achieving a scale, precision, and exploratory power that surpasses traditional human-driven research methods. The framework autonomously generates and refines research hypotheses, elucidating underlying mechanisms, design principles, and unexpected material properties. By integrating these capabilities in a modular fashion, the intelligent system yields material discoveries, critique and improve existing hypotheses, retrieve up-to-date data about existing research, and highlights their strengths and limitations. Our case studies demonstrate scalable capabilities to combine generative AI, ontological representations, and multi-agent modeling, harnessing a `swarm of intelligence' similar to biological systems. This provides new avenues for materials discovery and accelerates the development of advanced materials by unlocking Nature's design principles.

Computer Science,Physics
Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency

Zhihan Liu,Hao Hu,Shenao Zhang,Hongyi Guo,Shuqi Ke,Boyi Liu,Zhaoran Wang

2024-06-24

Abstract:Large language models (LLMs) demonstrate impressive reasoning abilities, but translating reasoning into actions in the real world remains challenging. In particular, it remains unclear how to complete a given task provably within a minimum number of interactions with the external environment, e.g., through an internal mechanism of reasoning. To this end, we propose a principled framework with provable regret guarantees to orchestrate reasoning and acting, which we call "reason for future, act for now" (\texttt{RAFA}). Specifically, we design a prompt template for reasoning that learns from the memory buffer and plans a future trajectory over a long horizon ("reason for future"). At each step, the LLM agent takes the initial action of the planned trajectory ("act for now"), stores the collected feedback in the memory buffer, and reinvokes the reasoning routine to replan the future trajectory from the new state. The key idea is to cast reasoning in LLMs as learning and planning in Bayesian adaptive Markov decision processes (MDPs). Correspondingly, we prompt LLMs to form an updated posterior of the unknown environment from the memory buffer (learning) and generate an optimal trajectory for multiple future steps that maximizes a value function (planning). The learning and planning subroutines are performed in an "in-context" manner to emulate the actor-critic update for MDPs. Our theoretical analysis proves that the novel combination of long-term reasoning and short-term acting achieves a $\sqrt{T}$ regret. Here, $T$ denotes the number of online interactions. In particular, the regret bound highlights an intriguing interplay between the prior knowledge obtained through pretraining and the uncertainty reduction achieved by reasoning and acting. Our empirical validation shows that it outperforms various existing frameworks and achieves nearly perfect scores on a few benchmarks.

Artificial Intelligence,Machine Learning

Agents that Reason and Learn

AgentKit: Structured LLM Reasoning with Dynamic Graphs

Agents Thinking Fast and Slow: A Talker-Reasoner Architecture

A Model for Learning and Acting Rationally in Multiagent System.

Agents: An Open-source Framework for Autonomous Language Agents

AGENT: A Benchmark for Core Psychological Reasoning

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

Practical Reasoning with Norms for Autonomous Software Agents (Full Edition)

XAgents: A Framework for Interpretable Rule-Based Multi-Agents Cooperation

AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents

ScienceWorld: Is your Agent Smarter than a 5th Grader?

Reasoning About Action and Change

Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents

CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents

Multi-Agent Cooperation Via Reasoning About The Behavior Of Others

Diversity of Logical Agents in Games

SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning

Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency