Abstract:A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of Large Language Models (LLMs) as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to use large - language models (LLMs) as symbolic reasoners, especially in the application in text games. Specifically, the authors explore, without the need for additional training, making LLMs able to perform complex symbolic tasks, such as mathematical operations, map reading, sorting, and common - sense application, etc., by designing appropriate prompting strategies. These tasks usually require strong symbolic reasoning abilities, while traditional reinforcement - learning methods or behavior - cloning methods have certain limitations when dealing with such tasks, such as reward delay and the difficulty in exploring large - scale action spaces. ### Main contributions of the paper 1. **Introducing LLMs for symbolic reasoning**: The paper proposes a method of using LLMs as neural - symbolic reasoners and provides a framework to achieve this goal. This method shows the potential of LLMs to complete complex tasks with the support of external modules such as calculators or navigators. 2. **Developing customized prompting mechanisms**: The researchers develop a customized prompting mechanism, enabling LLM agents to effectively utilize symbolic modules, thereby improving their performance in text games involving symbolic tasks. 3. **Experimental verification**: The experimental results show that the proposed LLM agent significantly outperforms strong baseline models in multiple benchmark tests, including the Deep Reinforcement - related Network (DRRN) with symbolic modules and the Behavior Cloned Transformer trained with a large amount of expert data, with an average performance reaching 88%. ### Method overview - **Role initialization**: First, the LLM agent is given a role and task description, as well as the actions that can be taken and their constraints. - **Action query**: At each time step, the LLM agent will receive the current game - state information, including the observation results, inventory state, score, and the set of valid actions. The agent needs to select an action from the set of valid actions to continue the task. - **Selection of the LLM agent**: The LLM agent selects an action according to the provided information to complete the task. ### Experimental setup - **Environment**: The research uses four text - game benchmark environments, respectively involving arithmetic, map - reading, sorting, and Text World Commonsense (TWC) tasks. - **Evaluation metrics**: The evaluation metrics are based on two factors: the average score at the end of each game and the average number of steps in a single episode. ### Conclusion The paper experimentally verifies the potential of LLMs in symbolic - reasoning tasks, especially in the application in text games. By designing appropriate prompting mechanisms, LLMs can effectively complete complex symbolic tasks without relying on labeled data. This provides a new direction for future research, especially in the hybrid methods combining deep learning and symbolic reasoning.

Large Language Models Are Neurosymbolic Reasoners

Can Large Language Models Act as Symbolic Reasoners?

Can Large Language Models Understand Symbolic Graphics Programs?

Investigating Symbolic Capabilities of Large Language Models

Large Language Models are Interpretable Learners

Large Language Models Are In-Context Semantic Reasoners Rather Than Symbolic Reasoners

Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning

Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models

Codenames as a Benchmark for Large Language Models

Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models

Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models

Symbolic Working Memory Enhances Language Models for Complex Rule Application

Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning

Reliable Reasoning Beyond Natural Language

Enhance Reasoning for Large Language Models in the Game Werewolf

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration

The Emergence of Strategic Reasoning of Large Language Models

Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming

LLM2: Let Large Language Models Harness System 2 Reasoning

Strategic Reasoning with Language Models