Abstract:Exploiting large language models (LLMs) to tackle reasoning has garnered growing attention. It still remains highly challenging to achieve satisfactory results in complex logical problems, characterized by plenty of premises within the prompt and requiring multi-hop reasoning. In particular, the reasoning capabilities of LLMs are brittle to disorder and distractibility. In this work, we first examine the mechanism from the perspective of information flow and reveal that LLMs exhibit failure patterns akin to human-like cognitive biases when dealing with disordered and irrelevant content in reasoning tasks. However, in contrast to LLMs, disordered and irrelevant content does not significantly decrease human performance, as humans have a propensity to distill the most relevant information and systematically organize their thoughts, aiding them in responding to questions. Stem from that, we further propose a novel reasoning approach named Concise and Organized Perception (COP). COP carefully analyzes the given statements to identify the most pertinent information while eliminating redundancy efficiently. It then prompts the LLMs in a more organized form that adapts to the model's inference process. By perceiving concise and organized context, the reasoning abilities of LLMs can be better elicited. Extensive experimental results on several popular logical benchmarks (ProofWriter, PrOntoQA, PrOntoQA-OOD, and FOLIO) and math benchmark (DI-GSM) show that COP significantly outperforms previous state-of-the-art methods.

Implementing Large-Scale ABox Materialization Using Subgraph Reasoning

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

TGR: Neural-symbolic ontological reasoner for domain-specific knowledge graphs

Fast ABox Consistency Checking Using Incomplete Reasoning and Caching

Approximate Ontology Reasoning for Domain-Specific Knowledge Graph based on Deep Learning

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models

AtomR: Atomic Operator-Empowered Large Language Models for Heterogeneous Knowledge Reasoning

Efficient Tool Use with Chain-of-Abstraction Reasoning

CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge

A set-theoretical approach for ABox reasoning services (Extended Version)

Large-Scale Reasoning with OWL

Reasoning with Fuzzy Ontologies

Scalable Distributed Ontology Reasoning Using DHT-Based Partitioning

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Reasoning over Multi-view Knowledge Graphs

Optimised Storage for Datalog Reasoning

More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings

Concise and Organized Perception Facilitates Reasoning in Large Language Models