Abstract:Chain-of-Thought(CoT) prompting and its variants explore equipping large language models (LLMs) with high-level reasoning abilities by emulating human-like linear cognition and logic. However, the human mind is complicated and mixed with both linear and nonlinear thinking. In this work, we propose \textbf{I}nferential \textbf{E}xclusion \textbf{P}rompting (IEP), a novel prompting that combines the principles of elimination and inference in order to guide LLMs to think non-linearly. IEP guides LLMs to plan and then utilize Natural Language Inference (NLI) to deduce each possible solution's entailment relation with context, commonsense, or facts, therefore yielding a broader perspective by thinking back for inferring. This forward planning and backward eliminating process allows IEP to better simulate the complex human thinking processes compared to other CoT-based methods, which only reflect linear cognitive processes. We conducted a series of empirical studies and have corroborated that IEP consistently outperforms CoT across various tasks. Additionally, we observe that integrating IEP and CoT further improves the LLMs' performance on certain tasks, highlighting the necessity of equipping LLMs with mixed logic processes. Moreover, to better evaluate comprehensive features inherent in human logic, we introduce \textbf{M}ental-\textbf{A}bility \textbf{R}easoning \textbf{B}enchmark (MARB). The benchmark comprises six novel subtasks with a total of 9,115 questions, among which 1,685 are developed with hand-crafted rationale references. We believe both \textsc{IEP} and \textsc{MARB} can serve as a promising direction for unveiling LLMs' logic and verbal reasoning abilities and drive further advancements. \textsc{MARB} will be available at ~\texttt{anonymity link} soon.

Euler/X: A Toolkit for Logic-based Taxonomy Integration

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts

dxo: A System for Relational Algebra and Differentiation

Multi-tool Integration Application for Math Reasoning Using Large Language Model

LeanReasoner: Boosting Complex Logical Reasoning with Lean

Towards Intuitive Reasoning in Axiomatic Geometry

Pushing the Boundaries of Tractable Multiperspective Reasoning: A Deduction Calculus for Standpoint EL+

Merging Frameworks for Interaction

Faithful Logical Reasoning via Symbolic Chain-of-Thought

CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication

CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models

When one logic is not enough: Integrating first-order annotations in OWL ontologies

Solving probability puzzles with logic toolkit

New reasoning algorithm based on EFALC

A Taxonomy of Hoare-Like Logics: Towards a Holistic View using Predicate Transformers and Kleene Algebras with Top and Tests

Eliminating Reasoning via Inferring with Planning: A New Framework to Guide LLMs' Non-linear Thinking

Automated reasoning support for Standpoint-OWL 2

SpEuler: Semantics-preserving Euler Diagrams