Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach

Qingchuan Li,Jiatong Li,Tongxuan Liu,Yuting Zeng,Mingyue Cheng,Weizhe Huang,Qi Liu

2024-10-29

Abstract:Large Language Models (LLMs) have exhibited remarkable potential across a wide array of reasoning tasks, including logical reasoning. Although massive efforts have been made to empower the logical reasoning ability of LLMs via external logical symbolic solvers, crucial challenges of the poor generalization ability to questions with different features and inevitable question information loss of symbolic solver-driven approaches remain unresolved. To mitigate these issues, we introduce LINA, a LLM-driven neuro-symbolic approach for faithful logical reasoning. By enabling an LLM to autonomously perform the transition from propositional logic extraction to sophisticated logical reasoning, LINA not only bolsters the resilience of the reasoning process but also eliminates the dependency on external solvers. Additionally, through its adoption of a hypothetical-deductive reasoning paradigm, LINA effectively circumvents the expansive search space challenge that plagues traditional forward reasoning methods. Empirical evaluations demonstrate that LINA substantially outperforms both established propositional logic frameworks and conventional prompting techniques across a spectrum of five logical reasoning tasks. Specifically, LINA achieves an improvement of 24.34% over LINC on the FOLIO dataset, while also surpassing prompting strategies like CoT and CoT-SC by up to 24.02%. Our code is available at <a class="link-external link-https" href="https://github.com/wufeiwuwoshihua/nshy" rel="external noopener nofollow">this https URL</a>.

Computation and Language

What problem does this paper attempt to address?

The main problem this paper attempts to address is the inadequacy of existing large language models (LLMs) in logical reasoning tasks, particularly the issues of information loss and poor generalization ability in methods that rely on external symbolic solvers. To overcome these problems, the authors propose a new neuro-symbolic approach called LINA (LLM-driven Neuro-Symbolic Approach), which enhances the reliability and generalization ability of the reasoning process by autonomously extracting propositional logic and performing complex logical reasoning.

Specifically, the main contributions of LINA include:

1. **Information Extraction Module**: By combining first-order logic (FOL) and natural language information, it reduces information loss and ensures that all valid information is fully utilized during the reasoning process.
2. **LLM-driven Symbolic Reasoning Module**: It employs hypothetical-deductive reasoning, using LLM for step-by-step reasoning, avoiding reliance on external tools and improving reasoning accuracy.
3. **Graphical Explanation and Complexity Analysis**: It provides a graphical explanation of the LINA algorithm and conducts complexity analysis, proving its theoretical correctness and effectiveness.

Experimental results show that LINA significantly outperforms existing neuro-symbolic methods and prompt-based techniques on multiple logical reasoning datasets, particularly excelling on challenging datasets such as ReClor and LogiQA.