Abstract:Large Language Models (LLMs) have revolutionized natural language processing, yet they struggle with inconsistent reasoning, particularly in novel domains and complex logical sequences. This research introduces Proof of Thought, a framework that enhances the reliability and transparency of LLM outputs. Our approach bridges LLM-generated ideas with formal logic verification, employing a custom interpreter to convert LLM outputs into First Order Logic constructs for theorem prover scrutiny. Central to our method is an intermediary JSON-based Domain-Specific Language, which by design balances precise logical structures with intuitive human concepts. This hybrid representation enables both rigorous validation and accessible human comprehension of LLM reasoning processes. Key contributions include a robust type system with sort management for enhanced logical integrity, explicit representation of rules for clear distinction between factual and inferential knowledge, and a flexible architecture that allows for easy extension to various domain-specific applications. We demonstrate Proof of Thought's effectiveness through benchmarking on StrategyQA and a novel multimodal reasoning task, showing improved performance in open-ended scenarios. By providing verifiable and interpretable results, our technique addresses critical needs for AI system accountability and sets a foundation for human-in-the-loop oversight in high-stakes domains.

Mechanized Reasoning Systems : a Preliminary Report ?

The OMRS Project: State of the Art

Reasoning Theories: Towards an Architecture for Open Mechanized Reasoning Systems

The OMRS Project: State of the Art Abstract of Invited Talk

The Control Component of Open Mechanized Reasoning Systems11We especially thank Paolo Pecchiari for his initial contributions to this work, and Carolyn Talcott for contributing to the technical development of the theory.

Toward an Architecture for Open Mechanized Reasoning Systems

The Control Layer in Open Mechanized Reasoning Systems: Annotations and Tactics

Specification and Integration of Theorem Provers and Computer Algebra Systems

Reasoning Theories

A Survey on Theorem Provers in Formal Methods

A Prolog-based Environment for Reasoning about Programming Languages (Extended abstract)

When Do Program-of-Thought Works for Reasoning?

A Logic Level Specification of the NQTHM Simplification Process

A Foundation for Metareasoning Part I: the Proof Theory

Dynamic Reasoning Systems

Hybrid Automated Reasoning Tools: from Black-box to Clear-box Integration

Closed Loop Interactive Embodied Reasoning for Robot Manipulation

Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning

Combining Proofs for Description Logic and Concrete Domain Reasoning (Technical Report)

The Imandra Automated Reasoning System (system description)