DANA: Domain-Aware Neurosymbolic Agents for Consistency and Accuracy

Vinh Luong,Sang Dinh,Shruti Raghavan,William Nguyen,Zooey Nguyen,Quynh Le,Hung Vo,Kentaro Maegaito,Loc Nguyen,Thao Nguyen,Anh Hai Ha,Christopher Nguyen

2024-09-28

Abstract:Large Language Models (LLMs) have shown remarkable capabilities, but their inherent probabilistic nature often leads to inconsistency and inaccuracy in complex problem-solving tasks. This paper introduces DANA (Domain-Aware Neurosymbolic Agent), an architecture that addresses these issues by integrating domain-specific knowledge with neurosymbolic approaches. We begin by analyzing current AI architectures, including AutoGPT, LangChain ReAct and OpenAI's ChatGPT, through a neurosymbolic lens, highlighting how their reliance on probabilistic inference contributes to inconsistent outputs. In response, DANA captures and applies domain expertise in both natural-language and symbolic forms, enabling more deterministic and reliable problem-solving behaviors. We implement a variant of DANA using Hierarchical Task Plans (HTPs) in the open-source OpenSSA framework. This implementation achieves over 90\% accuracy on the FinanceBench financial-analysis benchmark, significantly outperforming current LLM-based systems in both consistency and accuracy. Application of DANA in physical industries such as semiconductor shows that its flexible architecture for incorporating knowledge is effective in mitigating the probabilistic limitations of LLMs and has potential in tackling complex, real-world problems that require reliability and precision.

Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problems of **inconsistency and inaccuracy** in complex problem - solving tasks of large - language models (LLMs) due to their inherent probabilistic nature. Specifically: 1. **Problems Caused by the Probabilistic Nature of LLMs**: - Although LLMs have demonstrated excellent capabilities, their probabilistic reasoning mechanisms make the output results unstable and unreliable when dealing with complex tasks. - This uncertainty is especially evident in industrial fields that require reliability and precision, such as semiconductor manufacturing, financial analysis, etc. 2. **Limitations of Existing AI Architectures**: - Current AI architectures (such as AutoGPT, LangChain ReAct, and OpenAI's ChatGPT) mainly rely on LLMs to create programs from scratch, which leads to variability in solutions. - During the execution process, the deep involvement of LLMs also exacerbates the inconsistency at the individual task level. - The lack of effective integration of domain - specific knowledge makes these systems perform poorly in dealing with complex problems. 3. **Proposal of DANA**: - To overcome these problems, the paper proposes DANA (Domain - Aware Neurosymbolic Agent), an architecture that combines the neuro - symbolic method. - DANA improves the consistency and accuracy of problem - solving by integrating domain - specific knowledge, using knowledge representations in natural language and symbolic forms, and having an explicit knowledge - capture and - application process. ### Main Contributions of DANA: - **Enhanced Deterministic Behavior**: By introducing more symbolic structures and operations, the uncertainty of output results is reduced. - **Domain Knowledge Integration**: The knowledge of domain experts is stored in natural language and symbolic forms and applied in the problem - solving process. - **Efficient Problem - Solving**: In the FinanceBench financial analysis benchmark test, DANA achieved an accuracy rate of over 90% and significantly outperformed existing LLM - based systems. In conclusion, this paper is committed to solving the problems of inconsistency and inaccuracy in LLMs' complex problem - solving, especially in industrial application scenarios that require high reliability and precision, through the DANA architecture.

DANA: Domain-Aware Neurosymbolic Agents for Consistency and Accuracy

Neurosymbolic AI approach to Attribution in Large Language Models

The Accuracy of Domain Specific and Descriptive Analysis Generated by Large Language Models

Learning to Model the World with Language

Logically Consistent Language Models via Neuro-Symbolic Integration

DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs

Converging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Empowered Autonomous Agents

DANLI: Deliberative Agent for Following Natural Language Instructions

Exploiting Language Models as a Source of Knowledge for Cognitive Agents

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines

Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs

Building trustworthy NeuroSymbolic AI Systems: Consistency, reliability, explainability, and safety

LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination

DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

A Demonstration of Adaptive Collaboration of Large Language Models for Medical Decision-Making

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

Synergistic Integration of Large Language Models and Cognitive Architectures for Robust AI: An Exploratory Analysis

Dynamic Models of Neural Population Dynamics

Can We Trust AI Agents? An Experimental Study Towards Trustworthy LLM-Based Multi-Agent Systems for AI Ethics

From Prediction to Prescription: Large Language Model Agent for Context-Aware Maintenance Decision Support