Abstract:Integrating symbolic techniques with statistical ones is a long-standing problem in artificial intelligence. The motivation is that the strengths of either area match the weaknesses of the other, and $\unicode{x2013}$ by combining the two $\unicode{x2013}$ the weaknesses of either method can be limited. Neuro-symbolic AI focuses on this integration where the statistical methods are in particular neural networks. In recent years, there has been significant progress in this research field, where neuro-symbolic systems outperformed logical or neural models alone. Yet, neuro-symbolic AI is, comparatively speaking, still in its infancy and has not been widely adopted by machine learning practitioners. In this survey, we present the first mapping of neuro-symbolic techniques into families of frameworks based on their architectures, with several benefits: Firstly, it allows us to link different strengths of frameworks to their respective architectures. Secondly, it allows us to illustrate how engineers can augment their neural networks while treating the symbolic methods as black-boxes. Thirdly, it allows us to map most of the field so that future researchers can identify closely related frameworks.
What problem does this paper attempt to address?
This paper attempts to address the problem of combining neural networks with symbolic reasoning, that is, how to enhance deep learning through symbolic reasoning. Specifically, it aims to solve the following problems:
1. **Structured Reasoning**: Neural networks are good at pattern recognition, but perform poorly in hierarchical or compound reasoning, and have difficulty distinguishing between causality and correlation (Lake & Baroni, 2017).
\[
\text{Neural Network} \to \text{Pattern Recognition}
\]
\[
\text{Symbolic Reasoning} \to \text{Hierarchical Reasoning and Causal Reasoning}
\]
2. **Data Requirements**: In order to make the predictions of neural models robust, a large amount of data is required (Halevy et al., 2009; Ba & Caruana, 2014). However, in many application scenarios, a large amount of data is not available, which makes neural networks an infeasible choice.
3. **Knowledge Integration**: Humans have rich knowledge in many fields, and this knowledge can be used for machine - learning tasks. However, neural networks are not easy to integrate expert knowledge or common sense (Davis & Marcus, 2015). If this knowledge can be integrated, the amount of training data and training cost required will be reduced.
4. **Interpretability**: Neural networks are black - box systems, and it is difficult to understand the process by which they make predictions for specific inputs. This not only affects knowledge extraction, but also has serious consequences for ethics, safety, and the expansion of human knowledge (Ribeiro et al., 2016; Samek et al., 2017).
5. **Guarantee**: Neural networks calculate the probability distribution of possible results and may predict results that violate constraints, which can be fatal in critical safety applications (Gopinath et al., 2018; Cardelli et al., 2019; Ruan et al., 2019).
Although logical models perform well in symbolic reasoning tasks, they also have limitations:
- **Poor Uncertainty Handling Ability**: Traditionally, each proposition must be either true or false (Pearl, 1988).
- **Poor Scalability**: As the size of the alphabet and the length of the logical theory formula increase, the computational complexity usually increases exponentially (Bradley & Manna, 2007).
To overcome these limitations, the paper proposes Neuro - Symbolic AI, a method that combines logical models with neural networks. This method takes advantage of the benefits of Statistical Relational Learning (SRL) to achieve more powerful reasoning ability and higher interpretability by combining logical and probabilistic frameworks. The goal of Neuro - Symbolic AI is to improve overall performance by combining the advantages of the two methods and compensating for their respective shortcomings.
In summary, this paper attempts to address the limitations of neural networks in structured reasoning, data requirements, knowledge integration, interpretability, and guarantee through the method of Neuro - Symbolic AI, while overcoming the problems of logical models in uncertainty and scalability.