Neuro-symbolic Learning Yielding Logical Constraints

Zenan Li,Yunpeng Huang,Zhaoyu Li,Yuan Yao,Jingwei Xu,Taolue Chen,Xiaoxing Ma,Jian Lu

2024-10-28

Abstract:Neuro-symbolic systems combine the abilities of neural perception and logical reasoning. However, end-to-end learning of neuro-symbolic systems is still an unsolved challenge. This paper proposes a natural framework that fuses neural network training, symbol grounding, and logical constraint synthesis into a coherent and efficient end-to-end learning process. The capability of this framework comes from the improved interactions between the neural and the symbolic parts of the system in both the training and inference stages. Technically, to bridge the gap between the continuous neural network and the discrete logical constraint, we introduce a difference-of-convex programming technique to relax the logical constraints while maintaining their precision. We also employ cardinality constraints as the language for logical constraint learning and incorporate a trust region method to avoid the degeneracy of logical constraint in learning. Both theoretical analyses and empirical evaluations substantiate the effectiveness of the proposed framework.

Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve end - to - end and interpretable learning in the neural - symbolic system. Specifically, the author focuses on the following points:

1. **End - to - end learning**: Existing methods usually require human intervention to label latent variables, thereby decomposing the task into pure neural network subtasks and pure symbolic subtasks. This not only increases the cost but also limits the overall performance of the system. Therefore, the paper proposes a framework that can simultaneously learn neural perception and symbolic reasoning in a weakly - supervised setting without manual annotation.

2. **Interpretability**: Although existing methods can approximately perform symbolic reasoning, they lack explicit logical constraints, resulting in poor accuracy and interpretability of the reasoning results. By introducing explicit logical constraints, the paper enables the system to perform precise and interpretable logical reasoning.

3. **Interaction and cooperation**: During the training and reasoning processes, the interaction between the neural part and the symbolic part is crucial for improving system performance. The method proposed in the paper realizes effective interaction between neural networks and symbolic reasoning through symbol grounding and logical constraint synthesis.

To achieve these goals, the paper proposes a new neural - symbolic learning framework, which solves the above problems through the following technical means:

- **Bi - level optimization problem**: By modeling neural network training and logical constraint learning as a bi - level optimization problem, the semantic gap is bridged.
- **Difference - of - convex programming**: Using difference - of - convex programming as a relaxation technique for Boolean constraints, the representation gap is solved and the convergence of logical constraints is ensured.
- **Trust - region method**: The trust - region term is introduced to prevent the degeneracy problem in logical constraint learning, that is, to avoid learning simple or incomplete logical rules.

Overall, this paper aims to develop a new neural - symbolic learning framework to achieve end - to - end and interpretable learning while improving the performance and robustness of the system.