Abstract:The AI community is increasingly putting its attention towards combining symbolic and neural approaches, as it is often argued that the strengths and weaknesses of these approaches are complementary. One recent trend in the literature are weakly supervised learning techniques that employ operators from fuzzy logics. In particular, these use prior background knowledge described in such logics to help the training of a neural network from unlabeled and noisy data. By interpreting logical symbols using neural networks, this background knowledge can be added to regular loss functions, hence making reasoning a part of learning. We study, both formally and empirically, how a large collection of logical operators from the fuzzy logic literature behave in a differentiable learning setting. We find that many of these operators, including some of the most well-known, are highly unsuitable in this setting. A further finding concerns the treatment of implication in these fuzzy logics, and shows a strong imbalance between gradients driven by the antecedent and the consequent of the implication. Furthermore, we introduce a new family of fuzzy implications (called sigmoidal implications) to tackle this phenomenon. Finally, we empirically show that it is possible to use Differentiable Fuzzy Logics for semi-supervised learning, and compare how different operators behave in practice. We find that, to achieve the largest performance improvement over a supervised baseline, we have to resort to non-standard combinations of logical operators which perform well in learning, but no longer satisfy the usual logical laws.

Fuzzy Logic Interpretation of Quadratic Networks

Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks

On the learning dynamics of two-layer quadratic neural networks for understanding deep learning

Least Squares Training of Quadratic Convolutional Neural Networks with Applications to System Theory

Efficient Vectorized Backpropagation Algorithms for Training Feedforward Networks Composed of Quadratic Neurons

Quadratic models for understanding catapult dynamics of neural networks

Quadratic neural networks for solving inverse problems

Solving for a Quadratic Programming with a Quadratic Constraint Based on a Neural Network Frame

Avoiding Spurious Local Minima in Deep Quadratic Networks

Robust Generalization of Quadratic Neural Networks via Function Identification

Addressing the interpretability problem for deep learning using many valued quantum logic

A Fuzzy Quantum Neural Network and Its Application in Pattern Recognition

Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks

The learning algorithm for a novel fuzzy neural network

QuadraNet V2: Efficient and Sustainable Training of High-Order Neural Networks with Quadratic Adaptation

A Universal Logic Operator for Interpretable Deep Convolution Networks

Variational Bayesian deep fuzzy models for interpretable classification

Analyzing Differentiable Fuzzy Logic Operators

Classification with neural networks with quadratic decision functions

Optimization of Real, Hermitian Quadratic Forms: Real, Complex Hopfield-Amari Neural Network