Indeterminate Probability Neural Network

Tao Yang,Chuang Liu,Xiaofeng Ma,Weijia Lu,Ning Wu,Bingyang Li,Zhifei Yang,Peng Liu,Lin Sun,Xiaodong Zhang,Can Zhang
2023-03-21
Abstract:We propose a new general model called IPNN - Indeterminate Probability Neural Network, which combines neural network and probability theory together. In the classical probability theory, the calculation of probability is based on the occurrence of events, which is hardly used in current neural networks. In this paper, we propose a new general probability theory, which is an extension of classical probability theory, and makes classical probability theory a special case to our theory. Besides, for our proposed neural network framework, the output of neural network is defined as probability events, and based on the statistical analysis of these events, the inference model for classification task is deduced. IPNN shows new property: It can perform unsupervised clustering while doing classification. Besides, IPNN is capable of making very large classification with very small neural network, e.g. model with 100 output nodes can classify 10 billion categories. Theoretical advantages are reflected in experimental results.
Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition,Statistics Theory
What problem does this paper attempt to address?
The paper attempts to address the following issues: 1. **Large-scale classification problem**: How to efficiently classify datasets with a large number of categories. Traditional methods like softmax have high computational costs when dealing with large-scale datasets. 2. **Label requirement problem**: How to reduce the need for a large amount of labeled data. Traditional classification methods require a significant amount of labeled data to train the model, and the labeling process is time-consuming and labor-intensive. 3. **Combining the advantages of probability theory and neural networks**: Integrating probability theory (especially Bayesian methods) with neural networks to leverage the strengths of both. Traditional probability theory struggles to extract features from samples, while the features extracted by neural networks are often abstract and cannot be directly used for numerical probability calculations. To address these issues, the paper proposes a new framework—**Uncertain Probability Neural Network (IPNN)**. IPNN combines extended probability theory and deep neural networks, enabling the model to not only perform classification tasks but also conduct unsupervised clustering while performing classification. Additionally, it can accomplish large-scale classification tasks using a smaller network structure.