Grey Neural Network And Its Application In MCMQC

Shuhai Fan,Tianyuan Xiao,Yexiang Fang,Wenping Li,Linxuan Zhang

DOI: https://doi.org/10.1109/ICNC.2007.410

2007-01-01

Abstract:Knowledge about facts from the real world is very often uncertain, ambiguous and vague. As a result, the inference mechanism based on such facts generates uncertain and vague conclusions. In this paper we describe a formal model of knowledge representation ...

What problem does this paper attempt to address?

Simulation of Grey Neural Network in MCMQC

FAN Shu-hai,XIAO Tian-yuan,CAO xia,NI Wei-hong

2008-01-01

Abstract:To solve the problem of grey uncertainty of quality data in mass customization manufacturing quality control, the grey theory was integrated into the model building of neural network. Based on the kernel of grey theory (hierarchical, poly-dimensional and time-variant), the model building methods of grey system were built. To simplify the computation, the paper also gave out three simplified model building methods (crosswise, lengthwise and hybrid modeling method), which can be integrated into neural network modeling. Finally, the paper gave out an example for a typical grey BP neural network in which the crosswise modeling method was used and applied to mass customization manufacturing quality control. The final simulation result showed the feasibility and effectiveness of the grey neural network.
Quantitative Analysis of Generalization Ability of Neural Network with Prior-Knowledge

Jie Sun,Tie-Jun Wu

DOI: https://doi.org/10.1016/s1474-6670(17)56907-4

1999-01-01

IFAC Proceedings Volumes

Abstract:The theoretical analysis of neural network generalization is presented in this paper with techniques based on the PAC learning and the VC dimension. Generalization indexes are proposed to measure the approximation correctness of trained neural networks for given target functions. Neural networks with prior knowledge are discussed on the basis of the generalization measurement, and are proved to possess better generalization ability compared with ordinary networks. They usually require considerably fewer training examples while keeping better generalization performance
Neural Axiom Network for Knowledge Graph Reasoning

Juan Li,Xiangnan Chen,Hongtao Yu,Jiaoyan Chen,Wen Zhang

DOI: https://doi.org/10.3233/sw-233276

2024-01-01

Semantic Web

Abstract:Knowledge graph reasoning (KGR) aims to infer new knowledge or detect noises, which is essential for improving the quality of knowledge graphs. Recently, various KGR techniques, such as symbolic- and embedding-based methods, have been proposed and shown strong reasoning ability. Symbolic-based reasoning methods infer missing triples according to predefined rules or ontologies. Although rules and axioms have proven effective, it is difficult to obtain them. Embedding-based reasoning methods represent entities and relations as vectors, and complete KGs via vector computation. However, they mainly rely on structural information and ignore implicit axiom information not predefined in KGs but can be reflected in data. That is, each correct triple is also a logically consistent triple and satisfies all axioms. In this paper, we propose a novel NeuRal Axiom Network (NeuRAN) framework that combines explicit structural and implicit axiom information without introducing additional ontologies. Specifically, the framework consists of a KG embedding module that preserves the semantics of triples and five axiom modules that encode five kinds of implicit axioms. These axioms correspond to five typical object property expression axioms defined in OWL2, including ObjectPropertyDomain, ObjectPropertyRange, DisjointObjectProperties, IrreflexiveObjectProperty and AsymmetricObjectProperty. The KG embedding module and axiom modules compute the scores that the triple conforms to the semantics and the corresponding axioms, respectively. Compared with KG embedding models and CKRL, our method achieves comparable performance on noise detection and triple classification and achieves significant performance on link prediction. Compared with TransE and TransH, our method improves the link prediction performance on the Hits@1 metric by 22.0% and 20.8% on WN18RR-10% dataset, respectively.
New representation method for uncertain knowledge——grey probability measure set

HAN Sheng,LI Shu-jie,CHEN Zong-hai

DOI: https://doi.org/10.3969/j.issn.1001-506X.2013.05.20

2013-01-01

Abstract:A new description method for grey number by simulating man's description and fusion to uncertainty information is represented,and the definition of a grey probability measurable set and its intersecting operation is proposed.The grey probability measurable set can describe the uncertainty with the features such as "poor-information" and "clear-boundary,unclear-meaning",the intersecting operation reflects that as information increases,the "meaning" of the uncertainty that the grey probability measurable set represents becomes more certain and more proper.According to the example of topological localization of an autonomous mobile robot,it can be concluded that this structure can be well used to verify the effectiveness in the simutation of man's intelligence to uncertainty expression and reasoning.
A novel neural grey system model with Bayesian regularization and its applications

Xin Ma,Mei Xie,Johan A.K. Suykens

DOI: https://doi.org/10.1016/j.neucom.2021.05.048

IF: 6

2021-10-01

Neurocomputing

Abstract:<p>Most machine learning models are essentially "Black-box" models, of which the performance heavily relies on large scale data sets. In this work the idea of "Grey-box" modelling is adopted in order to take most advantage of known information represented by deterministic structure, and then the neural grey system model is developed. Levenberg-Marquardt algorithm is used to train the proposed model, and the Bayesian regularization is used to tune the regularized parameter automatically. Six real world case studies are presented to show the performance of the proposed model, comparing to 6 existing machine learning models and 17 grey system models. The results show that the proposed model can significantly overperform the other models and has very good generality, illustrating its high potential in a wide variety of real world applications and the efficiency of the proposed modelling method.</p>

computer science, artificial intelligence
Quantum-Cognitive Neural Networks: Assessing Confidence and Uncertainty with Human Decision-Making Simulations

Milan Maksimovic,Ivan S. Maksymov

2024-12-11

Abstract:Modern machine learning (ML) systems excel in recognising and classifying images with remarkable accuracy. However, like many computer software systems, they can fail by generating confusing or erroneous outputs or by deferring to human operators to interpret the results and make final decisions. In this paper, we employ the recently proposed quantum-tunnelling neural networks (QT-NNs), inspired by human brain processes, alongside quantum cognition theory, to classify image datasets while emulating human perception and judgment. Our findings suggest that the QT-NN model provides compelling evidence of its potential to replicate human-like decision-making and outperform traditional ML algorithms.

Machine Learning,Physics and Society,Quantum Physics
Epistemic Uncertainty Quantification For Pre-trained Neural Network

Hanjing Wang,Qiang Ji

2024-04-16

Abstract:Epistemic uncertainty quantification (UQ) identifies where models lack knowledge. Traditional UQ methods, often based on Bayesian neural networks, are not suitable for pre-trained non-Bayesian models. Our study addresses quantifying epistemic uncertainty for any pre-trained model, which does not need the original training data or model modifications and can ensure broad applicability regardless of network architectures or training techniques. Specifically, we propose a gradient-based approach to assess epistemic uncertainty, analyzing the gradients of outputs relative to model parameters, and thereby indicating necessary model adjustments to accurately represent the inputs. We first explore theoretical guarantees of gradient-based methods for epistemic UQ, questioning the view that this uncertainty is only calculable through differences between multiple models. We further improve gradient-driven UQ by using class-specific weights for integrating gradients and emphasizing distinct contributions from neural network layers. Additionally, we enhance UQ accuracy by combining gradient and perturbation methods to refine the gradients. We evaluate our approach on out-of-distribution detection, uncertainty calibration, and active learning, demonstrating its superiority over current state-of-the-art UQ methods for pre-trained models.

Machine Learning,Computer Vision and Pattern Recognition
Qualitative Cognition for Uncertainty Knowledge Using Cloud Model.

Yuchao Liu,Lin Li,Juanzi Li

DOI: https://doi.org/10.1007/978-1-4614-6880-6_35

2013-01-01

Abstract:Concepts are basic elements of natural language processing, studying on concept representation and transformation between connotation and extension become more and more important. Multi-granularity concept extraction is still a difficult problem in uncertainty knowledge representation. Cloud model is an uncertainty cognition model, which realizes the bidirectional transformation between a qualitative concept and quantitative data by Gaussian cloud algorithm. Gaussian cloud transformation provides a method to transform a group of data in problem domain to multiple concepts in different granularities in cognition domain. This paper introduces cloud model and Gaussian cloud transformation algorithm to describe the multi-granularity concepts. A case study is also given to prove the effectiveness of the proposed method.
Knowledge Graph Embedding Based on Hyperplane and Quantitative Credibility.

Shuo Chen,Lin Qiao,Biqi Liu,Jue Bo,Yuanning Cui,Jing Li

DOI: https://doi.org/10.1007/978-3-030-32388-2_50

2019-01-01

Abstract:Knowledge representation learning is one of the research hotspots in the field of knowledge graph in recent years. How to improve the training algorithm of knowledge representation model and improve the accuracy of knowledge graph knowledge completion prediction is the main research goal in this field. Applying more implicit semantic information to model training is the primary means of improving accuracy. The traditional method does not consider the change of knowledge validity with time. So for this problem, we study the distribution law of quantitative changes of knowledge, design the model to simulate the quantitative changes of knowledge, put forward the concept of quantitative credibility, and apply it to the training algorithm of the model, and put forward A new learning method of knowledge representation QCHyTE. We compare the trained model with the best-recognized algorithms, and the results show that our improved algorithm greatly improves the prediction accuracy of the model.
Uncertainty Quantification in Working Memory via Moment Neural Networks

Hengyuan Ma,Wenlian Lu,Jianfeng Feng

2024-11-21

Abstract:Humans possess a finely tuned sense of uncertainty that helps anticipate potential errors, vital for adaptive behavior and survival. However, the underlying neural mechanisms remain unclear. This study applies moment neural networks (MNNs) to explore the neural mechanism of uncertainty quantification in working memory (WM). The MNN captures nonlinear coupling of the first two moments in spiking neural networks (SNNs), identifying firing covariance as a key indicator of uncertainty in encoded information. Trained on a WM task, the model demonstrates coding precision and uncertainty quantification comparable to human performance. Analysis reveals a link between the probabilistic and sampling-based coding for uncertainty representation. Transferring the MNN's weights to an SNN replicates these results. Furthermore, the study provides testable predictions demonstrating how noise and heterogeneity enhance WM performance, highlighting their beneficial role rather than being mere biological byproducts. These findings offer insights into how the brain effectively manages uncertainty with exceptional accuracy.

Biological Physics,Neural and Evolutionary Computing,Applications
Research on Tacit Knowledge Acquisition Based on Gray Theory and RBF Neural Network

Xiaomin Guo,Zhiwei Zhou,Yamin Zhao,Mengmeng Zhang,Zhikang Lian,Lin Wei,Jianhua Zhang

DOI: https://doi.org/10.1051/matecconf/201710005016

2017-01-01

MATEC Web of Conferences

Abstract:In the era of knowledge economy, Knowledge has become the main means of production and the core source for value creation. However, especially tacit knowledge acquisition, knowledge acquisition is bottleneck in all kinds of knowledge system, cutting down the benefit of knowledge sharing, application and innovation. In view of this, in this paper, the main ideas and deficiencies of the existing methods of knowledge acquisition are generalized. And then, the basic principles, characteristics and the integrated advantage of gray theory and RBF neural networks are analyzed. On this basis, three kinds of models are designed and discussed, which is able to obtain tacit rule set by using gray theory integrated with RBF neural network. Simultaneously, an empirical analysis is carried out to analyze application results of tacit knowledge acquisition, which includes five kinds of models using independent and integrated strategies.
Knowledge-Aware Neuron Interpretation for Scene Classification

Yong Guan,Freddy Lecue,Jiaoyan Chen,Ru Li,Jeff Z. Pan

2024-01-29

Abstract:Although neural models have achieved remarkable performance, they still encounter doubts due to the intransparency. To this end, model prediction explanation is attracting more and more attentions. However, current methods rarely incorporate external knowledge and still suffer from three limitations: (1) Neglecting concept completeness. Merely selecting concepts may not sufficient for prediction. (2) Lacking concept fusion. Failure to merge semantically-equivalent concepts. (3) Difficult in manipulating model behavior. Lack of verification for explanation on original model. To address these issues, we propose a novel knowledge-aware neuron interpretation framework to explain model predictions for image scene classification. Specifically, for concept completeness, we present core concepts of a scene based on knowledge graph, ConceptNet, to gauge the completeness of concepts. Our method, incorporating complete concepts, effectively provides better prediction explanations compared to baselines. Furthermore, for concept fusion, we introduce a knowledge graph-based method known as Concept Filtering, which produces over 23% point gain on neuron behaviors for neuron interpretation. At last, we propose Model Manipulation, which aims to study whether the core concepts based on ConceptNet could be employed to manipulate model behavior. The results show that core concepts can effectively improve the performance of original model by over 26%.

Computer Vision and Pattern Recognition,Artificial Intelligence
A New Grey Neural Network and Its Application to Vibration Control of Offshore Platform

CUI Hong-yu,ZHAO De-you

2010-01-01

Abstract:To deal with the difficulties in confirming the structure of neuron model and low forecast accuracy of grey theory,the grey situation decision theory based on new effect measure is proposed for the first time,and it is adopted to construct the structure topology of neural networks and determine the number of neurons in various layers and input-output computation of neurons.Then the grey neural network is taken as adaptive predictive inverse controller,which is implemented to the active control of jacket offshore platform.The simulation results show that the obtained grey neural network has strong robustness,and thus can be used to effectively control the displacement response on the top of jacket offshore platform under the random loads.
One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

Pengfei Cao,Yuheng Chen,Zhuoran Jin,Yubo Chen,Kang Liu,Jun Zhao

2024-11-26

Abstract:Large language models (LLMs) have learned vast amounts of factual knowledge through self-supervised pre-training on large-scale corpora. Meanwhile, LLMs have also demonstrated excellent multilingual capabilities, which can express the learned knowledge in multiple languages. However, the knowledge storage mechanism in LLMs still remains mysterious. Some researchers attempt to demystify the factual knowledge in LLMs from the perspective of knowledge neurons, and subsequently discover language-agnostic knowledge neurons that store factual knowledge in a form that transcends language barriers. However, the preliminary finding suffers from two limitations: 1) High Uncertainty in Localization Results. Existing study only uses a prompt-based probe to localize knowledge neurons for each fact, while LLMs cannot provide consistent answers for semantically equivalent queries. Thus, it leads to inaccurate localization results with high uncertainty. 2) Lack of Analysis in More Languages. The study only analyzes language-agnostic knowledge neurons on English and Chinese data, without exploring more language families and languages. Naturally, it limits the generalizability of the findings. To address aforementioned problems, we first construct a new benchmark called Rephrased Multilingual LAMA (RML-LAMA), which contains high-quality cloze-style multilingual parallel queries for each fact. Then, we propose a novel method named Multilingual Integrated Gradients with Uncertainty Estimation (MATRICE), which quantifies the uncertainty across queries and languages during knowledge localization. Extensive experiments show that our method can accurately localize language-agnostic knowledge neurons. We also further investigate the role of language-agnostic knowledge neurons in cross-lingual knowledge editing, knowledge enhancement and new knowledge injection.

Computation and Language
Research on the application of Neural Network Model in Knowledge graph completion technology

Guangyue Wu,Jing Wang,Zibo Yang,Yali Qi,Zixuan Li

DOI: https://doi.org/10.1109/eki61071.2023.00023

2023-09-22

Abstract:Knowledge Graph (KG) is a technology that employs graph models to represent knowledge and model the interconnected relationships among entities. The purpose of knowledge graph completion techniques is to automatically fill in missing entities, attributes, and relationships within an existing knowledge graph through inference and learning methods. In recent years, neural network models have emerged as powerful learning frameworks and have found extensive applications in the domain of knowledge graph completion. This paper presents a comprehensive investigation and analysis of the application of neural network models in knowledge graph completion techniques. Explored the challenges faced by knowledge graph completion techniques such as sparse data. Point out the future development direction, multimodal fusion and incremental learning knowledge graph completion. The objective is to provide valuable insights and assistance for the advancement of research in knowledge graph completion techniques.

Computer Science
Approaching Neural Network Uncertainty Realism

Joachim Sicking,Alexander Kister,Matthias Fahrland,Stefan Eickeler,Fabian Hüger,Stefan Rüping,Peter Schlicht,Tim Wirtz

DOI: https://doi.org/10.48550/arXiv.2101.02974

2021-01-08

Abstract:Statistical models are inherently uncertain. Quantifying or at least upper-bounding their uncertainties is vital for safety-critical systems such as autonomous vehicles. While standard neural networks do not report this information, several approaches exist to integrate uncertainty estimates into them. Assessing the quality of these uncertainty estimates is not straightforward, as no direct ground truth labels are available. Instead, implicit statistical assessments are required. For regression, we propose to evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. An empirical evaluation reveals the need for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors. Alongside, we transfer the variational U-Net classification architecture to standard supervised image-to-image tasks. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.

Machine Learning
The general conformable fractional grey system model and its applications

Wanli Xie,Mingyong Pang,Wen-Ze Wu,Chong Liu,Caixia Liu

DOI: https://doi.org/10.48550/arXiv.2104.01114

2021-03-28

Other Statistics

Abstract:Grey system theory is an important mathematical tool for describing uncertain information in the real world. It has been used to solve the uncertainty problems specially caused by lack of information. As a novel theory, the theory can deal with various fields and plays an important role in modeling the small sample problems. But many modeling mechanisms of grey system need to be answered, such as why grey accumulation can be successfully applied to grey prediction model? What is the key role of grey accumulation? Some scholars have already given answers to a certain extent. In this paper, we explain the role from the perspective of complex networks. Further, we propose generalized conformable accumulation and difference, and clarify its physical meaning in the grey model. We use our newly proposed fractional accumulation and difference to our generalized conformable fractional grey model, or GCFGM(1,1), and employ practical cases to verify that GCFGM(1,1) has higher accuracy compared to traditional models.
An extended neural ordinary differential equation network with grey system and its applications

Fangxue Zhang,Xinping Xiao,Mingyun Gao

DOI: https://doi.org/10.1016/j.neucom.2024.127343

IF: 6

2024-04-01

Neurocomputing

Abstract:The neural ordinary differential equation (NODE) has attracted much attention for its applicability in dynamic system modeling and continuous time series analysis. When the sample size is limited, models often exhibit weak generalizability and robustness and are susceptible to overfitting. To address this, a novel multivariate grey neural differential equation model is proposed based on the grey model and NODE. The new model leverages the small-sample modeling capabilities of grey systems to enhance the overall generalizability. When the neural network structure changes, the proposed model can degenerate into other grey models, enhancing inclusiveness and adaptability. Two energy forecasting cases show that the new model achieves average MAPE values of 0.82% and 1.13% on the test sets. These values are significantly better than those of the other 10 benchmark models. Furthermore, the proposed model exhibits superior performance in terms of the MAE, RMSE, STD, and APE metrics compared to those of all contrastive models. This study demonstrates that the new model effectively enhances its predictive capabilities on limited nonlinear data, showcasing higher prediction accuracy, stronger adaptability, and better stability.

computer science, artificial intelligence
Hyper Evidential Deep Learning to Quantify Composite Classification Uncertainty

Changbin Li,Kangshuo Li,Yuzhe Ou,Lance M. Kaplan,Audun Jøsang,Jin-Hee Cho,Dong Hyun Jeong,Feng Chen

2024-04-17

Abstract:Deep neural networks (DNNs) have been shown to perform well on exclusive, multi-class classification tasks. However, when different classes have similar visual features, it becomes challenging for human annotators to differentiate them. This scenario necessitates the use of composite class labels. In this paper, we propose a novel framework called Hyper-Evidential Neural Network (HENN) that explicitly models predictive uncertainty due to composite class labels in training data in the context of the belief theory called Subjective Logic (SL). By placing a grouped Dirichlet distribution on the class probabilities, we treat predictions of a neural network as parameters of hyper-subjective opinions and learn the network that collects both single and composite evidence leading to these hyper-opinions by a deterministic DNN from data. We introduce a new uncertainty type called vagueness originally designed for hyper-opinions in SL to quantify composite classification uncertainty for DNNs. Our results demonstrate that HENN outperforms its state-of-the-art counterparts based on four image datasets. The code and datasets are available at:

Computer Vision and Pattern Recognition,Machine Learning
Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models

Yuheng Chen,Pengfei Cao,Yubo Chen,Yining Wang,Shengping Liu,Kang Liu,Jun Zhao

2024-06-17

Abstract:Large language models (LLMs) store extensive factual knowledge, but the underlying mechanisms remain unclear. Previous research suggests that factual knowledge is stored within multi-layer perceptron weights, and some storage units exhibit degeneracy, referred to as Degenerate Knowledge Neurons (DKNs). Despite the novelty and unique properties of this concept, it has not been rigorously defined or systematically studied. We first consider the connection weight patterns of MLP neurons and define DKNs from both structural and functional aspects. Based on this, we introduce the Neurological Topology Clustering method, which allows the formation of DKNs in any numbers and structures, leading to a more accurate DKN acquisition. Furthermore, inspired by cognitive science, we explore the relationship between DKNs and the robustness, evolvability, and complexity of LLMs. Our execution of 34 experiments under 6 settings demonstrates the connection between DKNs and these three properties. The code will be available soon.

Computation and Language,Artificial Intelligence

Grey Neural Network And Its Application In MCMQC

Simulation of Grey Neural Network in MCMQC

Quantitative Analysis of Generalization Ability of Neural Network with Prior-Knowledge

Neural Axiom Network for Knowledge Graph Reasoning

New representation method for uncertain knowledge——grey probability measure set

A novel neural grey system model with Bayesian regularization and its applications

Quantum-Cognitive Neural Networks: Assessing Confidence and Uncertainty with Human Decision-Making Simulations

Epistemic Uncertainty Quantification For Pre-trained Neural Network

Qualitative Cognition for Uncertainty Knowledge Using Cloud Model.

Knowledge Graph Embedding Based on Hyperplane and Quantitative Credibility.

Uncertainty Quantification in Working Memory via Moment Neural Networks

Research on Tacit Knowledge Acquisition Based on Gray Theory and RBF Neural Network

Knowledge-Aware Neuron Interpretation for Scene Classification

A New Grey Neural Network and Its Application to Vibration Control of Offshore Platform

One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

Research on the application of Neural Network Model in Knowledge graph completion technology

Approaching Neural Network Uncertainty Realism

The general conformable fractional grey system model and its applications

An extended neural ordinary differential equation network with grey system and its applications

Hyper Evidential Deep Learning to Quantify Composite Classification Uncertainty

Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models