Abstract:Deep neural networks (DNN) try to analyze given data, to come up with decisions regarding the inputs. The decision-making process of the DNN model is not entirely transparent. The confidence of the model predictions on new data fed into the network can vary. We address the question of certainty of decision making and adequacy of information capturing by DNN models during this process of decision-making. We introduce a measure called certainty index, which is based on the outputs in the most penultimate layer of DNN. In this approach, we employed iEEG (intracranial electroencephalogram) data to train and test DNN. When arriving at model predictions, the contribution of the entire information content of the input may be important. We explored the relationship between the certainty of DNN predictions and information content of the signal by estimating the sample entropy and using a heatmap of the signal. While it can be assumed that the entire sample must be utilized for arriving at the most appropriate decisions, an evaluation of DNNs from this standpoint has not been reported. We demonstrate that the robustness of the relationship between certainty index with the sample entropy, demonstrated through sample entropy-heatmap correlation, is higher than that with the original signal, indicating that the DNN focuses on information rich regions of the signal to arrive at decisions. Therefore, it can be concluded that the certainty of a decision is related to the DNN's ability to capture the information in the original signal. Our results indicate that, within its limitations, the certainty index can be used as useful tool in estimating the confidence of predictions. The certainty index appears to be related to how effectively DNN heatmaps captured the information content in the signal.

A Method for Estimating the Number of Hidden Neurons in Feed-Forward Neural Networks Based on Information Entropy

Improved Model Compression Method Based on Information Entropy

An Optimization Algorithm on the Number of Hidden Layer Nodes in Feed-forward Neural Network

An Improved Algorithm for Building Self-Organizing Feedforward Neural Networks

A New Approach to Improving Generalization Ability of Feed-Foward Neural Networks

Neural Joint Entropy Estimation

Determination of minimal artificial neural network based on decision tree algorithm

An Optimization Method for the Topological Structures of Feed-Forward Multi-Layer Neural Networks

Understanding Neural Networks with Logarithm Determinant Entropy Estimator

Adaptive Estimators Show Information Compression in Deep Neural Networks

Dense Optimizer : An Information Entropy-Guided Structural Search Method for Dense-like Neural Network Design

TWD-SFNN: Three-way decisions with a single hidden layer feedforward neural network

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

Anefficient Learning Algorithm For Binary Feedforward Neural Networks

A Hidden Layer Evaluation Method for Feedforward Neural Networks Based on Subspace Analysis

Efficient Design of Neural Networks with Random Weights

Information Entropy Measures for Evaluation of Reliability of Deep Neural Network Results

Identifying Effective Connectivity between Stochastic Neurons with Variable-Length Memory Using a Transfer Entropy Rate Estimator

Minimum number of neurons in fully connected layers of a given neural network (the first approximation)

A New Method for Decision on the Structure of RBF Neural Network

Measuring Information Transfer in Neural Networks