Abstract:Along with the proliferation of digital data collected using sensor technologies and a boost of computing power, Deep Learning (DL) based approaches have drawn enormous attention in the past decade due to their impressive performance in extracting complex relations from raw data and representing valuable information. Meanwhile, though, rooted in its notorious black-box nature, the appreciation of DL has been highly debated due to the lack of interpretability. On the one hand, DL only utilizes statistical features contained in raw data while ignoring human knowledge of the underlying system, which results in both data inefficiency and trust issues; on the other hand, a trained DL model does not provide to researchers any extra insight about the underlying system beyond its output, which, however, is the essence of most fields of science, e.g. physics and economics. This thesis addresses the issue of interpretability in general information modeling and endeavors to ease the problem from two scopes. Firstly, a problem-oriented perspective is applied to incorporate knowledge into modeling practice, where interesting mathematical properties emerge naturally which cast constraints on modeling. Secondly, given a trained model, various methods could be applied to extract further insights about the underlying system. These two pathways are termed as guided model design and secondary measurements. Remarkably, a novel scheme emerges for the modeling practice in statistical learning: Algebraic Learning (AgLr). Instead of being restricted to the discussion of any specific model, AgLr starts from idiosyncrasies of a learning task itself and studies the structure of a legitimate model class. This novel scheme demonstrates the noteworthy value of abstract algebra for general AI, which has been overlooked in recent progress, and could shed further light on interpretable information modeling.

Model Information As an Analysis Tool in Deep Learning

Visualizing and Understanding Neural Models in NLP

Measuring Information Transfer in Neural Networks

Measuring Information Transfer in Neural Networks.

Algebraic Learning: Towards Interpretable Information Modeling

Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective

Topological Interpretability for Deep-Learning

Entropy and mutual information in models of deep neural networks

Dissecting Deep Learning Networks—Visualizing Mutual Information

How Does Information Bottleneck Help Deep Learning?

Information-Theoretic Foundations for Machine Learning

Going Deeper, Generalizing Better: an Information-Theoretic View for Deep Learning.

Worth of knowledge in deep learning

Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training

A Survey of the Interpretability Aspect of Deep Learning Models

An Information Theoretic Interpretation to Deep Neural Networks.

How far from automatically interpreting deep learning

Information theoretic analysis of computational models as a tool to understand the neural basis of behaviors

An Information-Theoretic Analysis of In-Context Learning

On Learnable Parameters of Optimal and Suboptimal Deep Learning Models

Understanding Encoder-Decoder Structures in Machine Learning Using Information Measures