Abstract:Sparse regression and feature extraction are the cornerstones of knowledge discovery from massive data. Their goal is to discover interpretable and predictive models that provide simple relationships among scientific variables. While the statistical tools for model discovery are well established in the context of linear regression, their generalization to nonlinear regression in material modeling is highly problem-specific and insufficiently understood. Here we explore the potential of neural networks for automatic model discovery and induce sparsity by a hybrid approach that combines two strategies: regularization and physical constraints. We integrate the concept of Lp regularization for subset selection with constitutive neural networks that leverage our domain knowledge in kinematics and thermodynamics. We train our networks with both, synthetic and real data, and perform several thousand discovery runs to infer common guidelines and trends: L2 regularization or ridge regression is unsuitable for model discovery; L1 regularization or lasso promotes sparsity, but induces strong bias; only L0 regularization allows us to transparently fine-tune the trade-off between interpretability and predictability, simplicity and accuracy, and bias and variance. With these insights, we demonstrate that Lp regularized constitutive neural networks can simultaneously discover both, interpretable models and physically meaningful parameters. We anticipate that our findings will generalize to alternative discovery techniques such as sparse and symbolic regression, and to other domains such as biology, chemistry, or medicine. Our ability to automatically discover material models from data could have tremendous applications in generative material design and open new opportunities to manipulate matter, alter properties of existing materials, and discover new materials with user-defined properties.

Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

Optimizing for Interpretability in Deep Neural Networks with Tree Regularization

Regional Tree Regularization for Interpretability in Black Box Models

Improving Decision Sparsity

Sparsity regularization via tree-structured environments for disentangled representations

Inherently Interpretable Tree Ensemble Learning

From local explanations to global understanding with explainable AI for trees

Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods

Explaining the Predictions of Any Image Classifier via Decision Trees

Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree

Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention

A Survey of the Interpretability Aspect of Deep Learning Models

Sparsity in Optimal Randomized Classification Trees

A Double Penalty Model for Interpretability

Sparse oblique decision trees: a tool to understand and manipulate neural net features

Algorithms for interpretable machine learning

IGANN Sparse: Bridging Sparsity and Interpretability with Non-linear Insight

Interpretability Illusions in the Generalization of Simplified Models

On sparse regression, Lp-regularization, and automated model discovery

Cracking the Black Box: Distilling Deep Sports Analytics

Improving Interpretability via Regularization of Neural Activation Sensitivity