Tensor Polynomial Additive Model

Yang Chen,Ce Zhu,Jiani Liu,Yipeng Liu

2024-06-05

Abstract:Additive models can be used for interpretable machine learning for their clarity and simplicity. However, In the classical models for high-order data, the vectorization operation disrupts the data structure, which may lead to degenerated accuracy and increased computational complexity. To deal with these problems, we propose the tensor polynomial addition model (TPAM). It retains the multidimensional structure information of high-order inputs with tensor representation. The model parameter compression is achieved using a hierarchical and low-order symmetric tensor approximation. In this way, complex high-order feature interactions can be captured with fewer parameters. Moreover, The TPAM preserves the inherent interpretability of additive models, facilitating transparent decision-making and the extraction of meaningful feature values. Additionally, leveraging TPAM's transparency and ability to handle higher-order features, it is used as a post-processing module for other interpretation models by introducing two variants for class activation maps. Experimental results on a series of datasets demonstrate that TPAM can enhance accuracy by up to 30\%, and compression rate by up to 5 times, while maintaining a good interpretability.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper mainly aims to address the following issues:

1. **Loss of High-Order Data Structure**: Traditional additive models typically require data to be vectorized when handling high-order data, which leads to the loss of data structure information and increases computational complexity, thereby affecting system performance.

2. **Balance Between Model Interpretability and Accuracy**: Although additive models have good interpretability, capturing high-order feature interactions requires a large amount of computational resources and results in an exponential growth of the weight space.

3. **Accuracy of Class Activation Maps (CAM)**: Existing CAM techniques can only assign a single weight to feature maps, which limits the accuracy of the generated saliency maps.

To address the above issues, the authors propose the Tensor Polynomial Additive Model (TPAM), which directly uses high-order tensors as input, avoiding the loss of data structure caused by vectorization. It compresses model parameters through low-rank symmetric tensor approximation, achieving better performance, compression rate, and interpretability. Additionally, TPAM is used as a post-processing module for other interpretive models, introducing two variants—P-CAM and PI-CAM—to improve the granularity and accuracy of saliency maps. Experimental results show that TPAM improves classification accuracy on multiple datasets while maintaining good interpretability.