Abstract:Fuzzy integral fusion has been shown as an effective tool for enhancing classification accuracy while also achieving explainability. With the deep learning boom in the past decade, many researchers have investigated the advantages of fusing various deep neural networks with fuzzy integral techniques in computer vision. However, recent studies focus on only the explainable fusion process. Thus, features learned by each deep neural network are difficult to understand. Moreover, deep neural networks are usually trained on the ImageNet dataset. Whereas the effectiveness of applying the fuzzy integral methods to this dataset is yet to be investigated. This is the gap that motivates our research study. To address this issue, we explore fuzzy integral fusion classification models that make both the fusion process and extracted features explainable. Specifically, we use two well-known fuzzy integral fusion methods (i.e., Sugeno integral and Choquet integral) to combine three explainable deep learning features (i.e., shape, texture, and color) in a manner that mimics human visual recognition processes. The originality of our work includes the emphasis on complete explainability in the classification process, the investigation of applying fuzzy integral methods to the ImageNet dataset, and extensive experimental validation of the effectiveness of fuzzy integral. Computational experiments show that fuzzy integral fusion can improve classification accuracy by 14.6% compared with an individual deep neural network on subsets derived from the ImageNet dataset. Furthermore, fuzzy integral fusion helps understand contributions, relationships, and interactions among the three features (shape, texture, and color) for each class, providing convincing evidence for the final classification result. Consequently, the proposed models not only achieve impressive performance but also provide a thorough understanding of how these models work. The code is available at https://github.com/yifanwang97/Explainable-fuzzy-integral .

Integration of Feature and Decision Fusion With Deep Learning Architectures for Video Classification

Fusing Multi-Stream Deep Networks for Video Classification

Multi-Stream Multi-Class Fusion of Deep Networks for Video Classification.

Deep Fusion Module for Video Action Recognition

Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification

Look and Listen: A Multi-modality Late Fusion Approach to Scene Classification for Autonomous Machines

Deep Feature Fusion for High-Resolution Aerial Scene Classification

Multi-Modal Image Fusion Via Deep Laplacian Pyramid Hybrid Network

Decision Fusion Networks for Image Classification

Fusion of Explainable Deep Learning Features Using Fuzzy Integral in Computer Vision

Decision-Level Fusion Method Based on Deep Learning.

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond

Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification

Multimodal Deep Representation Learning for Video Classification

CAMF: an Interpretable Infrared and Visible Image Fusion Network Based on Class Activation Mapping

Two Stream LSTM: A Deep Fusion Framework for Human Action Recognition

Multi-scale Spatiotemporal Information Fusion Network for Video Action Recognition

A Video Classification Method Based on Spatiotemporal Detail Attention and Feature Fusion

A Deep Learning Framework for Infrared and Visible Image Fusion Without Strict Registration