Abstract:The fusion-based classification of hyperspectral (HS) and light detection and ranging (LiDAR) images has become a prominent research topic, as their complementary information can effectively improve classification performance. The current methods encompass pixel-, feature- and decision-level fusion. Among them, feature- and decision-level fusion have emerged as the mainstream approaches. Collaborative fusion of these two levels can enhance classification accuracy. Although various methods have been proposed, some shortcomings still exist. On one hand, current methods ignore the shared advanced features between HS and LiDAR images, impeding the integration of multimodal features and thereby limiting the classification performance. On the other hand, the existing methods face difficulties in achieving a balance between feature- and decision-level contributions, or they simply overlook the significance of one level and fail to utilize it effectively. In this paper, we propose a novel feature-decision level collaborative fusion network (FDCFNet) for hyperspectral and LiDAR classification to alleviate these problems. Specifically, a multilevel interactive fusion module is proposed to indirectly connect hyperspectral and LiDAR flows to refine the spectral-elevation information. Moreover, the fusion features of the intermediate branch can further enhance the shared-complementary information of hyperspectral and LiDAR to reduce the modality differences. In addition, a dynamic weight selection strategy is meticulously designed to adaptively assign weight to the output of three branches at the decision level. Experiments on three public benchmark datasets demonstrate the effectiveness of the proposed methods.

Cognitive Computing – ICCC 2020: 4th International Conference, Held as Part of the Services Conference Federation, SCF 2020, Honolulu, HI, USA, September 18-20, 2020, Proceedings

HFF - Hybrid Feature Fusion Model for Click-Through Rate Prediction.

A Late Fusion Approach for Harnessing Multi-Cnn Model High-Level Features

Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

HCFNN: High-order Coverage Function Neural Network for Image Classification

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

Feature Interaction Fusion Self-Distillation Network For CTR Prediction

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Fusion Matters: Learning Fusion in Deep Click-through Rate Prediction Models

HDCCT: Hybrid Densely Connected CNN and Transformer for Infrared and Visible Image Fusion

DCFusion: Dual-Headed Fusion Strategy and Contextual Information Awareness for Infrared and Visible Remote Sensing Image

End-Edge Collaborative Inference of Convolutional Fuzzy Neural Networks for Big Data-Driven Internet of Things

An interlayer feature fusion-based heterogeneous graph neural network

Deep Converged Network for Attention

IFCNN: A General Image Fusion Framework Based on Convolutional Neural Network.

HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion

DeConFuse : A Deep Convolutional Transform based Unsupervised Fusion Framework

A FEATURE EMBEDDING STRATEGY FOR HIGH-LEVEL CNN REPRESENTATIONS FROM MULTIPLE CONVNETS

Enabling Efficient Deep Convolutional Neural Network-based Sensor Fusion for Autonomous Driving

A Hierarchical Coarse–Fine Adaptive Fusion Network for the Joint Classification of Hyperspectral and LiDAR Data