Abstract:Existing event stream-based pattern recognition models usually represent the event stream as the point cloud, voxel, image, etc., and design various deep neural networks to learn their features. Although considerable results can be achieved in simple cases, however, the model performance may be limited by monotonous modality expressions, sub-optimal fusion, and readout mechanisms. In this paper, we propose a novel dual-stream framework for event stream-based pattern recognition via differentiated fusion, termed EFV++. It models two common event representations simultaneously, i.e., event images and event voxels. The spatial and three-dimensional stereo information can be learned separately by utilizing Transformer and Graph Neural Network (GNN). We believe the features of each representation still contain both efficient and redundant features and a sub-optimal solution may be obtained if we directly fuse them without differentiation. Thus, we divide each feature into three levels and retain high-quality features, blend medium-quality features, and exchange low-quality features. The enhanced dual features will be fed into the fusion Transformer together with bottleneck features. In addition, we introduce a novel hybrid interaction readout mechanism to enhance the diversity of features as final representations. Extensive experiments demonstrate that our proposed framework achieves state-of-the-art performance on multiple widely used event stream-based classification datasets. Specifically, we achieve new state-of-the-art performance on the Bullying10k dataset, i.e., $90.51\%$, which exceeds the second place by $+2.21\%$. The source code of this paper has been released on \url{<a class="link-external link-https" href="https://github.com/Event-AHU/EFV_event_classification/tree/EFVpp" rel="external noopener nofollow">this https URL</a>}.

Event Co-reference Disambiguation with Enhanced Semantics of Multi-layer Attention.

Entity disambiguation method based on Graph Attention Networks

Deep Neural Architectures for Joint Named Entity Recognition and Disambiguation

Mention and Entity Description Co-Attention for Entity Disambiguation

Advancing document-level event extraction: Integration across texts and reciprocal feedback

Semantic Structure Enhanced Event Causality Identification

Event Detection with Dual Relational Graph Attention Networks.

Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering

Hybrid Attention Based Neural Architecture for Text Semantics Similarity Measurement

Event Detection via Gated Multilingual Attention Mechanism.

Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition

Bearing Fault Diagnosis Method Based on Attention Mechanism and Multilayer Fusion Network.

Attention-Based Joint Entity Linking with Entity Embedding

Discriminative Reasoning with Sparse Event Representation for Document-level Event-Event Relation Extraction

Enhancing Both Local and Global Entity Linking Models with Attention.

Joint event causality extraction using dual-channel enhanced neural network

A graph propagation model with rich event structures for joint event relation extraction

Multi-level Connection Enhanced Representation Learning for Script Event Prediction

A Multi-Channel Hierarchical Graph Attention Network for Open Event Extraction

Multi-Head Self-Attention Gated-Dilated Convolutional Neural Network for Word Sense Disambiguation