Abstract:Recently, with the vigorous development of deep learning and multimedia technology, intelligent urban computing has received more and more extensive attention from academia and industry. Unfortunately, most of the related technologies are black-box paradigms that lack interpretability. Among them, video event recognition is a basic technology. Event contains multiple concepts and their rich interactions, which can assist us to construct explainable event recognition methods. However, the crucial concepts needed to recognize events have various temporal existing patterns, and the relationship between events and the temporal characteristics of concepts has not been fully exploited. This brings great challenges for concept-based event categorization. To address the above issues, we introduce the temporal concept receptive field, which is the length of the temporal window size required to capture key concepts for concept-based event recognition methods. Accordingly, we introduce the temporal dynamic convolution (TDC) to model the temporal concept receptive field dynamically according to different events. Its core idea is to combine the results of multiple convolution layers with the learned coefficients from two complementary perspectives. These convolution layers contain a variety of kernel sizes, which can provide temporal concept receptive fields of different lengths. Similarly, we also propose the cross-domain temporal dynamic convolution (CrTDC) with the help of the rich relationship between different concepts. Different coefficients can help us to capture suitable temporal concept receptive field sizes and highlight crucial concepts to obtain accurate and complete concept representations for event analysis. Based on the TDC and CrTDC, we introduce the temporal dynamic concept modeling network (TDCMN) for explainable video event recognition. We evaluate TDCMN on large-scale and challenging datasets FCVID, ActivityNet, and CCV. Experimental results show that TDCMN significantly improves the event recognition performance of concept-based methods, and the explainability of our method inspires us to construct more explainable models from the perspective of the temporal concept receptive field.

Incorporating Temporal Cues and AC-GCN to Improve Temporal Relation Classification

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Temporal Knowledge Graph Reasoning Based on Graph Convolution Network

Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction

Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence Labeling with GCN

A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events

TodyNet: Temporal Dynamic Graph Neural Network for Multivariate Time Series Classification

Extracting Temporal Event Relation with Syntax-guided Graph Transformer

Entity-Aware Self-Attention and Contextualized GCN for Enhanced Relation Extraction in Long Sentences

Content Temporal Relation Network for temporal action proposal generation

Combining contextual neural networks for time series classification

TC-GAT: Graph Attention Network for Temporal Causality Discovery

Syntax-based dynamic latent graph for event relation extraction

Using Context Events in Neural Network Models for Event Temporal Status Identification

Event detection from text using path-aware graph convolutional network

Tightly-coupled Convolutional Neural Network with Spatial-Temporal Memory for Text Classification.

Classifying Relations Via Long Short Term Memory Networks along Shortest Dependency Paths

DTGCN: a method combining dependency tree and graph convolutional networks for Chinese long-interval named entity relationship extraction

Temporal Dynamic Concept Modeling Network for Explainable Video Event Recognition

Novel target attention convolutional neural network for relation classification

Temporal Relation Extraction with Joint Semantic and Syntactic Attention