Abstract:Spiking neural networks (SNNs) are attracting widespread interest due to their biological plausibility, energy efficiency, and powerful spatiotemporal information representation ability. Given the critical role of attention mechanisms in enhancing neural network performance, the integration of SNNs and attention mechanisms exhibits tremendous potential to deliver energy-efficient and high-performance computing paradigms. In this article, we present a novel temporal-channel joint attention mechanism for SNNs, referred to as TCJA-SNN. The proposed TCJA-SNN framework can effectively assess the significance of spike sequence from both spatial and temporal dimensions. More specifically, our essential technical contribution lies on: 1) we employ the squeeze operation to compress the spike stream into an average matrix. Then, we leverage two local attention mechanisms based on efficient 1-D convolutions to facilitate comprehensive feature extraction at the temporal and channel levels independently and 2) we introduce the cross-convolutional fusion (CCF) layer as a novel approach to model the interdependencies between the temporal and channel scopes. This layer effectively breaks the independence of these two dimensions and enables the interaction between features. Experimental results demonstrate that the proposed TCJA-SNN outperforms the state-of-the-art (SOTA) on all standard static and neuromorphic datasets, including Fashion-MNIST, CIFAR10, CIFAR100, CIFAR10-DVS, N-Caltech 101, and DVS128 Gesture. Furthermore, we effectively apply the TCJA-SNN framework to image generation tasks by leveraging a variation autoencoder. To the best of our knowledge, this study is the first instance where the SNN-attention mechanism has been employed for high-level classification and low-level generation tasks. Our implementation codes are available at https://github.com/ridgerchu/TCJA.

Topic Augmented Neural Response Generation with a Joint Attention Mechanism

Context-Controlled Topic-Aware Neural Response Generation for Open-Domain Dialog Systems

Neural Response Generation with Dynamic Vocabularies

Response Selection with Topic Clues for Retrieval-based Chatbots

Topic-aware chatbot using Recurrent Neural Networks and Nonnegative Matrix Factorization

Get The Point of My Utterance! Learning Towards Effective Responses with Multi-Head Attention Mechanism

Neural Response Generation with Relevant Emotions for Short Text Conversation

TopicAns: Topic-informed Architecture for Answer Recommendation on Technical Q&A Site

Affective Neural Response Generation

Joint Learning for Addressee Selection and Response Generation in Multi-Party Conversation

Emotional Conversation Generation Based on a Bayesian Deep Neural Network

Topic Enhanced Controllable CVAE for Dialogue Generation (student Abstract).

Neural Conversation Generation with Auxiliary Emotional Supervised Models

Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation

Self-Attention-Based Message-Relevant Response Generation for Neural Conversation Model

Neural Contextual Conversation Learning with Labeled Question-Answering Pairs

A Hybrid Retrieval-Generation Neural Conversation Model.

Joint Learning of Question Answering and Question Generation

Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access

Topic-to-Essay Generation with Neural Networks

TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks