Abstract:Biologically inspired spiking neural networks (SNNs) have garnered considerable attention due to their low-energy consumption and spatio-temporal information processing capabilities. Most existing SNNs training methods first integrate output information across time steps, then adopt the cross-entropy (CE) loss to supervise the prediction of the average representations. However, in this work, we find the method above is not ideal for the SNNs training as it omits the temporal dynamics of SNNs and degrades the performance quickly with the decrease of inference time steps. One tempting method to model temporal correlations is to apply the same label supervision at each time step and treat them identically. Although it can acquire relatively consistent performance across various time steps, it still faces challenges in obtaining SNNs with high performance. Inspired by these observations, we propose Temporal-domain supervised Contrastive Learning (TCL) framework, a novel method to obtain SNNs with low latency and high performance by incorporating contrastive supervision with temporal domain information. Contrastive learning (CL) prompts the network to discern both consistency and variability in the representation space, enabling it to better learn discriminative and generalizable features. We extend this concept to the temporal domain of SNNs, allowing us to flexibly and fully leverage the correlation between representations at different time steps. Furthermore, we propose a Siamese Temporal-domain supervised Contrastive Learning (STCL) framework to enhance the SNNs via augmentation, temporal and class constraints simultaneously. Extensive experimental results demonstrate that SNNs trained by our TCL and STCL can achieve both high performance and low latency, achieving state-of-the-art performance on a variety of datasets (e.g., CIFAR-10, CIFAR-100, and DVS-CIFAR10).

Neural Network for Weighted Signal Temporal Logic

Weighted Graph-Based Signal Temporal Logic Inference Using Neural Networks

Multi-class Temporal Logic Neural Networks

Learning Signal Temporal Logic through Neural Network for Interpretable Classification

TLINet: Differentiable Neural Network Temporal Logic Inference

STONet: A Neural-Operator-Driven Spatio-temporal Network

A Neurosymbolic Approach to the Verification of Temporal Logic Properties of Learning enabled Control Systems

Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specifications

DTW-NN: A novel neural network for time series recognition using dynamic alignment between inputs and weights

Neural Network Weight Comparison for Industrial Causality Discovering and Its Soft Sensing Application

STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep Spiking Neural Networks

A Dual-Layer Network Deep Reinforcement Learning Algorithm for Multi-objective Signal Temporal Logic Tasks

Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networks

stl2vec: Semantic and Interpretable Vector Representation of Temporal Logic

Signal Temporal Logic Neural Predictive Control

Temporal Contrastive Learning for Spiking Neural Networks

Geographically and temporally neural network weighted regression for modeling spatiotemporal non-stationary relationships

Weightless for neural networks based on multi-valued probabilistic logic for node for pattern recognition

S-TLLR: STDP-inspired Temporal Local Learning Rule for Spiking Neural Networks

Differentiable Inference of Temporal Logic Formulas

Interpretable Fault Diagnosis of Rolling Element Bearings with Temporal Logic Neural Network