Abstract:Long Short-Term Memory (LSTM) network, a popular deep-learning model, is particularly useful for data with temporal correlation, such as texts, sequences, or time series data, thanks to its well-sought after recurrent network structures designed to capture temporal correlation. In this article, we propose to generalize LSTM to generic machine-learning tasks where data used for training do not have explicit temporal or sequential correlation. Our theme is to explore feature correlation in the original data and convert each instance into a synthetic sentence format by using a two-gram probabilistic language model. More specifically, for each instance represented in the original feature space, our conversion first seeks to horizontally align original features into a sequentially correlated feature vector, resembling to the letter coherence within a word. In addition, a vertical alignment is also carried out to create multiple time points and simulate word sequential order in a sentence ( i.e., word correlation). The two dimensional horizontal-and-vertical alignments not only ensure feature correlations are maximally utilized, but also preserve the original feature values in the new representation. As a result, LSTM model can be utilized to achieve good classification accuracy, even if the underlying data do not have temporal or sequential dependency. Experiments on 20 generic datasets show that applying LSTM to generic data can improve the classification accuracy, compared to conventional machine-learning methods. This research opens a new opportunity for LSTM deep learning to be broadly applied to generic machine-learning tasks.

A Deployment Method to Improve the Generalizability of Recurrent Neural Network

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Wordreg: Mitigating the Gap Between Training and Inference with Worst-Case Drop Regularization

HyperLoRA: Efficient Cross-task Generalization Via Constrained Low-Rank Adapters Generation

DyRep: Bootstrapping Training with Dynamic Re-parameterization

Generalization and Risk Bounds for Recurrent Neural Networks

GRAM: Generalization in Deep RL with a Robust Adaptation Module

Generalizing Long Short-Term Memory Network for Deep Learning from Generic Data

Understanding Generalization in Recurrent Neural Networks.

Evaluating Deep Neural Networks in Deployment (A Comparative and Replicability Study)

Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training

Aware: Adaptive Distributed Training with Computation, Communication and Position Awareness for Deep Learning Model.

On the Provable Generalization of Recurrent Neural Networks

DRRNets: Dynamic Recurrent Routing Via Low-Rank Regularization in Recurrent Neural Networks.

Improving Systematic Generalization Through Modularity and Augmentation

State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations

RigLSTM: Recurrent Independent Grid LSTM for Generalizable Sequence Learning

Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning

Generalize Deep Neural Networks with Adaptive Regularization for Classifying

Investigation on Improving Generalization of Neural Network

Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning