Abstract:Tabular data is the most widely used data format in machine learning (ML). While tree-based methods outperform DL-based methods in supervised learning, recent literature reports that self-supervised learning with Transformer-based models outperforms tree-based methods. In the existing literature on self-supervised learning for tabular data, contrastive learning is the predominant method. In contrastive learning, data augmentation is important to generate different views. However, data augmentation for tabular data has been difficult due to the unique structure and high complexity of tabular data. In addition, three main components are proposed together in existing methods: model structure, self-supervised learning methods, and data augmentation. Therefore, previous works have compared the performance without comprehensively considering these components, and it is not clear how each component affects the actual performance. In this study, we focus on data augmentation to address these issues. We propose a novel data augmentation method, $\textbf{M}$ask $\textbf{T}$oken $\textbf{R}$eplacement ($\texttt{MTR}$), which replaces the mask token with a portion of each tokenized column; $\texttt{MTR}$ takes advantage of the properties of Transformer, which is becoming the predominant DL-based architecture for tabular data, to perform data augmentation for each column embedding. Through experiments with 13 diverse public datasets in both supervised and self-supervised learning scenarios, we show that $\texttt{MTR}$ achieves competitive performance against existing data augmentation methods and improves model performance. In addition, we discuss specific scenarios in which $\texttt{MTR}$ is most effective and identify the scope of its application. The code is available at <a class="link-external link-https" href="https://github.com/somaonishi/MTR/" rel="external noopener nofollow">this https URL</a>.

Retrieval Augmented Deep Anomaly Detection for Tabular Data

Feature Interaction-Based Reinforcement Learning for Tabular Anomaly Detection

ATDAD: One-class Adversarial Learning for Tabular Data Anomaly Detection

Data-Efficient and Interpretable Tabular Anomaly Detection

ActiveAD: Enhancing Anomaly Detection in Tabular Data through Active Learning Strategies

MCM: Masked Cell Modeling for Anomaly Detection in Tabular Data

Disentangling Tabular Data towards Better One-Class Anomaly Detection

Prototype-oriented Hypergraph Representation Learning for Anomaly Detection in Tabular Data

SemanticMask: A Contrastive View Design for Anomaly Detection in Tabular Data

Boosting Anomaly Detection Using Unsupervised Diverse Test-Time Augmentation

Unsupervised Anomaly Detection for Tabular Data Using Noise Evaluation

DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data

Retrieval-Based Transformer for Table Augmentation

Deep Anomaly Detection Via Active Anomaly Search.

Anomaly Detection of Tabular Data Using LLMs

Rethinking Data Augmentation for Tabular Data in Deep Learning

AnoGAN for Tabular Data: A Novel Approach to Anomaly Detection

TracInAD: Measuring Influence for Anomaly Detection

BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data

SORTAD: Self-Supervised Optimized Random Transformations for Anomaly Detection in Tabular Data