Abstract:Data organized in tabular format is ubiquitous in real-world applications, and users often craft tables with biased feature definitions and flexibly set prediction targets of their interests. Thus, a rapid development of a robust, effective, dataset-versatile, user-friendly tabular prediction approach is highly desired. While Gradient Boosting Decision Trees (GBDTs) and existing deep neural networks (DNNs) have been extensively utilized by professional users, they present several challenges for casual users, particularly: (i) the dilemma of model selection due to their different dataset preferences, and (ii) the need for heavy hyperparameter searching, failing which their performances are deemed inadequate. In this paper, we delve into this question: Can we develop a deep learning model that serves as a sure bet solution for a wide range of tabular prediction tasks, while also being user-friendly for casual users? We delve into three key drawbacks of deep tabular models, encompassing: (P1) lack of rotational variance property, (P2) large data demand, and (P3) over-smooth solution. We propose ExcelFormer, addressing these challenges through a semi-permeable attention module that effectively constrains the influence of less informative features to break the DNNs' rotational invariance property (for P1), data augmentation approaches tailored for tabular data (for P2), and attentive feedforward network to boost the model fitting capability (for P3). These designs collectively make ExcelFormer a sure bet solution for diverse tabular datasets. Extensive and stratified experiments conducted on real-world datasets demonstrate that our model outperforms previous approaches across diverse tabular data prediction tasks, and this framework can be friendly to casual users, offering ease of use without the heavy hyperparameter tuning. The codes are available at https://github.com/whatashot/excelformer.

Web Table Column Type Detection Using Deep Learning and Probability Graph Model.

Graph Neural Network Approach to Semantic Type Detection in Tables

Learning Semantic Annotations for Tabular Data

Web Table Understanding by Collective Inference

DGA-GNN: Dynamic Grouping Aggregation GNN for Fraud Detection

Predicting Semantically Linkable Knowledge In Developer Online Forums Via Convolutional Neural Network

KGLink: A column type annotation method that combines knowledge graph and pre-trained language model

AdaTyper: Adaptive Semantic Column Type Detection

Understanding the Semantic Structures of Tables with a Hybrid Deep Neural Network Architecture

Sherlock: A Deep Learning Approach to Semantic Data Type Detection

DCoM: A Deep Column Mapper for Semantic Data Type Detection

Annotating Columns with Pre-trained Language Models

Scalable Column Concept Determination for Web Tables Using Large Knowledge Bases

Research and application of Chinese semantic understanding teaching system based on knowledge graph and deep learning

Comprehending Semantic Types in JSON Data with Graph Neural Networks

TableDet: An end-to-end deep learning approach for table detection and table image classification in data sheet images

UTTSR: A Novel Non-Structured Text Table Recognition Model Powered by Deep Learning Technology

A Deep Semantic Segmentation Model for Image-based Table Structure Recognition

An efficient multi-relational Naïve Bayesian classifier based on semantic relationship graph

Table Detection from Plain Text Using Machine Learning and Document Structure

Can a Deep Learning Model Be a Sure Bet for Tabular Prediction?