Abstract:Data representation is a fundamental task in machine learning. The representation of data affects the performance of the whole machine learning system. In a long history, the representation of data is done by feature engineering, and researchers aim at designing better features for specific tasks. Recently, the rapid development of deep learning and representation learning has brought new inspiration to various domains. In natural language processing, the most widely used feature representation is the Bag-of-Words model. This model has the data sparsity problem and cannot keep the word order information. Other features such as part-of-speech tagging or more complex syntax features can only fit for specific tasks in most cases. This thesis focuses on word representation and document representation. We compare the existing systems and present our new model. First, for generating word embeddings, we make comprehensive comparisons among existing word embedding models. In terms of theory, we figure out the relationship between the two most important models, i.e., Skip-gram and GloVe. In our experiments, we analyze three key points in generating word embeddings, including the model construction, the training corpus and parameter design. We evaluate word embeddings with three types of tasks, and we argue that they cover the existing use of word embeddings. Through theory and practical experiments, we present some guidelines for how to generate a good word embedding. Second, in Chinese character or word representation. We introduce the joint training of Chinese character and word. ... Third, for document representation, we analyze the existing document representation models, including recursive NNs, recurrent NNs and convolutional NNs. We point out the drawbacks of these models and present our new model, the recurrent convolutional neural networks. ...

Text Classification With Document Embeddings

Knowledge-based Document Embedding for Cross-Domain Text Classification

Incorporating Knowledge into Neural Network for Text Representation.

Improve Document Embedding for Text Categorization Through Deep Siamese Neural Network

Bag-of-Embeddings for Text Classification.

NewsEmbed: Modeling News through Pre-trained Document Representations

Word and Document Embeddings based on Neural Network Approaches

Doc2Im: document to image conversion through self-attentive embedding

Improving Document Classification with Multi-Sense Embeddings

DCNN-BiGRU Text Classification Model Based on BERT Embedding

Document classification with distributions of word vectors

Category Enhanced Word Embedding.

Khmer Text Classification Using Word Embedding and Neural Networks

Adaptive Region Embedding for Text Classification.

Neural Text Classification by Jointly Learning to Cluster and Align

A Deep Learning Short Text Classification Model Integrating Part of Speech Features

Document Classification Based on Word Vectors

Semantic Regularities in Document Representations.

Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds

Empirical Study of Named Entity Recognition Performance Using Distribution-aware Word Embedding

It's All in the Embedding! Fake News Detection Using Document Embeddings