Abstract:Data representation is a fundamental task in machine learning. The representation of data affects the performance of the whole machine learning system. In a long history, the representation of data is done by feature engineering, and researchers aim at designing better features for specific tasks. Recently, the rapid development of deep learning and representation learning has brought new inspiration to various domains. In natural language processing, the most widely used feature representation is the Bag-of-Words model. This model has the data sparsity problem and cannot keep the word order information. Other features such as part-of-speech tagging or more complex syntax features can only fit for specific tasks in most cases. This thesis focuses on word representation and document representation. We compare the existing systems and present our new model. First, for generating word embeddings, we make comprehensive comparisons among existing word embedding models. In terms of theory, we figure out the relationship between the two most important models, i.e., Skip-gram and GloVe. In our experiments, we analyze three key points in generating word embeddings, including the model construction, the training corpus and parameter design. We evaluate word embeddings with three types of tasks, and we argue that they cover the existing use of word embeddings. Through theory and practical experiments, we present some guidelines for how to generate a good word embedding. Second, in Chinese character or word representation. We introduce the joint training of Chinese character and word. ... Third, for document representation, we analyze the existing document representation models, including recursive NNs, recurrent NNs and convolutional NNs. We point out the drawbacks of these models and present our new model, the recurrent convolutional neural networks. ...

A comparative study for wordnet guided text representation

Incorporating Knowledge into Neural Network for Text Representation.

WordNet-based Concept Vector Space Model for Text Classification

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Learning Word Representations by Jointly Modeling Syntagmatic and Paradigmatic Relations.

Adaptive Concept Resolution for document representation and its applications in text mining.

An Approach To Concept-Obtained Text Summarization

A Unified Model for Word Sense Representation and Disambiguation.

Concept chain based text clustering

Document Clustering Based on Word Sense Cluster

Statistical Analysis of Multisensory and Text-Derived Representations on Concept Learning

xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks

Sentence Vector Model Based on Implicit Word Vector Expression

Semantic Hilbert Space for Text Representation Learning

Word and Document Embeddings based on Neural Network Approaches

Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchy

A Comparative Study of Sentence Embedding Models for Assessing Semantic Variation

A Semantic approach for effective document clustering using WordNet

DRWS: A Model for Learning Distributed Representations for Words and Sentences.

RC-NET: A General Framework for Incorporating Knowledge into Word Representations.

Graph-Based Text Similarity Measurement by Exploiting Wikipedia As Background Knowledge