Abstract:Semantic representation and inference is essential for Natural Language Processing (NLP). The state of the art for semantic representation and inference is deep learning, and particularly Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and transformer Self-Attention models. This thesis investigates the use of deep learning for novel semantic representation and inference, and makes contributions in the following three areas: creating training data, improving semantic representations and extending inference learning. In terms of creating training data, we contribute the largest publicly available dataset of real-life factual claims for the purpose of automatic claim verification (MultiFC), and we present a novel inference model composed of multi-scale CNNs with different kernel sizes that learn from external sources to infer fact checking labels. In terms of improving semantic representations, we contribute a novel model that captures non-compositional semantic indicators. By definition, the meaning of a non-compositional phrase cannot be inferred from the individual meanings of its composing words (e.g., hot dog). Motivated by this, we operationalize the compositionality of a phrase contextually by enriching the phrase representation with external word embeddings and knowledge graphs. Finally, in terms of inference learning, we propose a series of novel deep learning architectures that improve inference by using syntactic dependencies, by ensembling role guided attention heads, incorporating gating layers, and concatenating multiple heads in novel and effective ways. This thesis consists of seven publications (five published and two under review).

Enhancing and Combining Sequential and Tree LSTM for Natural Language Inference.

Enhanced Lstm For Natural Language Inference

Natural Language Inference Using Lstm Model With Sentence Fusion

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Context-Aware Tree-Based Convolutional Neural Networks for Natural Language Inference.

Generating Natural Language Inference Chains

DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference

Natural Language Inference by Tree-Based Convolution and Heuristic Matching

Top-down Tree Long Short-Term Memory Networks

Semantics-Aware Inferential Network for Natural Language Understanding

Reading and Thinking: Re-read LSTM Unit for Textual Entailment Recognition.

Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference.

Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference.

Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation

Neural Natural Language Inference Models Enhanced with External Knowledge

Semantic Representation and Inference for NLP

Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference

A Sequential Neural Encoder with Latent Structured Description for Modeling Sentences.

Transformer-Based Contextualized Language Models Joint with Neural Networks for Natural Language Inference in Vietnamese

Syntax-based Attention Model for Natural Language Inference.

Syntax Aware LSTM model for Semantic Role Labeling.