Abstract:Entailment recognition is an important paradigm of reasoning that judges if a hypothesis can be inferred from given premises. However, previous efforts mainly concentrate on text-based reasoning as recognizing textual entailment (RTE) , where the hypotheses and premises are both textual. In fact, humans’ reasoning process has the characteristic of cross-media reasoning . It is naturally based on the joint inference with different sensory organs, which represent complementary reasoning cues from unique perspectives as language, vision, and audition. How to realize cross-media reasoning has been a significant challenge to achieve the breakthrough for width and depth of entailment recognition. Therefore, this article extends RTE to a novel reasoning paradigm: recognizing cross-media entailment (RCE) , and proposes heterogeneous interactive learning (HIL) approach. Specifically, HIL recognizes entailment relationships via cross-media joint inference, from image-text premises to text hypotheses. It is an end-to-end architecture with two parts: (1) Cross-media hybrid embedding is proposed to perform cross embedding of premises and hypotheses for generating their fine-grained representations. It aims to achieve the alignment of cross-media inference cues via image-text and text-text interactive attention. (2) Heterogeneous joint inference is proposed to construct a heterogeneous interaction tensor space and extract semantic features for entailment recognition. It aims to simultaneously capture the interaction between cross-media premises and hypotheses and distinguish their entailment relationships. Experimental results on widely used Stanford natural language inference (SNLI) dataset with image premises from Flickr30K dataset verify the effectiveness of HIL and the intrinsic inter-media complementarity in reasoning.

Use of multi-strategy to Textual Entailment recognition

Recognizing Textual Entailment with Synthetic Analysis Based on SVM and Feature Value Control

Word Alignment Based Recognizing Textural Entailment

Recognizing Textual Entailment via Multi-task Knowledge Assisted LSTM.

Recognizing Textual Entailment Using Probabilistic Inference.

Labeled Alignment for Recognizing Textual Entailment.

A Survey on Recognizing Textual Entailment as an NLP Evaluation

ICRC_HITSZ at RITE: Leveraging Multiple Classifiers Voting for Textual Entailment Recognition.

The Inference and Identification Models for Textual Entailment

A Logic-based Approach for Recognizing Textual Entailment Supported by Ontological Background Knowledge

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Max-Cosine Matching Based Neural Models for Recognizing Textual Entailment

Exploring Lexical, Syntactic, and Semantic Features for Chinese Textual Entailment in NTCIR RITE Evaluation Tasks

What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?

Reading and Thinking: Re-read LSTM Unit for Textual Entailment Recognition.

Chinese Textual Entailment Recognition Enhanced With Word Embedding

BCMI-NLP Labeled-Alignment-Based Entailment System for NTCIR-10 RITE-2 Task.

Recognizing Text Entailment Via Bidirectional Lstm Model With Inner-Attention

RCE-HIL

NWNU Minimum Information Recognizing Entailment System for NTCIR-11 RITE-3 Task.

ECNUCS: Recognizing Cross-lingual Textual Entailment Using Multiple Text Similarity and Text Difference Measures.