Abstract:Entailment recognition is an important paradigm of reasoning that judges if a hypothesis can be inferred from given premises. However, previous efforts mainly concentrate on text-based reasoning as recognizing textual entailment (RTE) , where the hypotheses and premises are both textual. In fact, humans’ reasoning process has the characteristic of cross-media reasoning . It is naturally based on the joint inference with different sensory organs, which represent complementary reasoning cues from unique perspectives as language, vision, and audition. How to realize cross-media reasoning has been a significant challenge to achieve the breakthrough for width and depth of entailment recognition. Therefore, this article extends RTE to a novel reasoning paradigm: recognizing cross-media entailment (RCE) , and proposes heterogeneous interactive learning (HIL) approach. Specifically, HIL recognizes entailment relationships via cross-media joint inference, from image-text premises to text hypotheses. It is an end-to-end architecture with two parts: (1) Cross-media hybrid embedding is proposed to perform cross embedding of premises and hypotheses for generating their fine-grained representations. It aims to achieve the alignment of cross-media inference cues via image-text and text-text interactive attention. (2) Heterogeneous joint inference is proposed to construct a heterogeneous interaction tensor space and extract semantic features for entailment recognition. It aims to simultaneously capture the interaction between cross-media premises and hypotheses and distinguish their entailment relationships. Experimental results on widely used Stanford natural language inference (SNLI) dataset with image premises from Flickr30K dataset verify the effectiveness of HIL and the intrinsic inter-media complementarity in reasoning.

The Inference and Identification Models for Textual Entailment

Recognizing Textual Entailment Using Probabilistic Inference.

Chinese Textual Entailment Recognition Based on Syntactic Tree Clipping

A Prototype-Based Rule Inference System Incorporating Linear Functions

Identification of Entailment and Contradiction Relations between Natural Language Sentences: A Neurosymbolic Approach

Entailment Graph Learning with Textual Entailment and Soft Transitivity

A Logic-based Approach for Recognizing Textual Entailment Supported by Ontological Background Knowledge

EntailE: Introducing Textual Entailment in Commonsense Knowledge Graph Completion

Textual Entailment Recognition with Semantic Features from Empirical Text Representation

Max-Cosine Matching Based Neural Models for Recognizing Textual Entailment

Natural Language Inference by Tree-Based Convolution and Heuristic Matching

A Survey of Paraphrasing and Textual Entailment Methods

Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification

XTE: Explainable Text Entailment

A Study of the State of the Art Approaches and Datasets for Multilingual Natural Language Inference

Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds

Revising Image-Text Retrieval via Multi-Modal Entailment

Recognizing Inference in Texts with Markov Logic Networks.

Chinese textual entailment recognition model based on contextual feature extraction

A Logical Pattern Memory Pre-trained Model for Entailment Tree Generation

RCE-HIL