Abstract:Natural language understanding (NLU) of text is a fundamental challenge in AI, and it has received significant attention throughout the history of NLP research. This primary goal has been studied under different tasks, such as Question Answering (QA) and Textual Entailment (TE). In this thesis, we investigate the NLU problem through the QA task and focus on the aspects that make it a challenge for the current state-of-the-art technology. This thesis is organized into three main parts: In the first part, we explore multiple formalisms to improve existing machine comprehension systems. We propose a formulation for abductive reasoning in natural language and show its effectiveness, especially in domains with limited training data. Additionally, to help reasoning systems cope with irrelevant or redundant information, we create a supervised approach to learn and detect the essential terms in questions. In the second part, we propose two new challenge datasets. In particular, we create two datasets of natural language questions where (i) the first one requires reasoning over multiple sentences; (ii) the second one requires temporal common sense reasoning. We hope that the two proposed datasets will motivate the field to address more complex problems. In the final part, we present the first formal framework for multi-step reasoning algorithms, in the presence of a few important properties of language use, such as incompleteness, ambiguity, etc. We apply this framework to prove fundamental limitations for reasoning algorithms. These theoretical results provide extra intuition into the existing empirical evidence in the field.

AmazUtah_NLP at SemEval-2024 Task 9: A MultiChoice Question Answering System for Commonsense Defying Reasoning

BAMO at SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

DaVinci at SemEval-2024 Task 9: Few-shot prompting GPT-3.5 for Unconventional Reasoning

Hard to Cheat: A Turing Test based on Answering Questions about Images

uTeBC-NLP at SemEval-2024 Task 9: Can LLMs be Lateral Thinkers?

SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

BRAINTEASER: Lateral Thinking Puzzles for Large Language Models

NLP at UC Santa Cruz at SemEval-2024 Task 5: Legal Answer Validation using Few-Shot Multi-Choice QA

AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzles

Abdelhak at SemEval-2024 Task 9 : Decoding Brainteasers, The Efficacy of Dedicated Models Versus ChatGPT

MasonTigers at SemEval-2024 Task 9: Solving Puzzles with an Ensemble of Chain-of-Thoughts

Reasoning-Driven Question-Answering for Natural Language Understanding

Addressing a Question Answering Challenge by Combining Statistical Methods with Inductive Rule Learning and Reasoning

COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes

SemEval-2020 Task 4: Commonsense Validation and Explanation.

Declarative Question Answering over Knowledge Bases Containing Natural Language Text with Answer Set Programming

Missed Connections: Lateral Thinking Puzzles for Large Language Models

Question-Answering of UGC

Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversals

Neural Module Networks for Reasoning over Text

Answering Science Exam Questions Using Query Rewriting with Background Knowledge