Abstract:While Large Language Models (LLMs) have demonstrated proficiency in handling complex queries, much of the past work has depended on extensively annotated datasets by human experts. However, this reliance on fully-supervised annotations poses scalability challenges, particularly as models and data requirements grow. To mitigate this, we explore the potential of enhancing LLMs' reasoning abilities with minimal human supervision. In this work, we introduce self-reinforcement, which begins with Supervised Fine-Tuning (SFT) of the model using a small collection of annotated questions. Then it iteratively improves LLMs by learning from the differences in responses from the SFT and unfinetuned models on unlabeled questions. Our approach provides an efficient approach without relying heavily on extensive human-annotated explanations. However, current reasoning benchmarks typically only include golden-reference answers or rationales. Therefore, we present \textsc{PuzzleBen}, a weakly supervised benchmark that comprises 25,147 complex questions, answers, and human-generated rationales across various domains, such as brainteasers, puzzles, riddles, parajumbles, and critical reasoning tasks. A unique aspect of our dataset is the inclusion of 10,000 unannotated questions, enabling us to explore utilizing fewer supersized data to boost LLMs' inference capabilities. Our experiments underscore the significance of \textsc{PuzzleBen}, as well as the effectiveness of our methodology as a promising direction in future endeavors. Our dataset and code will be published soon on \texttt{Anonymity Link}.

Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge.

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems.

Cause-Effect Knowledge Acquisition and Neural Association Model for Solving A Set of Winograd Schema Problems

WSC+: Enhancing The Winograd Schema Challenge Using Tree-of-Experts

Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction

Commonsense Knowledge Enhanced Embeddings for Solving Pronoun Disambiguation Problems in Winograd Schema Challenge

Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation

Generalised Winograd Schema and its Contextuality

Schema-Aware Multi-Task Learning for Complex Text-to-SQL

Task-Related Pretraining with Whole Word Masking for Chinese Coherence Evaluation.

Optimizing Language Model's Reasoning Abilities with Weak Supervision

Unsupervised Deep Structured Semantic Models for Commonsense Reasoning.

EvoGrad: A Dynamic Take on the Winograd Schema Challenge with Human Adversaries

DeepStruct: Pretraining of Language Models for Structure Prediction.

TUTA: Tree-based Transformers for Generally Structured Table Pre-training

Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge

Structure-aware Fine-tuning for Code Pre-trained Models

Structure-aware Pre-training for Table Understanding with Tree-based Transformers.

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale