Abstract:The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.

Cruciform: Solving Crosswords with Natural Language Processing

Cruciform: Solving Crosswords with Natural Language Processing.

A Crossword Solving System Based on Monte Carlo Tree Search

The WebCrow French Crossword Solver

Italian Crossword Generator: Enhancing Education through Interactive Word Puzzles

Crossword Puzzle Resolution Via Monte Carlo Tree Search

Language Models are Crossword Solvers

SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning

Generating News-Centric Crossword Puzzles As A Constraint Satisfaction and Optimization Problem

What Makes Cryptic Crosswords Challenging for LLMs?

Harnessing LLMs for Educational Content-Driven Italian Crossword Generation

Proving that Cryptic Crossword Clue Answers are Correct

Are LLMs Good Cryptic Crossword Solvers?

A Turkish Educational Crossword Puzzle Generator

Clue-Instruct: Text-Based Clue Generation for Educational Crossword Puzzles

ArabIcros: AI-Powered Arabic Crossword Puzzle Generation for Educational Applications

Missed Connections: Lateral Thinking Puzzles for Large Language Models

Solving and Generating NPR Sunday Puzzles with Large Language Models

Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models

AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzles

PuzzLing Machines: A Challenge on Learning From Small Data