Abstract:Selectively processing noisy utterances while effectively disregarding speech-specific elements poses no considerable challenge for humans, as they exhibit remarkable cognitive abilities to separate semantically significant content from speech-specific noise (i.e. filled pauses, disfluencies, and restarts). These abilities may be driven by mechanisms based on acquired grammatical rules that compose abstract syntactic-semantic structures within utterances. Segments without syntactic and semantic significance are consistently disregarded in these structures. The structures, in tandem with lexis, likely underpin language comprehension and thus facilitate effective communication. In our study, grounded in linguistically motivated experiments, we investigate whether large language models (LLMs) can effectively perform analogical speech comprehension tasks. In particular, we examine the ability of LLMs to extract well-structured utterances from transcriptions of noisy dialogues. We conduct two evaluation experiments in the Polish language scenario, using a~dataset presumably unfamiliar to LLMs to mitigate the risk of data contamination. Our results show that not all extracted utterances are correctly structured, indicating that either LLMs do not fully acquire syntactic-semantic rules or they acquire them but cannot apply them effectively. We conclude that the ability of LLMs to comprehend noisy utterances is still relatively superficial compared to human proficiency in processing them.

From Submit to Submitted via Submission: On Lexical Rules in Large-Scale Lexicon Acquisition

When is Wall a Pared and when a Muro? -- Extracting Rules Governing Lexical Selection

Rule-Based Spanish Morphological Analyzer Built From Spell Checking Lexicon

Distilling Task-specific Logical Rules from Large Pre-trained Models

Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognition

Sparse Logistic Regression with High-order Features for Automatic Grammar Rule Extraction from Treebanks

Lexicalization and Grammar Development

A Bootstrap Approach to Automatically Generating Lexical Transfer Rules

Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts

Evaluating Large Language Models with Tests of Spanish as a Foreign Language: Pass or Fail?

Generating bilingual example sentences with large language models as lexicography assistants

Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterances

Words as Beacons: Guiding RL Agents with High-Level Language Prompts

A New Corpus of Lexical Substitution and Word Blend Errors: Probing the Semantic Structure of Lemma Access Failures

Can LLMs Follow Simple Rules?

Memory-Based Lexical Acquisition and Processing

The Greedy and Recursive Search for Morphological Productivity

Interpretation of Natural Language Rules in Conversational Machine Reading

Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic

Automated generation of terminological dictionary from textual business rules

Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL