Análise de ambiguidade linguística em modelos de linguagem de grande escala (LLMs)
Lavínia de Carvalho Moraes,Irene Cristina Silvério,Rafael Alexandre Sousa Marques,Bianca de Castro Anaia,Dandara Freitas de Paula,Maria Carolina Schincariol de Faria,Iury Cleveston,Alana de Santana Correia,Raquel Meister Ko Freitag
2024-04-25
Abstract:Linguistic ambiguity continues to represent a significant challenge for natural language processing (NLP) systems, notwithstanding the advancements in architectures such as Transformers and BERT. Inspired by the recent success of instructional models like ChatGPT and Gemini (In 2023, the artificial intelligence was called Bard.), this study aims to analyze and discuss linguistic ambiguity within these models, focusing on three types prevalent in Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a corpus comprising 120 sentences, both ambiguous and unambiguous, for classification, explanation, and disambiguation. The models capability to generate ambiguous sentences was also explored by soliciting sets of sentences for each type of ambiguity. The results underwent qualitative analysis, drawing on recognized linguistic references, and quantitative assessment based on the accuracy of the responses obtained. It was evidenced that even the most sophisticated models, such as ChatGPT and Gemini, exhibit errors and deficiencies in their responses, with explanations often providing inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the need for descriptive studies for supervised learning.
Computation and Language,Artificial Intelligence