Abstract:Recently, Large Language Models (LLMs) have garnered increasing attention in the field of natural language processing, revolutionizing numerous downstream tasks with powerful reasoning and generation abilities. For example, In-Context Learning (ICL) introduces a fine-tuning-free paradigm, allowing out-of-the-box LLMs to execute downstream tasks by analogy learning without any fine-tuning. Besides, in a fine-tuning-dependent paradigm where substantial training data exists, Parameter-Efficient Fine-Tuning (PEFT), as the cost-effective methods, enable LLMs to achieve excellent performance comparable to full fine-tuning. However, these fascinating techniques employed by LLMs have not been fully exploited in the ABSA field. Previous works probe LLMs in ABSA by merely using randomly selected input-output pairs as demonstrations in ICL, resulting in an incomplete and superficial evaluation. In this paper, we shed light on a comprehensive evaluation of LLMs in the ABSA field, involving 13 datasets, 8 ABSA subtasks, and 6 LLMs. Specifically, we design a unified task formulation to unify ``multiple LLMs for multiple ABSA subtasks in multiple paradigms.'' For the fine-tuning-dependent paradigm, we efficiently fine-tune LLMs using instruction-based multi-task learning. For the fine-tuning-free paradigm, we propose 3 demonstration selection strategies to stimulate the few-shot abilities of LLMs. Our extensive experiments demonstrate that LLMs achieve a new state-of-the-art performance compared to fine-tuned Small Language Models (SLMs) in the fine-tuning-dependent paradigm. More importantly, in the fine-tuning-free paradigm where SLMs are ineffective, LLMs with ICL still showcase impressive potential and even compete with fine-tuned SLMs on some ABSA subtasks.

Analyzing Large Language Models for Classroom Discussion Assessment

Evaluating Large Language Models in Analysing Classroom Dialogue

Spoken Language Intelligence of Large Language Models for Language Learning

Applying Large Language Models for Automated Essay Scoring for Non-Native Japanese

A Comprehensive Analysis of the Effectiveness of Large Language Models As Automatic Dialogue Evaluators

A Comprehensive Evaluation of Large Language Models on Aspect-Based Sentiment Analysis

A Large Language Model Approach to Educational Survey Feedback Analysis

Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course

DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation

Large Language Models as Partners in Student Essay Evaluation

LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models

Assessing Large Language Models on Climate Information

Comparison of Large Language Models for Generating Contextually Relevant Questions

Evaluating Large Language Models at Evaluating Instruction Following

Pronunciation Assessment with Multi-modal Large Language Models

Large Language Models for In-Context Student Modeling: Synthesizing Student's Behavior in Visual Programming

NLPBench: Evaluating Large Language Models on Solving NLP Problems

Evaluating Large Language Models for automatic analysis of teacher simulations

Large Language Models for Code Analysis: Do LLMs Really Do Their Job?

Are Large Language Models Good Essay Graders?