Abstract:The rapid development of large language models (LLMs), like ChatGPT, has resulted in the widespread presence of LLM-generated content on social media platforms, raising concerns about misinformation, data biases, and privacy violations, which can undermine trust in online discourse. While detecting LLM-generated content is crucial for mitigating these risks, current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-AI collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content. This approach introduces two novel tasks: LLM Role Recognition (LLM-RR), a multi-class classification task that identifies specific roles of LLM in content generation, and LLM Influence Measurement (LLM-IM), a regression task that quantifies the extent of LLM involvement in content creation. To support these tasks, we propose LLMDetect, a benchmark designed to evaluate detectors' performance on these new tasks. LLMDetect includes the Hybrid News Detection Corpus (HNDC) for training detectors, as well as DetectEval, a comprehensive evaluation suite that considers five distinct cross-context variations and multi-intensity variations within the same LLM role. This allows for a thorough assessment of detectors' generalization and robustness across diverse contexts. Our empirical validation of 10 baseline detection methods demonstrates that fine-tuned PLM-based models consistently outperform others on both tasks, while advanced LLMs face challenges in accurately detecting their own generated content. Our experimental results and analysis offer insights for developing more effective detection models for LLM-generated content. This research enhances the understanding of LLM-generated content and establishes a foundation for more nuanced detection methodologies.

Collaborate SLM and LLM with latent answers for event detection

OntoED: Low-resource Event Detection with Ontology Embedding

Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models

One Small and One Large for Document-level Event Argument Extraction

Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs

Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection

Neural Cross-Lingual Event Detection with Minimal Parallel Resources.

Improving Large Language Models in Event Relation Logical Prediction

Exploiting Semantic Space to Enhance Event Detection Combined with Event Knowledge

Event Detection via Gated Multilingual Attention Mechanism.

Improving Low-Resource Chinese Event Detection with Multi-task Learning.

Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method

ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Refinement

Collaboration of Large and Small Models for Event Type Discovery in Completely Open Domains

Cada-Fvae-Gan: Adversarial Training for Few-Shot Event Detection

Semantic Fusion Enhanced Event Detection Via Multi-Graph Attention Network with Skip Connection

Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement

Adversarial Training for Weakly Supervised Event Detection

Are LLMs Good Annotators for Discourse-level Event Relation Extraction?

Document-Level Event Extraction with Definition-Driven ICL

DALD: Improving Logits-based Detector without Logits from Black-box LLMs