Abstract:The rapid development of large language models (LLMs), like ChatGPT, has resulted in the widespread presence of LLM-generated content on social media platforms, raising concerns about misinformation, data biases, and privacy violations, which can undermine trust in online discourse. While detecting LLM-generated content is crucial for mitigating these risks, current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-AI collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content. This approach introduces two novel tasks: LLM Role Recognition (LLM-RR), a multi-class classification task that identifies specific roles of LLM in content generation, and LLM Influence Measurement (LLM-IM), a regression task that quantifies the extent of LLM involvement in content creation. To support these tasks, we propose LLMDetect, a benchmark designed to evaluate detectors' performance on these new tasks. LLMDetect includes the Hybrid News Detection Corpus (HNDC) for training detectors, as well as DetectEval, a comprehensive evaluation suite that considers five distinct cross-context variations and multi-intensity variations within the same LLM role. This allows for a thorough assessment of detectors' generalization and robustness across diverse contexts. Our empirical validation of 10 baseline detection methods demonstrates that fine-tuned PLM-based models consistently outperform others on both tasks, while advanced LLMs face challenges in accurately detecting their own generated content. Our experimental results and analysis offer insights for developing more effective detection models for LLM-generated content. This research enhances the understanding of LLM-generated content and establishes a foundation for more nuanced detection methodologies.

Dispute Generation in Law Documents via Joint Context and Topic Attention.

Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement

De-Biased Court's View Generation with Causality

A Goal-Oriented Document-Grounded Dialogue Based on Evidence Generation

Partially-Aligned Data-to-Text Generation with Distant Supervision

Learning from Litigation: Graphs and LLMs for Retrieval and Reasoning in eDiscovery

Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions

Explicitly Integrating Judgment Prediction with Legal Document Retrieval: A Law-Guided Generative Approach

The Detection of Distributional Discrepancy for Text Generation

WSD-GAN: Word Sense Disambiguation Using Generative Adversarial Networks.

GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives

Discovering Significant Topics from Legal Decisions with Selective Inference

A Piece of My Mind: A Sentiment Analysis Approach for Online Dispute Detection

The Science of Detecting LLM-Generated Texts

AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation

Automatic Discovery of Controversial Legal Judgments by an Entropy-Based Measurement (S).

Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement

Automatic Text Revision with Application to Legal Documents

Judicial Intelligent Assistant System: Extracting Events from Divorce Cases to Detect Disputes for the Judge

SeqXGPT: Sentence-Level AI-Generated Text Detection

A Simple yet Efficient Ensemble Approach for AI-generated Text Detection