Abstract:Commentary provides readers with a deep understanding of events by presenting diverse arguments and evidence. However, creating commentary is a time-consuming task, even for skilled commentators. Large language models (LLMs) have simplified the process of natural language generation, but their direct application in commentary creation still faces challenges due to unique task requirements. These requirements can be categorized into two levels: 1) fundamental requirements, which include creating well-structured and logically consistent narratives, and 2) advanced requirements, which involve generating quality arguments and providing convincing evidence. In this paper, we introduce Xinyu, an efficient LLM-based system designed to assist commentators in generating Chinese commentaries. To meet the fundamental requirements, we deconstruct the generation process into sequential steps, proposing targeted strategies and supervised fine-tuning (SFT) for each step. To address the advanced requirements, we present an argument ranking model for arguments and establish a comprehensive evidence database that includes up-to-date events and classic books, thereby strengthening the substantiation of the evidence with retrieval augmented generation (RAG) technology. To evaluate the generated commentaries more fairly, corresponding to the two-level requirements, we introduce a comprehensive evaluation metric that considers five distinct perspectives in commentary generation. Our experiments confirm the effectiveness of our proposed system. We also observe a significant increase in the efficiency of commentators in real-world scenarios, with the average time spent on creating a commentary dropping from 4 hours to 20 minutes. Importantly, such an increase in efficiency does not compromise the quality of the commentaries.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to address the inefficiencies and varying quality in the process of generating review articles. Specifically: 1. **Improving generation efficiency**: Traditional review article creation can take several hours, but by introducing the Xinyu system, this process can be shortened to less than 20 minutes. 2. **Ensuring content quality**: While improving efficiency, it ensures that the generated articles have a reasonable structure, consistent logic, high-quality arguments, and sufficient evidence support. 3. **Meeting specific task requirements**: It proposes a comprehensive solution to meet the unique requirements of review article generation, including basic requirements such as structural integrity and logical consistency, as well as advanced requirements such as argument novelty and evidence persuasiveness. Through these objectives, the study hopes to leverage the advantages of large language models (LLMs) to develop a system, Xinyu, that can assist reviewers in efficiently creating high-quality Chinese review articles.

Xinyu: An Efficient LLM-based System for Commentary Generation

Enhancing Commentary Strategies for Imperfect Information Card Games: A Study of Large Language Models in Guandan Commentary

Coherent Comments Generation for Chinese Articles with a Graph-to-Sequence Model.

Automating Just-In-Time Comment Updating

Multi-Intent Inline Code Comment Generation via Large Language Model

Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning

Integrating Extractive and Abstractive Models for Code Comment Generation

Guiding Developers to Make Informative Commenting Decisions in Source Code

MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment Generation

CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation

ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate

Cross-Modal Commentator: Automatic Machine Commenting Based on Cross-Modal Information.

Fine-grained LLM Agent: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

Code to Comment "Translation": Data, Metrics, Baselining & Evaluation

Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization

Code Needs Comments: Enhancing Code LLMs with Comment Augmentation

DeepCommenter: a Deep Code Comment Generation Tool with Hybrid Lexical and Syntactical Information

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation

Automatic Article Commenting: the Task and Dataset.

Enhancing MOBA Game Commentary Generation with Fine-Grained Prototype Retrieval