Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Changli Tang,Yixuan Li,Yudong Yang,Jimin Zhuang,Guangzhi Sun,Wei Li,Zujun Ma,Chao Zhang

2024-10-11

Abstract:Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual large language model (LLM) with low-rank adaptation (LoRA) designed for enhanced video (with paired audio) captioning through directed preference optimization (DPO). We propose new metrics to evaluate the completeness and accuracy of video descriptions, which are optimized using DPO. To further improve training, we introduce a novel multi-round DPO (mrDPO) approach, which involves periodically updating the DPO reference model, merging and re-initializing the LoRA module as a proxy for parameter updates after each training round (1,000 steps), and incorporating guidance from ground-truth video captions to stabilize the process. To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels. Experiments show that mrDPO significantly enhances video-SALMONN 2's captioning accuracy, reducing global and local error rates by 40\% and 20\%, respectively, while decreasing the repetition rate by 35\%. The final video-SALMONN 2 model, with just 7 billion parameters, surpasses leading models such as GPT-4o and Gemini-1.5-Pro in video captioning tasks, while maintaining competitive performance to the state-of-the-art on widely used video question-answering benchmark among models of similar size. Upon acceptance, we will release the code, model checkpoints, and training and test data. Demos are available at \href{<a class="link-external link-https" href="https://video-salmonn-2.github.io" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://video-salmonn-2.github.io" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition,Computation and Language,Image and Video Processing

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address several key issues in the task of video description (video captioning):

1. **Detail and Accuracy**: Existing Multimodal Large Language Models (Multimodal LLMs) often lack detail and accuracy when generating video descriptions. Video content is rich, containing not only visual information but also audio information, making it challenging to generate high-quality video descriptions.

2. **Lack of Evaluation Metrics**: There is currently a lack of effective quantitative metrics to evaluate the quality of video descriptions, making it difficult to optimize the models.

3. **Insufficient Training Methods**: Existing training methods, while improving the completeness of video descriptions, tend to cause issues such as hallucinations (generating unrealistic content) and repetition in the models.

4. **Lack of Audio Understanding**: Most current visual large language models lack the ability to understand audio information, leading to the omission of important audio information in the generated descriptions.

To address these issues, the paper proposes **video-SALMONN 2**, an enhanced multimodal large language model that improves the detail and accuracy of video descriptions through multi-round preference optimization (mrDPO) and Low-Rank Adaptation (LoRA) techniques. Additionally, the paper introduces new evaluation metrics and training strategies to ensure the model maintains performance in other tasks while generating high-quality video descriptions.