Quality Assessment in the Era of Large Models: A Survey

Zicheng Zhang,Yingjie Zhou,Chunyi Li,Baixuan Zhao,Xiaohong Liu,Guangtao Zhai

2024-08-17

Abstract:Quality assessment, which evaluates the visual quality level of multimedia experiences, has garnered significant attention from researchers and has evolved substantially through dedicated efforts. Before the advent of large models, quality assessment typically relied on small expert models tailored for specific tasks. While these smaller models are effective at handling their designated tasks and predicting quality levels, they often lack explainability and robustness. With the advancement of large models, which align more closely with human cognitive and perceptual processes, many researchers are now leveraging the prior knowledge embedded in these large models for quality assessment tasks. This emergence of quality assessment within the context of large models motivates us to provide a comprehensive review focusing on two key aspects: 1) the assessment of large models, and 2) the role of large models in assessment tasks. We begin by reflecting on the historical development of quality assessment. Subsequently, we move to detailed discussions of related works concerning quality assessment in the era of large models. Finally, we offer insights into the future progression and potential pathways for quality assessment in this new era. We hope this survey will enable a rapid understanding of the development of quality assessment in the era of large models and inspire further advancements in the field.

Human-Computer Interaction,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily explores the development and challenges of multimedia quality assessment in the era of large-scale models. Specifically, the paper attempts to address the following core issues:

1. **Evolution of Quality Assessment Methods**: The paper reviews quality assessment methods before the advent of large-scale models and compares them with the current methods in the era of large-scale models. Early quality assessments typically relied on small expert models tailored for specific tasks. While these models were effective in handling designated tasks, they lacked interpretability and robustness. In contrast, large-scale models can better simulate human cognitive and perceptual processes, thus being widely applied in quality assessment tasks.

2. **Evaluation of Large-Scale Models**: The paper discusses how to evaluate the performance of large-scale models themselves, including language models (LLMs) and multimodal models (LMMs). These models excel in handling various media content such as images, videos, and texts, but face numerous challenges in assessing the quality of their generated content, such as text alignment and specific distortions in generation.

3. **Multimodal Benchmarking**: To better evaluate the capabilities of large-scale multimodal models, researchers have developed a series of multimodal benchmark datasets. These benchmarks not only focus on the basic performance of the models but also examine their performance in complex tasks such as image understanding and question-answering systems. Through these benchmarks, a more comprehensive understanding of the models' strengths and weaknesses can be achieved.

4. **Quality Assessment of AI-Generated Content (AIGC)**: With the enhancement of large-scale models' generation capabilities, a significant amount of AI-generated content has emerged. The quality assessment of this content has become a new research hotspot. The paper explores how to develop innovative quality assessment methods to address these challenges.

In summary, this paper aims to provide guidance and insights for future research and development by comprehensively reviewing the field of quality assessment in the era of large-scale models.