MoE-AGIQA: Mixture-of-Experts Boosted Visual Perception-Driven and Semantic-Aware Quality Assessment for AI-Generated Images

Wei Zhang,Han Peng,Junfeng Yang,Wenzhi Cao,Jing Fu,Limei Liu

DOI: https://doi.org/10.1109/CVPRW63382.2024.00641

2024-06-17

Abstract:Recently, there has been a surge of interest in AI-Generated Image Quality Assessment (AGIQA). Unlike images in common image quality assessment tasks, AI-generated images may suffer from some unique degradations. To this end, we propose a novel mixture-of-experts boosted visual perception-driven and semantic-aware quality assessment for AI-generated images (MoE-AGIQA). Firstly, we design a visual degradation-aware network to ascertain perceptual rules by emulating human perception of visual degradation. To enhance the diversity of visual degradation-aware features, we additionally devise a prior knowledge injection module, which is pre-trained on specific natural images. Secondly, we devise a semantic-aware network to assess the inconsistency between input text prompts and AI-generated images, and further detect potential semantic problems. Thirdly, we propose to conduct cross-attention on visual degradation-aware and semantic-aware features, so that we can obtain comprehensive quality-aware features and the inherent correlation between these features. Finally, we propose a mixture-of-experts module, involving multiple experts working collaboratively. Each expert is responsible for a specific set of features and outputs a corresponding prediction score. The mixture of multiple experts will ultimately yield a holistic, perceptual quality score. Experimental results on benchmark AGIQA datasets and the NTIRE 2024 Quality Assessment for AI-Generated Content - Track 1 Image Challenge demonstrate our superior performance. The source code is available at https://github.com/37s/MoE-AGIQA.

Computer Science