HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment

Armin Shafiee Sarvestani,Sheyang Tang,Zhou Wang

2024-12-03

Abstract:Mesh quality assessment (MQA) models play a critical role in the design, optimization, and evaluation of mesh operation systems in a wide variety of applications. Current MQA models, whether model-based methods using topology-aware features or projection-based approaches working on rendered 2D projections, often fail to capture the intricate interactions between texture and 3D geometry. We introduce HybridMQA, a first-of-its-kind hybrid full-reference colored MQA framework that integrates model-based and projection-based approaches, capturing complex interactions between textural information and 3D structures for enriched quality representations. Our method employs graph learning to extract detailed 3D representations, which are then projected to 2D using a novel feature rendering process that precisely aligns them with colored projections. This enables the exploration of geometry-texture interactions via cross-attention, producing comprehensive mesh quality representations. Extensive experiments demonstrate HybridMQA's superior performance across diverse datasets, highlighting its ability to effectively leverage geometry-texture interactions for a thorough understanding of mesh quality. Our implementation will be made publicly available.

Computer Vision and Pattern Recognition,Multimedia

What problem does this paper attempt to address?

### Problems the paper attempts to solve

This paper aims to solve the problem that existing three - dimensional mesh quality assessment (Mesh Quality Assessment, MQA) methods are unable to effectively capture the complex interactions between texture and geometric structure. Specifically:

1. **Limitations of existing MQA methods**:
- **Model - based methods**: These methods directly extract topology - aware quality features in 3D vertex space, but their performance is limited due to the lack of access to the actual appearance (rendered projection) of the object.
- **Projection - based methods**: These methods evaluate quality by processing rendered 2D projections. Although they are more effective in evaluating texture distortion, they lack an understanding of the 3D topology of the object, resulting in poor performance in detecting geometric distortion.

2. **Importance of geometric - texture interaction**:
- Figure 1 shows examples of this geometric - texture interaction, where changes in geometry or texture can affect each other's visual distortion effects. For example, a complex texture can make geometric distortion less obvious, and modifying the geometric structure can change the appearance of texture distortion.
- This complex interaction is an important factor often overlooked by existing methods, so an MQA method that can capture these interactions is required.

### Proposed solutions

To solve the above problems, the paper proposes HybridMQA, a hybrid full - reference (FR) colored mesh quality assessment framework. The main features of HybridMQA include:

- **Integrating model and projection methods**: Combining the advantages of both model - based and projection - based methods, detailed 3D representations are extracted through graph learning, and these representations are aligned with color projections through a novel feature projection technique, thereby comprehensively exploring geometric - texture interactions.
- **Application of graph convolutional network (GCN)**: Using GCN to learn detailed surface representations from vertices in 3D space and establish a 3D understanding of the object's topology.
- **Feature projection technique**: A new feature projection technique is proposed to render 3D surface representations into 2D projections, ensuring alignment with the color projection pixels of the mesh, thereby exploring geometric - texture interactions.
- **Cross - attention mechanism**: Capturing the interactions between geometric and texture representations at different scales through a cross - attention module, further improving the accuracy of quality assessment.

### Experimental results

The experimental results show that HybridMQA significantly outperforms existing model - based and projection - based MQA methods on multiple public datasets, especially when dealing with mixed - type geometric and texture distortion. In addition, HybridMQA also demonstrates good generalization ability and can maintain high performance on unseen datasets.

### Summary

By integrating the advantages of model - based and projection - based methods and introducing novel feature projection techniques and cross - attention mechanisms, HybridMQA successfully solves the problem that existing MQA methods are unable to effectively capture geometric - texture interactions, providing a new solution for high - quality three - dimensional mesh assessment.