Abstract:Semantic communication, as a revolutionary communication architecture, is considered a promising novel communication paradigm. Unlike traditional symbol-based error-free communication systems, semantic-based visual communication systems extract, compress, transmit, and reconstruct images at the semantic level. However, widely used image similarity evaluation metrics, whether pixel-based MSE or PSNR or structure-based MS-SSIM, struggle to accurately measure the loss of semantic-level information of the source during system transmission. This presents challenges in evaluating the performance of visual semantic communication systems, especially when comparing them with traditional communication systems. To address this, we propose a semantic evaluation metric -- SeSS (Semantic Similarity Score), based on Scene Graph Generation and graph matching, which shifts the similarity scores between images into semantic-level graph matching scores. Meanwhile, semantic similarity scores for tens of thousands of image pairs are manually annotated to fine-tune the hyperparameters in the graph matching algorithm, aligning the metric more closely with human semantic perception. The performance of the SeSS is tested on different datasets, including (1)images transmitted by traditional and semantic communication systems at different compression rates, (2)images transmitted by traditional and semantic communication systems at different signal-to-noise ratios, (3)images generated by large-scale model with different noise levels introduced, and (4)cases of images subjected to certain special transformations. The experiments demonstrate the effectiveness of SeSS, indicating that the metric can measure the semantic-level differences in semantic-level information of images and can be used for evaluation in visual semantic communication systems.

An effective algorithm for semantic similarity metric of word pairs

A Novel Comprehensive Approach for Estimating Concept Semantic Similarity in WordNet

A New Hypred Improved Method for Measuring Concept Semantic Similarity in WordNet.

A Hybrid Semantic Similarity Measurement for Geospatial Entities

Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy

An improved method for semantic similarity calculation based on stop-words

Research on Chinese Semantic Similarity Algorithm

An Approach to Measuring Semantic Similarity and Relatedness Between Concepts in an Ontology.

Research on the Semantic Measurement in Co-word Analysis.

Semantic Similarity Score for Measuring Visual Similarity at Semantic Level

A Combined Measure for Text Semantic Similarity

Word Similarity Computation with Extreme-Similar Method.

A Large Probabilistic Semantic Network Based Approach to Compute Term Similarity

Short Text Similarity Calculation Using Semantic Information

Semantic Similarity Calculation Method of Comprehensive Concept in WordNet

Semantic Similarity Calculation Based on Sememe Set

Study on quantifying the semantic correlative relation of Chinese word-pair

Evaluation of taxonomic and neural embedding methods for calculating semantic similarity

Semantic Word-formation Based Chinese Word Similarity Computing

An Approach of Semantic Similarity by Combining HowNet and Cilin.

Computing Semantic Text Similarity Using Rich Features.