Abstract:Unsupervised machine learning is the training of an artificial intelligence system using information that is neither classified nor labeled, with a view to modeling the underlying structure or distribution in a dataset. Since unsupervised machine learning systems are widely used in many real-world applications, assessing the appropriateness of these systems and validating their implementations with respect to individual users' requirements and specific application scenarios/contexts are indisputably two important tasks. Such assessments and validation tasks, however, are fairly challenging due to the absence of a priori knowledge of the data. In view of this challenge, in this article, we develop a MET amorphic T esting approach to assessing and validating unsupervised machine LE arning systems, abbreviated as mettle. Our approach provides a new way to unveil the (possibly latent) characteristics of various machine learning systems, by explicitly considering the specific expectations and requirements of these systems from individual users' perspectives. To support mettle, we have further formulated 11 generic metamorphic relations (MRs), covering users' generally expected characteristics that should be possessed by machine learning systems. We have performed an experiment and a user evaluation study to evaluate the viability and effectiveness of mettle. Our experiment and user evaluation study have shown that, guided by user-defined MR-based adequacy criteria, end users are able to assess, validate, and select appropriate clustering systems in accordance with their own specific needs. Our investigation has also yielded insightful understanding and interpretation of the behavior of the machine learning systems from an end-user software engineering's perspective, rather than a designer's or implementor's perspective, who normally adopts a theoretical appr-ach.

Supervised Ontology and Instance Matching with MELT

Matching with Transformers in MELT

Ontology Matching Based On Probabilistic Description Logic

Interactive Ontology Matching with Cost-Efficient Learning

LLMs4OM: Matching Ontologies with Large Language Models

Actively Learning Ontology Matching via User Interaction

Contextualized Structural Self-supervised Learning for Ontology Matching

OLaLa: Ontology Matching with Large Language Models

METTLE: A METamorphic Testing Approach to Assessing and Validating Unsupervised Machine Learning Systems

Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching

HIERMATCH: Leveraging Label Hierarchies for Improving Semi-Supervised Learning

HybridVocab: Towards Multi-Modal Machine Translation Via Multi-Aspect Alignment

METTLE: A Metamorphic Testing Approach To Validating Unsupervised Machine Learning Methods

MEDTO: Medical Data to Ontology Matching Using Hybrid Graph Neural Networks

Matchmaker: Self-Improving Large Language Model Programs for Schema Matching

Large Scale Instance Matching Via Multiple Indexes and Candidate Selection

MLTEing Models: Negotiating, Evaluating, and Documenting Model and System Qualities

SMUTF: Schema Matching Using Generative Tags and Hybrid Features

Augmenting Ontology Alignment by Semantic Embedding and Distant Supervision

Leveraging Structural Information in Ontology Matching

UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models