METTLE: A Metamorphic Testing Approach To Validating Unsupervised Machine Learning Methods

Xiaoyuan Xie,Zhiyi Zhang,Tsong Yueh Chen,Yang Liu,Pak-Lok Poon,Baowen Xu

2018-01-01

Abstract:Unsupervised (machine) learning is the training of an artificial intelligence (AI) algorithm using information that is neither classified nor labeled, with a view to modeling the underlying structure or distribution in a dataset. Since unsupervised learning algorithms are widely used in many real-world applications, validating the implementations of these algorithms is indisputably an important task in software quality assurance. This validation task, however, is fairly challenging due to the absence of a priori knowledge of the data. In view of this challenge, we develop a $textbf{MET}$amorphic $textbf{T}$esting approach to validating unsupervised machine $textbf{LE}$arning methods, abbreviated as METTLE. This validation approach takes consideration of the specific expectations and requirements on the learning methods by individual users. To demonstrate the viability and effectiveness of MTTLE, we describe an experiment involving six common clustering algorithms. Furthermore, we discuss how METTLE can be used by the users to devise their own and specific quality assessment schemes for selecting the suitable unsupervised learning algorithms to use in different application scenarios.