Abstract:PDF HTML XML Export Cite reminder Deep Learning Test Optimization Method Using Multi-objective Optimization DOI: 10.21655/ijsi.1673-7288.00282 Author: Affiliation: Clc Number: Fund Project: Article | Figures | Metrics | Reference | Related | Cited by | Materials | Comments Abstract:With the rapid development of deep learning technology, research on its quality assurance is raising more attention. Meanwhile, it is no longer difficult to collect test data owing to the mature sensor technology, but it costs a lot to label the collected data. To reduce the cost of labeling, the existing studies attempt to select a test subset from the original test set. The test subset, however, only ensures that the overall accuracy (the accuracy of the target deep learning model on all test inputs of the test set) of the test subset is similar to that of the original test set; it cannot maintain other test properties similar to those of the original test set. For example, it cannot fully cover all kinds of test input in the original test set. This study proposes a method based on multi-objective optimization called Deep Multi-Objective Selection (DMOS). It firstly analyzes the data distribution of the original test set by Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN). Then, it designs multiple optimization objectives given the characteristics of the clustering results and then carries out multi-objective optimization to find out the appropriate selection solution. Massive experiments are carried out on eight pairs of classic deep learning test sets and models. The results reveal that the best test subset selected by the DMOS method (the test subset corresponding to the Pareto optimal solution with the best performance) can not only cover more test input categories in the original test set but also estimate the accuracy of each test input category extremely close to that of the original test set. Meanwhile, it can also ensure that the overall accuracy and test adequacy are close to those of the original test set: the average error of the overall accuracy estimation is only 1.081%, which is 0.845% lower than that of Practical ACcuracy Estimation (PACE), an improvement of 43.87%. The average error of the accuracy estimation of each test input category is only 5.547%, which is 2.926% less than that of PACE, an improvement of 34.53%. The average estimation error of the five test adequacy measures is only 8.739%, which is 7.328% lower than that of PACE, an improvement of 45.61%. Reference Related Cited by

SMART OPTIMIZER SELECTION TECHNIQUE: A COMPARATIVE STUDY OF MODIFIED DENSNET201 WITH OTHER DEEP LEARNING MODELS

An Efficient Optimization Technique for Training Deep Neural Networks

Performance Comparison of Multiple ANN Optimizer on IoT-enabled Sensor Fire Dataset

Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks

A comparative study of recently deep learning optimizers

Optimization of deep learning models: benchmark and analysis

A methodological framework for optimizing the energy consumption of deep neural networks: a case study of a cyber threat detector

Impact of optimizers functions on detection of Melanoma using transfer learning architectures

Deep Learning Test Optimization Method Using Multi-objective Optimization.

Comparison of Optimizers for Drone Signal Detection Using Convolutional Neural Networks (CNN)

Automatic Polyp Semantic Segmentation Using Wireless Capsule Endoscopy Images With Various Convolutional Neural Network And Optimization Techniques: A Comparison And Performance Evaluation

Effectiveness of Optimization Algorithms in Deep Image Classification

Assessment of Optimizers impact on Image Recognition with Convolutional Neural Network to Adversarial Datasets

A Comparison of Optimization Algorithms for Deep Learning

Optimizing Deep Learning Model for Software Cost Estimation Using Hybrid Meta-Heuristic Algorithmic Approach

Optimizing neural networks using spider monkey optimization algorithm for intrusion detection system

A modified Adam algorithm for deep neural network optimization

Novel Crow Swarm Optimization Algorithm and Selection Approach for Optimal Deep Learning COVID-19 Diagnostic Model

Drug-based recommendation system based on deep learning approach for data optimization

Attack classification in network intrusion detection system based on optimization strategy and deep learning methodology

Design of robust deep learning-based object detection and classification model for autonomous driving applications