Abstract:Machine/deep learning models have been widely adopted for predicting the configuration performance of software systems. However, a crucial yet unaddressed challenge is how to cater for the sparsity inherited from the configuration landscape: the influence of configuration options (features) and the distribution of data samples are highly sparse. In this paper, we propose a model-agnostic and sparsity-robust framework for predicting configuration performance, dubbed DaL, based on the new paradigm of dividable learning that builds a model via "divide-and-learn". To handle sample sparsity, the samples from the configuration landscape are divided into distant divisions, for each of which we build a sparse local model, e.g., regularized Hierarchical Interaction Neural Network, to deal with the feature sparsity. A newly given configuration would then be assigned to the right model of division for the final prediction. Further, DaL adaptively determines the optimal number of divisions required for a system and sample size without any extra training or profiling. Experiment results from 12 real-world systems and five sets of training data reveal that, compared with the state-of-the-art approaches, DaL performs no worse than the best counterpart on 44 out of 60 cases with up to 1.61x improvement on accuracy; requires fewer samples to reach the same/better accuracy; and producing acceptable training overhead. In particular, the mechanism that adapted the parameter d can reach the optimal value for 76.43% of the individual runs. The result also confirms that the paradigm of dividable learning is more suitable than other similar paradigms such as ensemble learning for predicting configuration performance. Practically, DaL considerably improves different global models when using them as the underlying local models, which further strengthens its flexibility.

Efficient Identification of Approximate Best Configuration of Training in Large Datasets

ConEx: Efficient Exploration of Big-Data System Configurations for Better Performance

Using Bad Learners to find Good Configurations

Approximate Approach to Train SVM on Very Large Data Sets

DeepCAT+: A Low-Cost and Transferrable Online Configuration Auto-Tuning Approach for Big Data Frameworks

On Statistical Efficiency in Learning

Dividable Configuration Performance Learning

Learning to Optimize Computational Resources: Frugal Training with Generalization Guarantees

Enhancing Deep Stochastic Configuration Networks: Efficient Training via Low-Rank Matrix Approximation

Silhouette: Efficient Cloud Configuration Exploration for Large-Scale Analytics

Convergence-aware optimal checkpointing for exploratory deep learning training jobs

How to Shrink Confidence Sets for Many Equivalent Discrete Distributions?

Boosting Operational DNN Testing Efficiency Through Conditioning

An Adaptive Auto-configuration Tool for Hadoop

SmartTuning: Selecting Hyper-Parameters of a ConvNet System for Fast Training and Small Working Memory

An Efficient Instance Selection Algorithm to Reconstruct Training Set for Support Vector Machine

Learn to Optimize - A Brief Overview

Find Important Training Dataset by Observing the Training Sequence Similarity

Delving into Identify-Emphasize Paradigm for Combating Unknown Bias

A unified feature and instance selection framework using optimum experimental design.

Toward Better Accuracy-Efficiency Trade-Offs: Divide and Co-Training