Abstract:Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.

Multi-objective Ensemble Generation

Generating Diverse and Accurate Classifier Ensembles Using Multi-Objective Optimization

Multi-objective Ensemble Construction , Learning and Evolution

Trade-Off Between Diversity and Accuracy in Ensemble Generation

Ensemble Learning through Diversity Management: Theory, Algorithms, and Applications

Testing of Diversity Strategy and Ensemble Strategy in SVM-Based Multiagent Ensemble Learning

An Effective Ensemble Framework for Multiobjective Optimization

Ensemble Mating Selection in Evolutionary Many-Objective Search.

Evolutionary Multi-Objective Generation of Recurrent Neural Network Ensembles for Time Series Prediction

Pairwise Diversity Measures Based Selective Ensemble Method

Evolutionary Multiobjective Ensemble Learning Based on Bayesian Feature Selection.

Mitigating Unfairness Via Evolutionary Multiobjective Ensemble Learning

Ensemble Learning Using Multi-Objective Evolutionary Algorithms

Multi-Aspect Diversification of Samples and Features for Advanced Creation of Rule Ensemble

Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems

On Diversity and Accuracy of Homogeneous and Heterogeneous Ensembles

Genetic Algorithm Based Selective Ensemble with Multiset Representation

Multi-information Ensemble Diversity

Ensemble of Selection Operators for Decomposition-Based Multi-Objective Evolutionary Optimization.

Maximizing Diversity by Transformed Ensemble Learning

Pareto-Based Multiobjective Machine Learning: an Overview and Case Studies