Abstract:Hyperparameter tuning is a critical function necessary for the effective deployment of most machine learning (ML) algorithms. It is used to find the optimal hyperparameter settings of an ML algorithm in order to improve its overall output performance. To this effect, several optimization strategies have been studied for fine-tuning the hyperparameters of many ML algorithms, especially in the absence of model-specific information. However, because most ML training procedures need a significant amount of computational time and memory, it is frequently necessary to build an optimization technique that converges within a small number of fitness evaluations. As a result, a simple deterministic selection genetic algorithm (SDSGA) is proposed in this article. The SDSGA was realized by ensuring that both chromosomes and their accompanying fitness values in the original genetic algorithm are selected in an elitist-like way. We assessed the SDSGA over a variety of mathematical test functions. It was then used to optimize the hyperparameters of two well-known machine learning models, namely, the convolutional neural network (CNN) and the random forest (RF) algorithm, with application on the MNIST and UCI classification datasets. The SDSGA’s efficiency was compared to that of the Bayesian Optimization (BO) and three other popular metaheuristic optimization algorithms (MOAs), namely, the genetic algorithm (GA), particle swarm optimization (PSO) and biogeography-based optimization (BBO) algorithms. The results obtained reveal that the SDSGA performed better than the other MOAs in solving 11 of the 17 known benchmark functions considered in our study. While optimizing the hyperparameters of the two ML models, it performed marginally better in terms of accuracy than the other methods while taking less time to compute.

Superior Genetic Algorithms for the Target Set Selection Problem Based on Power-Law Parameter Choices and Simple Greedy Heuristics

Component Thermodynamical Selection Based Gene Expression Programming for Function Finding

A Census-Based Genetic Algorithm for Target Set Selection Problem in Social Networks

Optimization and Improvement of Genetic Algorithms Solving Traveling Salesman Problem

Maximizing influence in a social network: Improved results using a genetic algorithm

Rethinking Selection in Generational Genetic Algorithms to Solve Combinatorial Optimization Problems: An Upper Bound-based Parent Selection Strategy for Recombination

Balancing Pareto Front exploration of Non-dominated Tournament Genetic Algorithm (B-NTGA) in solving multi-objective NP-hard problems with constraints

Solving Distribution Logistics Intelligent Planning Problem By 2-Tier Genetic Algorithm

MCTS guided Genetic Algorithm for optimization of neural network weights

A Spark-based Gaussian Bare-bones Cuckoo Search with Dynamic Parameter Selection

Gene Targeting Particle Swarm Optimization for Large-Scale Optimization Problem

A Novel Selection Approach for Genetic Algorithms for Global Optimization of Multimodal Continuous Functions

Multi-objective Peer-to-Peer Neighbor-Selection Strategy Using Genetic Algorithm

Comparison Of Binary Coded Genetic Algorithms With Different Selection Strategies For Continuous Optimization Problems

Optimization of Adaptive Genetic Algorithm Parameters in Traveling Salesman Problem

NeuroCrossover: an Intelligent Genetic Locus Selection Scheme for Genetic Algorithm Using Reinforcement Learning

Biased Random-Key Genetic Algorithms: A Review

Simple Deterministic Selection-Based Genetic Algorithm for Hyperparameter Tuning of Machine Learning Models

A Powerful Genetic Algorithm for Traveling Salesman Problem

Solving TSP problem with improved genetic algorithm