Abstract:While the optimization algorithm community boasts numerous members, the primary wellspring of inspiration largely stems from collective and social behaviors observed in nature. This paper introduces a new metaphor-free meta-heuristic algorithm called Aitken optimizer (ATK) based on the basic idea of solving groups of equations using iterative methods. ATK is based on the Aitken sequence, which accelerates the immobile point iteration method to improve the algorithm's convergence speed and solution accuracy and reduce the possibility of falling into local optimums. Taking inspiration from the Aitken acceleration method, the ATK integrates two essential rules, namely the Aitken acceleration method search mechanism and the random weighted exponential operator, into its entire search process, which are instrumental in further enhancing the exploration of optimal solutions. ATK was validated using three CEC benchmarks (CEC2017, CEC2020, and CEC2022), and its results were compared with those of three categories of existing optimization algorithms, as follows: (1) the most cited optimizers, including the grey wolf optimizer, whale optimization algorithm, and Harris Hawks optimization, (2) published high-performance algorithms, including BWO, FHO, NRBO, SGA, SCA, CPSOGSA, and KOA, and (3) high-performance optimizers, such as SaDE, JaDE, CJADE, IMODE, EBOwithCMAR, SHADE, LSHADE, AL-SHADE, and LSHADE-cnEpSin. Statistical analysis shows that ATK achieves a winning rate of up to 94.23% across 51 functions in the three test sets compared to 10 competing optimizers. When benchmarked against 9 state-of-the-art high-performance optimizers, ATK's winning rates on the CEC 2017, CEC 2020, and CEC 2022 test suites are 27.59%, 25%, and 66.67%, respectively, with Friedman mean rankings of 2, 3, and 1. The results indicate that ATK, as a high-performance optimizer, demonstrates superior global search capabilities in high-dimensional, non-convex, and multimodal problems, outperforming simpler optimization methods suited for low-dimensional or convex problems. This makes ATK particularly well suited for applications such as engineering design and neural network tuning.

Auptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning

HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Services

AntTune: An Efficient Distributed Hyperparameter Optimization System for Large-Scale Data.

Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale

Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools

HomOpt: A Homotopy-Based Hyperparameter Optimization Method

Adaptive Optimizer for Automated Hyperparameter Optimization Problem

A System for Massively Parallel Hyperparameter Tuning

Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm

Two-step hyperparameter optimization method: Accelerating hyperparameter search by using a fraction of a training dataset

Code-Aware Cross-Program Transfer Hyperparameter Optimization.

Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimization

Tune: A Research Platform for Distributed Model Selection and Training

Automatic Hyper-Parameter Optimization Based on Mapping Discovery from Data to Hyper-Parameters

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

Aitken optimizer: an efficient optimization algorithm based on the Aitken acceleration method

Agent-based Collaborative Random Search for Hyper-parameter Tuning and Global Function Optimization

Hyperopt: a Python library for model selection and hyperparameter optimization

Boosting Aquila Optimizer by Marine Predators Algorithm for Combinatorial Optimization

MANGO: A Python Library for Parallel Hyperparameter Tuning

BTTackler: A Diagnosis-based Framework for Efficient Deep Learning Hyperparameter Optimization