Abstract:Fractals, Ahead of Print. Metaheuristic techniques are capable of representing optimization frames with their specific theories as well as objective functions owing to their being adjustable and effective in various applications. Through the optimization of deep learning models, metaheuristic algorithms inspired by nature, imitating the behavior of living and non-living beings, have been used for about four decades to solve challenging, complex, and chaotic problems. These algorithms can be categorized as evolution-based, swarm-based, nature-based, human-based, hybrid, or chaos-based. Chaos theory, as a useful approach to understanding neural network optimization, has the basic idea of viewing the neural network optimization as a dynamical system in which the equation schemes are utilized from the space pertaining to learnable parameters, namely optimization trajectory, to itself, which enables the description of the evolution of the system by understanding the training behavior, which is to say the number of iterations over time. The examination of the recent studies reveals the importance of chaos theory, which is sensitive to initial conditions with randomness and dynamical properties that are principally emerging on the complex multimodal landscape. Chaotic optimization, in this regard, accelerates the speed of the algorithm while also enhancing the variety of movement patterns. The significance of hybrid algorithms developed through their applications in different domains concerning real-world phenomena and well-known benchmark problems in the literature is also evident. Metaheuristic optimization algorithms have also been applied to deep learning or deep neural networks (DNNs), a branch of machine learning. In this respect, the basic features of deep learning and DNNs and the extensive use of metaheuristic algorithms are overviewed and explained. Accordingly, the current review aims at providing new insights into the studies that deal with metaheuristic algorithms, hybrid-based metaheuristics, chaos-based metaheuristics as well as deep learning besides presenting recent information on the development of the essence of this branch of science with emerging opportunities, applicability-based optimization aspects and generation of well-informed decisions.

Mengejar Kinerja Maksimal: Teknik Pengoptimalan Terkini dalam Pembelajaran Mesin

Machine Learning Optimization Techniques: A Survey, Classification, Challenges, and Future Research Issues

On hyperparameter optimization of machine learning algorithms: Theory and practice

Learn to Optimize - A Brief Overview

An efficient optimization approach for designing machine learning models based on genetic algorithm

An Efficient Optimization Technique for Training Deep Neural Networks

Chaos Theory, Advanced Metaheuristic Algorithms And Their Newfangled Deep Learning Architecture Optimization Applications: A Review

Discrete Simulation Optimization for Tuning Machine Learning Method Hyperparameters

A Survey of Optimization Methods From a Machine Learning Perspective

Practical Bayesian Optimization of Machine Learning Algorithms

MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters

Hyperopt: a Python library for model selection and hyperparameter optimization

Application of AI in Optimizing Energy and Resource Management: Effectiveness of Deep Learning Models

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

Elevating fraud detection: machine learning models with computational intelligence optimization

Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization

Hyper-Parameter Optimization: A Review of Algorithms and Applications

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

Hyperparameter Tuning of Deep learning Models in Keras

Enhancing Support Vector Machine Performance: A Hybrid Approach with Davidon-Fletcher-Powell Algorithm and Elephant Herding Optimization (EHO-DFP) for Parameter Optimization