Abstract:In the field of global optimization, finding the global optimum for complex problems remains a significant challenge. Traditional optimization methods often struggle to escape local minima and achieve global solutions, particularly when the initial solutions are far from the global optimum. This study addresses these challenges by introducing a novel algorithm called neural network transformation based global optimization. Our approach transforms original decision variables into higher-dimensional neural network parameters and constructs an empirical loss function using multiple sample points. By employing stochastic gradient descent for training, our approach effectively navigates the optimization landscape, escaping local minima and reaching low-loss solutions with high probability, even from distant starting points. We also propose a hybrid optimization method that combines the strength of metaheuristic strategies. The experimental results show that our hybrid method surpasses traditional global optimization algorithms, achieving an average 5% improvement in the success rate across benchmark functions. In practical applications, such as the B-spline curve approximation, our method reduces the fitting error by at least 10% compared with conventional approaches, delivering more accurate results. This study contributes a new gradient-based algorithm to the global optimization field, particularly effective for complex real-world problems where the initial points are far from the global minima.

Comparative Study of the CG and HBF ODEs Used in the Global Minimization of Nonconvex Functions

Unified Control Liapunov Function Based Design of Neural Networks That Aim at Global Minimization of Nonconvex Functions

The Controlled Conjugate Gradient Type Trajectory-Following Neural Net for Minimization of Nonconvex Functions.

Neural Network Solution for General Nonlinear Optimization Problems

A Neural Network Model For General Minimax Problem

Design of Second Order Neural Networks As Dynamical Control Systems That Aim to Minimize Nonconvex Scalar Functions

Control Liapunov function design of neural networks that solve convex optimization and variational inequality problems

Decentralized Conjugate Gradient and Memoryless BFGS Methods

Global Versus Local Orthogonal Collocation in Simultaneous Approach

Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian

Generalized Levenberg-Marquardt Neural Nets for Minimization of Quasiconvex Scalar Functions

Proposed CG method to solve unconstrained optimization problems

A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

Modified parameter of Dai Liao conjugacy condition of the conjugate gradient method

A HYBRID CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION WITH APPLICATION

An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations

Subgradient-based Feedback Neural Networks for Non-Differentiable Convex Optimization Problems

A Neural Network Transformation based Global Optimization Algorithm

Neural Control of Discrete Weak Formulations: Galerkin, Least-Squares and Minimal-Residual Methods with Quasi-Optimal Weights

Universal heavy-ball method for nonconvex optimization under Hölder continuous Hessians