Genetic algorithms for the numerical solution of variational problems without analytic trial functions

Carlos D. Toledo

DOI: https://doi.org/10.48550/arXiv.physics/0506188

2005-06-25

Abstract:A coding of functions that allows a genetic algorithm to minimize functionals without analytic trial functions is presented and implemented for solving numerically some instances of variational problems from physics.

Computational Physics

What problem does this paper attempt to address?

The Use of Genetic Algorithms for Optimizing the Regularized Solutions of the Ill-Posed Problems

Mingfeng Jiang,Wenqing Huang,Ling Xia,Guofa Shou

DOI: https://doi.org/10.1109/IITA.2008.214

2008-01-01

Abstract:Numerical solution of ill-posed problems is often accomplished by regularization method, such as Tikhonove method, truncated singular-value decomposition (TSVD). Due to large noises condition, those conventional regularization methods cannot provide available solutions for ill-posed problem. Considering that genetic algorithms (GA) is a stochastic optimization technique which may be useful for optimizing the regularized solutions. Computing the epicardial potentials from the body surface potentials constitutes one form of the ill-posed inverse problems of electrocardiography (ECG), which is considered as an example to illustrate the performance of GA when applied to optimize the regularized solutions. The result suggests that the GA may be a good scheme for optimizing the regularized solutions in solving the inverse ECG problem.
Genetic Algorithm-Accelerated Optimization for Runge-Kutta Scheme Derivation in High-Dimensional Problems With Hard Constraints

Gavin-Lee Goodship,Stephen O'Sullivan,Luis Miralles-Pechuán

DOI: https://doi.org/10.1109/access.2024.3460484

IF: 3.9

2024-09-27

IEEE Access

Abstract:The numerical derivation of high-order Extended Stability Runge-Kutta (ERSK) integration schemes presents significant challenges due to the high-dimensional, non-convex, non-linear optimization problems with hard equality constraints. Traditional optimization techniques often struggle to converge on valid solutions efficiently. This paper proposes a hybrid approach that integrates Genetic Algorithm (GA) techniques with an interior-point optimizer to address these challenges. GA-generated values are used as starting points for the interior-point method, which refines candidate solutions to meet the stringent order conditions. Experimental results demonstrate a significant improvement in efficiency when GA-generated values are used. The average number of iterations required for convergence is reduced from 477 (with random initialization) to 143 when GA values are used. The median number of iterations drops from 280 to 105. The standard deviation is also reduced from 361 to 112, indicating more consistent performance. These reductions highlight the efficiency of the GA-plus-polish approach, which outperforms traditional random initialization in terms of convergence time and consistency. Despite some remaining variance, this hybrid method significantly reduces the computational cost of deriving stable Butcher tableaux, particularly in high-dimensional, non-convex optimization problems.

computer science, information systems,telecommunications,engineering, electrical & electronic
Variations of Genetic Algorithms

Alison Jenkins,Vinika Gupta,Alexis Myrick,Mary Lenoir

DOI: https://doi.org/10.48550/arXiv.1911.00490

2019-11-02

Abstract:The goal of this project is to develop the Genetic Algorithms (GA) for solving the Schaffer F6 function in fewer than 4000 function evaluations on a total of 30 runs. Four types of Genetic Algorithms (GA) are presented - Generational GA (GGA), Steady-State (mu+1)-GA (SSGA), Steady-Generational (mu,mu)-GA (SGGA), and (mu+mu)-GA.

Neural and Evolutionary Computing,Machine Learning
Easily implementable iterative methods for variational inequalities with nonlinear diffusion–convection operator and constraints to the gradient of solution

Erkki Laitinen,Alexander Lapin,Sergey Lapin

DOI: https://doi.org/10.1515/rnam-2015-0005

2015-01-01

Russian Journal of Numerical Analysis and Mathematical Modelling

Abstract:Abstract New iterative solution methods are proposed for the finite element approximation of a class of variational inequalities with nonlinear diffusion-convection operator and constraints to the gradient of solution. Implementation of every iteration of these methods reduces to the solution of a system of linear equations and a set of two-dimensional minimization problems. Convergence is proved by the application of a general result on the convergence of the iterative methods for a nonlinear constrained saddle point problem.

mathematics, applied
Kinetic description and convergence analysis of genetic algorithms for global optimization

Giacomo Borghi,Lorenzo Pareschi

2023-10-13

Abstract:Genetic Algorithms (GA) are a class of metaheuristic global optimization methods inspired by the process of natural selection among individuals in a population. Despite their widespread use, a comprehensive theoretical analysis of these methods remains challenging due to the complexity of the heuristic mechanisms involved. In this work, relying on the tools of statistical physics, we take a first step towards a mathematical understanding of GA by showing how their behavior for a large number of individuals can be approximated through a time-discrete kinetic model. This allows us to prove the convergence of the algorithm towards a global minimum under mild assumptions on the objective function for a popular choice of selection mechanism. Furthermore, we derive a time-continuous model of GA, represented by a Boltzmann-like partial differential equation, and establish relations with other kinetic and mean-field dynamics in optimization. Numerical experiments support the validity of the proposed kinetic approximation and investigate the asymptotic configurations of the GA particle system for different selection mechanisms and benchmark problems.

Optimization and Control
Genetic optimization algorithms applied toward mission computability models

Mee Seong Im,Venkat R. Dasari

DOI: https://doi.org/10.48550/arXiv.2005.13105

2020-05-27

Neural and Evolutionary Computing

Abstract:Genetic algorithms are modeled after the biological evolutionary processes that use natural selection to select the best species to survive. They are heuristics based and low cost to compute. Genetic algorithms use selection, crossover, and mutation to obtain a feasible solution to computational problems. In this paper, we describe our genetic optimization algorithms to a mission-critical and constraints-aware computation problem.
Solution to Quadratic Equation Using Genetic Algorithm

Tanistha Nayak,Tirtharaj Dash

DOI: https://doi.org/10.48550/arXiv.1306.4622

2013-06-20

Abstract:Solving Quadratic equation is one of the intrinsic interests as it is the simplest nonlinear equations. A novel approach for solving Quadratic Equation based on Genetic Algorithms (GAs) is presented. Genetic Algorithms (GAs) are a technique to solve problems which need optimization. Generation of trial solutions have been formed by this method. Many examples have been worked out, and in most cases we find out the exact solution. We have discussed the effect of different parameters on the performance of the developed algorithm. The results are concluded after rigorous testing on different equations.

Neural and Evolutionary Computing
Numerical approximation of variational problems with orthotropic growth

Anna Kh. Balci,Lars Diening,Abner J. Salgado

DOI: https://doi.org/10.1007/s00211-024-01432-9

IF: 2.5

2024-09-26

Numerische Mathematik

Abstract:We consider the numerical approximation of variational problems with orthotropic growth, that is those where the integrand depends strongly on the coordinate directions with possibly different growth in each direction. Under realistic regularity assumptions we derive optimal error estimates. These estimates depend on the existence of an orthotropically stable interpolation operator. Over certain meshes we construct an orthotropically stable interpolant that is also a projection. Numerical experiments illustrate and explore the limits of our theory.

mathematics, applied
Optimizing Variational Physics-Informed Neural Networks Using Least Squares

Carlos Uriarte,Manuela Bastidas,David Pardo,Jamie M. Taylor,Sergio Rojas

2024-08-30

Abstract:Variational Physics-Informed Neural Networks often suffer from poor convergence when using stochastic gradient-descent-based optimizers. By introducing a Least Squares solver for the weights of the last layer of the neural network, we improve the convergence of the loss during training in most practical scenarios. This work analyzes the computational cost of the resulting hybrid Least-Squares/Gradient-Descent optimizer and explains how to implement it efficiently. In particular, we show that a traditional implementation based on backward-mode automatic differentiation leads to a prohibitively expensive algorithm. To remedy this, we propose using either forward-mode automatic differentiation or an ultraweak-type scheme that avoids the differentiation of trial functions in the discrete weak formulation. The proposed alternatives are up to one hundred times faster than the traditional one, recovering a computational cost-per-iteration similar to that of a conventional gradient-descent-based optimizer alone. To support our analysis, we derive computational estimates and conduct numerical experiments in one- and two-dimensional problems.

Numerical Analysis
Parallel iterative methods for variational integration applied to navigation problems

Sebastián J. Ferraro,David Martín de Diego,Rodrigo T. Sato Martín de Almagro

DOI: https://doi.org/10.48550/arXiv.2109.05559

2021-09-13

Abstract:Discrete variational methods have shown an excellent performance in numerical simulations of different mechanical systems. In this paper, we introduce an iterative method for discrete variational methods appropriate for boundary value problems. More concretely, we explore a parallelization strategy that leverages the power of multicore CPUs and GPUs (graphics cards). We study this parallel method for first-order and second-order Lagrangians and we illustrate its excellent behavior in some interesting applications, namely Zermelo's navigation problem, a fuel-optimal navigation problem, and an interpolation problem.

Numerical Analysis,Optimization and Control
Algorithm for Variational Inequality Problems Based on a Gradient Dynamical System Designed Using a Control Liapunov Function

Fernando Agustin Pazos,Amit Bhaya

DOI: https://doi.org/10.1109/isic.2007.4450953

2007-01-01

Abstract:We present an algorithm to find the optimal point of a variational inequality problem. The domain of the function that defines the variational inequality is a convex set, determined by convex inequality constraints and affine equality constraints. The algorithm is based on a discrete variable structure closed-loop control system which presents sliding mode trajectories on the boundary of the feasible set until the optimal point is reached. The update law is designed using control Liapunov function (CLF), which guarantees the decrease of a discrete Liapunov function inside and outside the feasible set. The step size is optimized using Liapunov optimizing control (LOC).
Automating Variational Differentiation

Kangbo Li,Anil Damle

2024-06-24

Abstract:Many problems in Physics and Chemistry are formulated as the minimization of a functional. Therefore, methods for solving these problems typically require differentiating maps whose input and/or output are functions -- commonly referred to as variational differentiation. Such maps are not addressed at the mathematical level by the chain rule, which underlies modern symbolic and algorithmic differentiation (AD) systems. Although there are algorithmic solutions such as tracing and reverse accumulation, they do not provide human readability and introduce strict programming constraints that bottleneck performance, especially in high-performance computing (HPC) environments. In this manuscript, we propose a new computer theoretic model of differentiation by combining the pullback of the $\mathbf{B}$ and $\mathbf{C}$ combinators from the combinatory logic. Unlike frameworks based on the chain rule, this model differentiates a minimal complete basis for the space of computable functions. Consequently, the model is capable of analytic backpropagation and variational differentiation while supporting complex numbers. To demonstrate the generality of this approach we build a system named CombDiff, which can differentiate nontrivial variational problems such as Hartree-Fock (HF) theory and multilayer perceptrons.

Mathematical Software,Programming Languages,Numerical Analysis
Genetic algorithm dynamics on a rugged landscape

Stefan Bornholdt

DOI: https://doi.org/10.48550/arXiv.cond-mat/9903076

1999-03-04

Statistical Mechanics

Abstract:The genetic algorithm is an optimization procedure motivated by biological evolution and is successfully applied to optimization problems in different areas. A statistical mechanics model for its dynamics is proposed based on the parent-child fitness correlation of the genetic operators, making it applicable to general fitness landscapes. It is compared to a recent model based on a maximum entropy ansatz. Finally it is applied to modeling the dynamics of a genetic algorithm on the rugged fitness landscape of the NK model.
Genetic Programming Based Symbolic Regression for Analytical Solutions to Differential Equations

Hongsup Oh,Roman Amici,Geoffrey Bomarito,Shandian Zhe,Robert Kirby,Jacob Hochhalter

DOI: https://doi.org/10.48550/arXiv.2302.03175

2023-02-07

Abstract:In this paper, we present a machine learning method for the discovery of analytic solutions to differential equations. The method utilizes an inherently interpretable algorithm, genetic programming based symbolic regression. Unlike conventional accuracy measures in machine learning we demonstrate the ability to recover true analytic solutions, as opposed to a numerical approximation. The method is verified by assessing its ability to recover known analytic solutions for two separate differential equations. The developed method is compared to a conventional, purely data-driven genetic programming based symbolic regression algorithm. The reliability of successful evolution of the true solution, or an algebraic equivalent, is demonstrated.

Machine Learning
A Variational Perspective on High-Resolution ODEs

Hoomaan Maskan,Konstantinos C. Zygalakis,Alp Yurtsever

2023-11-04

Abstract:We consider unconstrained minimization of smooth convex functions. We propose a novel variational perspective using forced Euler-Lagrange equation that allows for studying high-resolution ODEs. Through this, we obtain a faster convergence rate for gradient norm minimization using Nesterov's accelerated gradient method. Additionally, we show that Nesterov's method can be interpreted as a rate-matching discretization of an appropriately chosen high-resolution ODE. Finally, using the results from the new variational perspective, we propose a stochastic method for noisy gradients. Several numerical experiments compare and illustrate our stochastic algorithm with state of the art methods.

Optimization and Control,Machine Learning,Numerical Analysis
A neural network approach to learning solutions of a class of elliptic variational inequalities

Amal Alphonse,Michael Hintermüller,Alexander Kister,Chin Hang Lun,Clemens Sirotenko

2024-11-28

Abstract:We develop a weak adversarial approach to solving obstacle problems using neural networks. By employing (generalised) regularised gap functions and their properties we rewrite the obstacle problem (which is an elliptic variational inequality) as a minmax problem, providing a natural formulation amenable to learning. Our approach, in contrast to much of the literature, does not require the elliptic operator to be symmetric. We provide an error analysis for suitable discretisations of the continuous problem, estimating in particular the approximation and statistical errors. Parametrising the solution and test function as neural networks, we apply a modified gradient descent ascent algorithm to treat the problem and conclude the paper with various examples and experiments. Our solution algorithm is in particular able to easily handle obstacle problems that feature biactivity (or lack of strict complementarity), a situation that poses difficulty for traditional numerical methods.

Optimization and Control,Numerical Analysis
A Constraint-Handling Technique for Genetic Algorithms using a Violation Factor

Adam Chehouri,Rafic Younes,Jean Perron,Adrian Ilinca

DOI: https://doi.org/10.3844/jcssp.2016.350.362

2016-10-04

Abstract:Over the years, several meta-heuristic algorithms were proposed and are now emerging as common methods for constrained optimization problems. Among them, genetic algorithms (GA's) shine as popular evolutionary algorithms (EA's) in engineering optimization. Most engineering design problems are difficult to resolve with conventional optimization algorithms because they are highly nonlinear and contain constraints. In order to handle these constraints, the most common technique is to apply penalty functions. The major drawback is that they require tuning of parameters, which can be very challenging. In this paper, we present a constraint-handling technique for GA's solely using the violation factor, called VCH (Violation Constraint-Handling) method. Several benchmark problems from the literature are examined. The VCH technique was able to provide a consistent performance and match results from other GA-based techniques.

Artificial Intelligence,Optimization and Control
An improved genetic algorithm for numerical function optimization

Yingying Song,Fulin Wang,Xinxin Chen

DOI: https://doi.org/10.1007/s10489-018-1370-4

IF: 5.3

2018-12-15

Applied Intelligence

Abstract:To avoid problems such as premature convergence and falling into a local optimum, this paper proposes an improved real-coded genetic algorithm (RCGA-rdn) to improve the performance in solving numerical function optimization. These problems are mainly caused by the poor search ability of the algorithm and the loss of population diversity. Therefore, to improve the search ability, the algorithm integrates three specially designed operators: ranking group selection (RGS), direction-based crossover (DBX) and normal mutation (NM). In contrast to the traditional strategy framework, RCGA-rdn introduces a new step called the replacement operation, which periodically performs a local initialization operation on the population to increase the population diversity. In this paper, comparisons with several advanced algorithms were performed on 21 complex constrained optimization problems and 10 high-dimensional unconstrained optimization problems to verify the effectiveness of RCGA-rdn. Based on the results, to further verify the feasibility of the algorithm, it was applied to a series of practical engineering optimization problems. The experimental results show that the proposed operations can effectively improve the performance of the algorithm. Compared with the other algorithms, the improved algorithm (RCGA-rdn) has a better search ability, faster convergence speed and can maintain a certain population diversity.

computer science, artificial intelligence
Genetic Algorithm: An Approach on Optimization

Savio D Immanuel,Udit Kr. Chakraborty

DOI: https://doi.org/10.1109/icces45898.2019.9002372

2019-07-01

Abstract:Solutions for both constrained and unconstrained problems of optimization pose a challenge from the past till date. The genetic algorithm is a technique for solving such optimization problems based on biological laws of evolution particularly natural selection. In simple terms, a genetic algorithm is a successor to the traditional evolutionary algorithm where at each step it will select random solutions from the present population and labels those as parents and uses them to reproduce to the next generation as children with a series of biological operations, namely reproduction, selection, crossover and mutation.
Primal Methods for Variational Inequality Problems with Functional Constraints

Liang Zhang,Niao He,Michael Muehlebach

2024-03-20

Abstract:Constrained variational inequality problems are recognized for their broad applications across various fields including machine learning and operations research. First-order methods have emerged as the standard approach for solving these problems due to their simplicity and scalability. However, they typically rely on projection or linear minimization oracles to navigate the feasible set, which becomes computationally expensive in practical scenarios featuring multiple functional constraints. Existing efforts to tackle such functional constrained variational inequality problems have centered on primal-dual algorithms grounded in the Lagrangian function. These algorithms along with their theoretical analysis often require the existence and prior knowledge of the optimal Lagrange multipliers. In this work, we propose a simple primal method, termed Constrained Gradient Method (CGM), for addressing functional constrained variational inequality problems, without necessitating any information on the optimal Lagrange multipliers. We establish a non-asymptotic convergence analysis of the algorithm for variational inequality problems with monotone operators under smooth constraints. Remarkably, our algorithms match the complexity of projection-based methods in terms of operator queries for both monotone and strongly monotone settings, while utilizing significantly cheaper oracles based on quadratic programming. Furthermore, we provide several numerical examples to evaluate the efficacy of our algorithms.

Optimization and Control,Machine Learning

Genetic algorithms for the numerical solution of variational problems without analytic trial functions

The Use of Genetic Algorithms for Optimizing the Regularized Solutions of the Ill-Posed Problems

Genetic Algorithm-Accelerated Optimization for Runge-Kutta Scheme Derivation in High-Dimensional Problems With Hard Constraints

Variations of Genetic Algorithms

Easily implementable iterative methods for variational inequalities with nonlinear diffusion–convection operator and constraints to the gradient of solution

Kinetic description and convergence analysis of genetic algorithms for global optimization

Genetic optimization algorithms applied toward mission computability models

Solution to Quadratic Equation Using Genetic Algorithm

Numerical approximation of variational problems with orthotropic growth

Optimizing Variational Physics-Informed Neural Networks Using Least Squares

Parallel iterative methods for variational integration applied to navigation problems

Algorithm for Variational Inequality Problems Based on a Gradient Dynamical System Designed Using a Control Liapunov Function

Automating Variational Differentiation

Genetic algorithm dynamics on a rugged landscape

Genetic Programming Based Symbolic Regression for Analytical Solutions to Differential Equations

A Variational Perspective on High-Resolution ODEs

A neural network approach to learning solutions of a class of elliptic variational inequalities

A Constraint-Handling Technique for Genetic Algorithms using a Violation Factor

An improved genetic algorithm for numerical function optimization

Genetic Algorithm: An Approach on Optimization

Primal Methods for Variational Inequality Problems with Functional Constraints