New experimental evidence for mechanism of arrhythmogenic membrane potential alternans based on balance of electrogenic I(NCX)/I(Ca) currents.

X. Wan,M. Cutler,Z. Song,A. Karma,T. Matsuda,A. Baba,D. Rosenbaum

DOI: https://doi.org/10.1016/j.hrthm.2012.06.031

IF: 6.779

2012-10-01

Heart Rhythm

Abstract:

What problem does this paper attempt to address?

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

Xinggao Liu,Yunqing Hu,Jianghua Feng,Kean Liu

DOI: https://doi.org/10.1109/tac.2014.2317293

IF: 6.549

2014-01-01

IEEE Transactions on Automatic Control

Abstract:The control vector parameterization (CVP) approach is popular to solve nonlinear dynamic optimization problems, but handling inequality path constraints within its framework is a challenge. This study deals with inequality path constraints by using the l(1) penalty function and a novel smoothing technique. After clarifying some theories of the proposed approach, a concomitant algorithm is also put forward and verified.
Neural Network Solution for General Nonlinear Optimization Problems

YL Zheng,LH Ma,JX Qian

DOI: https://doi.org/10.1109/tencon.2002.1182664

2002-01-01

Abstract:Nonlinear optimization is the foundation of multiobjective programming. In this paper, our focus is augmented Lagrange multiplier(ALM) method which is employed in constrained nonlinear optimization propositions. A dynamic approach is raised for minimization subproblem in augmented Lagrange multiplier method and a neural network alternative algorithm for general constrained programming is proposed.
Subgradient-based Feedback Neural Networks for Non-Differentiable Convex Optimization Problems

Li Guocheng,Song Shiji,Wu Cheng

DOI: https://doi.org/10.1007/s11432-006-2007-5

2006-01-01

Science in China Series F Information Sciences

Abstract:This paper developed the dynamic feedback neural network model to solve the convex nonlinear programming problem proposed by Leung et al. and introduced subgradient-based dynamic feedback neural networks to solve non-differentiable convex optimization problems. For unconstrained non-differentiable convex optimization problem, on the assumption that the objective function is convex coercive, we proved that with arbitrarily given initial value, the trajectory of the feedback neural network constructed by a projection subgradient converges to an asymptotically stable equilibrium point which is also an optimal solution of the primal unconstrained problem. For constrained non-differentiable convex optimization problem, on the assumption that the objective function is convex coercive and the constraint functions are convex also, the energy functions sequence and corresponding dynamic feedback subneural network models based on a projection subgradient are successively constructed respectively, the convergence theorem is then obtained and the stopping condition is given. Furthermore, the effective algorithms are designed and some simulation experiments are illustrated.
A discrete-time neural network for optimization problems with hybrid constraints.

Huajin Tang,Haizhou Li,Zhang Yi

DOI: https://doi.org/10.1109/TNN.2010.2049368

2010-01-01

Abstract:Recurrent neural networks have become a prominent tool for optimizations including linear or nonlinear variational inequalities and programming, due to its regular mathematical properties and well-defined parallel structure. This brief presents a general discrete-time recurrent network for linear variational inequalities and related optimization problems with hybrid constraints. In contrary to the existing discrete-time networks, this general model can operate not only on bound constraints, but also on hybrid constraints comprised of inequality, equality and bound constraints. The model has dynamical properties of global convergence, asymptotical and exponential convergences under some weaker conditions. Numerical examples demonstrate its efficacy and performance.
Differential inclusions-based neural networks for nonsmooth convex optimization on a closed convex subset

Shiji Song,Guocheng Li,Xiaohong Guan

DOI: https://doi.org/10.1007/11759966_54

2006-01-01

Abstract:Differential inclusions-based dynamic feedback neural network models are introduced to solve in real time nonsmooth convex optimization problems restricted on a closed convex subset of Rn. First,a differential inclusion-based dynamic feedback neural network model for solving unconstrained optimization problem is established, and its stability and convergence are investigated, then based on the preceding results and the method of successive approximation, differential inclusions-based dynamic feedback neural network models for solving in real time nonsmooth optimization problem on a closed convex subset are successively constructed, and its dynamical behavior and optimization capabilities are analyzed rigorously.
A nonautonomous-differential-inclusion neurodynamic approach for nonsmooth distributed optimization on multi-agent systems

Xingnan Wen,Yixuan Wang,Sitian Qin

DOI: https://doi.org/10.1007/s00521-021-06026-2

2021-04-26

Neural Computing and Applications

Abstract:This paper considers a category of nonsmooth distributed optimization on multi-agent systems, where agents own privacies and collectively minimize a sum of local cost functions. Taking the restrictions on communication among agents into consideration, a nonautonomous-differential-inclusion neurodynamic approach is proposed over a weighed topology graph. The convergence of neural network is analyzed and its state exponentially converges to an optimal solution of distributed optimization under certain conditions. Since no additional conditions are required to guarantee the convergence, the neural network is superior to distributed algorithms based on penalty method, which need to estimate penalty parameters. Compared with some existed approaches, the neural network has the advantage of possessing fewer state variables. Finally, illustrative examples and an application in distributed quantile regression are delineated to testify the effectiveness of the presented neural network.

computer science, artificial intelligence
Neurodynamic approaches for multi-agent distributed optimization

Luyao Guo,Iakov Korovin,Sergey Gorbachev,Xinli Shi,Nadezhda Gorbacheva,Jinde Cao

DOI: https://doi.org/10.1016/j.neunet.2023.11.025

Abstract:This paper considers a class of multi-agent distributed convex optimization with a common set of constraints and provides several continuous-time neurodynamic approaches. In problem transformation, l1 and l2 penalty methods are used respectively to cast the linear consensus constraint into the objective function, which avoids introducing auxiliary variables and only involves information exchange among primal variables in the process of solving the problem. For nonsmooth cost functions, two differential inclusions with projection operator are proposed. Without convexity of the differential inclusions, the asymptotic behavior and convergence properties are explored. For smooth cost functions, by harnessing the smoothness of l2 penalty function, finite- and fixed-time convergent algorithms are provided via a specifically designed average consensus estimator. Finally, several numerical examples in the multi-agent simulation environment are conducted to illustrate the effectiveness of the proposed neurodynamic approaches.
A neurodynamic optimization approach to distributed nonconvex optimization based on an HP augmented Lagrangian function

Huimin Guan,Yang Liu,Kit Ian Kou,Weihua Gui

DOI: https://doi.org/10.1016/j.neunet.2024.106791

IF: 7.8

2024-10-23

Neural Networks

Abstract:This paper develops a neurodynamic model for distributed nonconvex-constrained optimization. In the distributed constrained optimization model, the objective function and inequality constraints do not need to be convex, and equality constraints do not need to be affine. A Hestenes–Powell augmented Lagrangian function for handling the nonconvexity is established, and a neurodynamic system is developed based on this. It is proved that it is stable at a local optimal solution of the optimization model. Two illustrative examples are provided to evaluate the enhanced stability and optimality of the developed neurodynamic systems.

computer science, artificial intelligence,neurosciences
A neural network model for non-smooth optimization over a compact convex subset

Guocheng Li,Shiji Song,Cheng Wu,Zifang Du

DOI: https://doi.org/10.1007/11759966_53

2006-01-01

Abstract:A neural network model is introduced which is aimed to solve non-smooth optimization problem on a nonempty compact convex subset of Rn. By using the subgradient, this neural network model is shown to obey a gradient system of differential inclusion. It is proved that the compact convex subset is a positive invariant and is a attractive to the neural network system, and that all the network trajectories starting from the inside of the compact convex subset converge to the set of equilibrium points of the neural network. The above every equilibrium point of the neural network is an optimal solution of the primal problem. A numerical simulation example is also given to illustrate the qualitative properties of the proposed neural network model.
Computing subgradients of convex relaxations for solutions of parametric ordinary differential equations

Yingkai Song,Kamil A. Khan

DOI: https://doi.org/10.1080/10556788.2024.2346641

2024-05-23

Optimization Methods and Software

Abstract:A novel subgradient evaluation method is proposed for nonsmooth convex relaxations of parametric solutions of ordinary differential equations (ODEs) arising in global dynamic optimization, assuming that the relaxations always lie strictly within interval bounds during integration. We argue that this assumption is reasonable in practice. These subgradients are computed as the unique solution of an auxiliary parametric affine ODE, analogous to classical forward/tangent sensitivity evaluation methods for smooth dynamic systems. Unlike established subgradient evaluation approaches for nonsmooth dynamic systems, this new method does not require smoothness or transversality assumptions, and is compatible with existing subgradient evaluation methods for closed-form convex functions, as implemented in subgradient evaluation software such as EAGO.jl and MC ++ . Moreover, we show that a subgradient for a lower-bounding problem in global dynamic optimization can be directly evaluated using reverse/adjoint sensitivity analysis, which may reduce the overall computational effort for an overarching global optimization method. Numerical examples are presented, based on a proof-of-concept implementation in Julia.

operations research & management science,computer science, software engineering,mathematics, applied
A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization

Songtao Lu

2024-12-03

Abstract:Nonconvex constrained optimization problems can be used to model a number of machine learning problems, such as multi-class Neyman-Pearson classification and constrained Markov decision processes. However, such kinds of problems are challenging because both the objective and constraints are possibly nonconvex, so it is difficult to balance the reduction of the loss value and reduction of constraint violation. Although there are a few methods that solve this class of problems, all of them are double-loop or triple-loop algorithms, and they require oracles to solve some subproblems up to certain accuracy by tuning multiple hyperparameters at each iteration. In this paper, we propose a novel gradient descent and perturbed ascent (GDPA) algorithm to solve a class of smooth nonconvex inequality constrained problems. The GDPA is a primal-dual algorithm, which only exploits the first-order information of both the objective and constraint functions to update the primal and dual variables in an alternating way. The key feature of the proposed algorithm is that it is a single-loop algorithm, where only two step-sizes need to be tuned. We show that under a mild regularity condition GDPA is able to find Karush-Kuhn-Tucker (KKT) points of nonconvex functional constrained problems with convergence rate guarantees. To the best of our knowledge, it is the first single-loop algorithm that can solve the general nonconvex smooth problems with nonconvex inequality constraints. Numerical results also showcase the superiority of GDPA compared with the best-known algorithms (in terms of both stationarity measure and feasibility of the obtained solutions).

Optimization and Control,Artificial Intelligence,Machine Learning,Numerical Analysis
[Pharmacology of diuretics--molecular pharmacology--introduction and the future of quantum chemistry].

Y. Orita,A. Ando,Y. Takamitsu

1970-04-01

Abstract:
A Two-Timescale Duplex Neurodynamic Approach to Mixed-Integer Optimization

Hangjun Che,Jun Wang

DOI: https://doi.org/10.1109/tnnls.2020.2973760

IF: 14.255

2021-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:This article presents a two-timescale duplex neurodynamic approach to mixed-integer optimization, based on a biconvex optimization problem reformulation with additional bilinear equality or inequality constraints. The proposed approach employs two recurrent neural networks operating concurrently at two timescales. In addition, particle swarm optimization is used to update the initial neuronal states iteratively to escape from local minima toward better initial states. In spite of its minimal system complexity, the approach is proven to be almost surely convergent to optimal solutions. Its superior performance is substantiated via solving five benchmark problems.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Algorithm for Variational Inequality Problems Based on a Gradient Dynamical System Designed Using a Control Liapunov Function

Fernando Agustin Pazos,Amit Bhaya

DOI: https://doi.org/10.1109/isic.2007.4450953

2007-01-01

Abstract:We present an algorithm to find the optimal point of a variational inequality problem. The domain of the function that defines the variational inequality is a convex set, determined by convex inequality constraints and affine equality constraints. The algorithm is based on a discrete variable structure closed-loop control system which presents sliding mode trajectories on the boundary of the feasible set until the optimal point is reached. The update law is designed using control Liapunov function (CLF), which guarantees the decrease of a discrete Liapunov function inside and outside the feasible set. The step size is optimized using Liapunov optimizing control (LOC).
A discrete adjoint gradient approach for equality and inequality constraints in dynamics

Daniel Lichtenecker,Karin Nachbagauer

DOI: https://doi.org/10.1007/s11044-024-09965-5

IF: 3.4

2024-01-30

Multibody System Dynamics

Abstract:The optimization of multibody systems requires accurate and efficient methods for sensitivity analysis. The adjoint method is probably the most efficient way to analyze sensitivities, especially for optimization problems with numerous optimization variables. This paper discusses sensitivity analysis for dynamic systems in gradient-based optimization problems. A discrete adjoint gradient approach is presented to compute sensitivities of equality and inequality constraints in dynamic simulations. The constraints are combined with the dynamic system equations, and the sensitivities are computed straightforwardly by solving discrete adjoint algebraic equations. The computation of these discrete adjoint gradients can be easily adapted to deal with different time integrators. This paper demonstrates discrete adjoint gradients for two different time-integration schemes and highlights efficiency and easy applicability. The proposed approach is particularly suitable for problems involving large-scale models or high-dimensional optimization spaces, where the computational effort of computing gradients by finite differences can be enormous. Three examples are investigated to validate the proposed discrete adjoint gradient approach. The sensitivity analysis of an academic example discusses the role of discrete adjoint variables. The energy optimal control problem of a nonlinear spring pendulum is analyzed to discuss the efficiency of the proposed approach. In addition, a flexible multibody system is investigated in a combined optimal control and design optimization problem. The combined optimization provides the best possible mechanical structure regarding an optimal control problem within one optimization.

mechanics
Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

You Zhao,Xing He,Mingliang Zhou,Tingwen Huang

DOI: https://doi.org/10.1109/jas.2024.124380

2024-05-28

IEEE/CAA Journal of Automatica Sinica

Abstract:The Nesterov accelerated dynamical approach serves as an essential tool for addressing convex optimization problems with accelerated convergence rates. Most previous studies in this field have primarily concentrated on unconstrained smooth convex optimization problems. In this paper, on the basis of primal-dual dynamical approach, Nesterov accelerated dynamical approach, projection operator and directional gradient, we present two accelerated primal-dual projection neurodynamic approaches with time scaling to address convex optimization problems with smooth and nonsmooth objective functions subject to linear and set constraints, which consist of a second-order ODE (ordinary differential equation) or differential conclusion system for the primal variables and a first-order ODE for the dual variables. By satisfying specific conditions for time scaling, we demonstrate that the proposed approaches have a faster convergence rate. This only requires assuming convexity of the objective function. We validate the effectiveness of our proposed two accelerated primal-dual projection neurodynamic approaches through numerical experiments.

automation & control systems
Stochastic subgradient for composite optimization with functional constraints

Ion Necoara,Nitesh Kumar Singh

2024-01-15

Abstract:In this paper we consider convex optimization problems with stochastic composite objective function subject to (possibly) infinite intersection of constraints. The objective function is expressed in terms of expectation operator over a sum of two terms satisfying a stochastic bounded gradient condition, with or without strong convexity type properties. In contrast to the classical approach, where the constraints are usually represented as intersection of simple sets, in this paper we consider that each constraint set is given as the level set of a convex but not necessarily differentiable function. Based on the flexibility offered by our general optimization model we consider a stochastic subgradient method with random feasibility updates. At each iteration, our algorithm takes a stochastic proximal (sub)gradient step aimed at minimizing the objective function and then a subsequent subgradient step minimizing the feasibility violation of the observed random constraint. We analyze the convergence behavior of the proposed algorithm for diminishing stepsizes and for the case when the objective function is convex or strongly convex, unifying the nonsmooth and smooth cases. We prove sublinear convergence rates for this stochastic subgradient algorithm, which are known to be optimal for subgradient methods on this class of problems. When the objective function has a linear least-square form and the constraints are polyhedral, it is shown that the algorithm converges linearly. Numerical evidence supports the effectiveness of our method in real problems.

Optimization and Control
Enhancing neurodynamic approach with physics-informed neural networks for solving non-smooth convex optimization problems

Dawen Wu,Abdel Lisser

DOI: https://doi.org/10.1016/j.neunet.2023.08.014

Abstract:This paper proposes a deep learning approach for solving non-smooth convex optimization problems (NCOPs), which have broad applications in computer science, engineering, and physics. Our approach combines neurodynamic optimization with physics-informed neural networks (PINNs) to provide an efficient and accurate solution. We first use neurodynamic optimization to formulate an initial value problem (IVP) that involves a system of ordinary differential equations for the NCOP. We then introduce a modified PINN as an approximate state solution to the IVP. Finally, we develop a dedicated algorithm to train the model to solve the IVP and minimize the NCOP objective simultaneously. Unlike existing numerical integration methods, a key advantage of our approach is that it does not require the computation of a series of intermediate states to produce a prediction of the NCOP. Our experimental results show that this computational feature results in fewer iterations being required to produce more accurate prediction solutions. Furthermore, our approach is effective in finding feasible solutions that satisfy the NCOP constraint.
A mixture varying-gain dynamic learning network for solving nonlinear and nonconvex constrained optimization problems

Rongxiu Lu,Guanhua Qiu,Zhijun Zhang,Xianzhi Deng,Hui Yang,Zhenmin Zhu,Jianyong Zhu

DOI: https://doi.org/10.1016/j.neucom.2021.05.037

IF: 6

2021-10-01

Neurocomputing

Abstract:<p>Nonlinear and nonconvex optimization problem (NNOP) is a challenging problem in control theory and applications. In this paper, a novel mixture varyinggain dynamic learning network (MVG-DLN) is proposed to solve NNOP with inequality constraints. To do so, first, this NNOP is transformed into some equations through Karush-Kuhn-Tucker (KKT) conditions and projection theorem, and the neuro-dynamics function can be obtained. Second, the time varying convergence parameter is utilized to obtain a faster convergence speed. Third, an integral term is used to strengthen the robustness. Theoretical analysis proves that the proposed MVG-DLN has global convergence and good robustness. Three numerical simulation comparisons between FT-FP-CDNN and MVG-DLN substantiate the faster convergence performance and greater robustness of the MVG-DLN in solving the nonlinear and nonconvex optimization problems.</p>

computer science, artificial intelligence
An inverse-free Zhang neural dynamic for time-varying convex optimization problems with equality and affine inequality constraints

Shuai Shao,Haojin Li,Sitian Qin,Guocheng Li,Cheng Luo

DOI: https://doi.org/10.1016/j.neucom.2020.06.051

IF: 6

2020-10-01

Neurocomputing

Abstract:<p>Time-varying convex optimization problems have attracted a great deal of attention in many fields due to its widespread application. Particularly, the approach to time-varying convex optimization problems with equality and affine inequality constraints simultaneously is a comprehensive but complicated problem at present. In this paper, three types of inverse-free Zhang neural dynamic (ZND) including two noise-tolerance ZND models are proposed and investigated for solving time-varying convex optimization problems with equality and affine inequality constraints. It is noted that the proposed noise-tolerance ZND models own the ability to suppress noise and one of those even achieves finite-time convergency. Compared with previous work, the proposed inverse-free ZND models effectively reduce the computation complexity by simplifying the model structure and conquer the problem of solving real-time matrix inverse during the computation process, which is more appropriate for a wider practical application.</p>

computer science, artificial intelligence

New experimental evidence for mechanism of arrhythmogenic membrane potential alternans based on balance of electrogenic I(NCX)/I(Ca) currents.

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

Neural Network Solution for General Nonlinear Optimization Problems

Subgradient-based Feedback Neural Networks for Non-Differentiable Convex Optimization Problems

A discrete-time neural network for optimization problems with hybrid constraints.

Differential inclusions-based neural networks for nonsmooth convex optimization on a closed convex subset

A nonautonomous-differential-inclusion neurodynamic approach for nonsmooth distributed optimization on multi-agent systems

Neurodynamic approaches for multi-agent distributed optimization

A neurodynamic optimization approach to distributed nonconvex optimization based on an HP augmented Lagrangian function

A neural network model for non-smooth optimization over a compact convex subset

Computing subgradients of convex relaxations for solutions of parametric ordinary differential equations

A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization

[Pharmacology of diuretics--molecular pharmacology--introduction and the future of quantum chemistry].

A Two-Timescale Duplex Neurodynamic Approach to Mixed-Integer Optimization

Algorithm for Variational Inequality Problems Based on a Gradient Dynamical System Designed Using a Control Liapunov Function

A discrete adjoint gradient approach for equality and inequality constraints in dynamics

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Stochastic subgradient for composite optimization with functional constraints

Enhancing neurodynamic approach with physics-informed neural networks for solving non-smooth convex optimization problems

A mixture varying-gain dynamic learning network for solving nonlinear and nonconvex constrained optimization problems

An inverse-free Zhang neural dynamic for time-varying convex optimization problems with equality and affine inequality constraints