Optimal Control of Hybrid Systems Based on Dynamic Programming

ZS Yin,P Li

DOI: https://doi.org/10.1109/acc.2001.945828

2002-01-01

Abstract:The optimization of hybrid systems is studied in the paper, dynamic programming based on reachable net is proposed. This method can get the optimum and reduce the computing effectively.

What problem does this paper attempt to address?

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

HF Zhai,HY Su,J Chu,WM Wu,HH Wu

DOI: https://doi.org/10.1109/wcica.2000.862442

2000-01-01

Abstract:In this paper, the model of hybrid systems restricted to time-deterministic discrete transitions is presented, and the quadratic optimal control problem for the hybrid systems is investigated using a new method, called the mixed dynamic programming method. The stability of the closed-loop hybrid systems is proved by the Lyapunov method. The proposed design approach is illustrated by a numerical example
Study on optimal control strategy for a class of piecewise linear hybrid systems

翟海峰,苏宏业,董利达,王肖,褚健

DOI: https://doi.org/10.3321/j.issn:1001-0920.2002.06.008

2002-01-01

Abstract:The model of hybrid systems restricted to time-deterministic discrete transitions is presented. The quadratic optimal control problem for the hybrid systems is investigated based on mixed dynamical programming methods. The stability of the closed-loop hybrid systems is proved by Lyapunov method. A numerical example illustrates the effectiveness of the design approach.
On Modeling and Optimal Control of a Class of Hybrid Systems

ZHANG Li-yan,XU Hua-zhong,QIAN Ji-xin

DOI: https://doi.org/10.3969/j.issn.1671-7848.2005.05.003

2005-01-01

Abstract:Optimal control of hybrid systems encounters the difficulty of computational complexity due to hybrid systems involving continuous variable and discrete variable.Hybrid model mixed with logics and rules is presented. Mixed integer nonlinear programming algorithm integrated constraint programming is applied to solve optimal control problems of hybrid model. The practical result shows that this method can build more precision model by utilizing mechanism model of industrial process and expert experience. And it also shows that the algorithm of mixed integer nonlinear programming integrated constraint programming can solve the optimal control problem of hybrid model rapidly. So this method can be used in real-time control of industrial process.
AN APPROACH TO MULTI-OBJECTIVE OPTIMAL CONTROL PROBLEMS FOR HYBRID DYNAMIC SYSTEMS

李艳君,吴铁军

DOI: https://doi.org/10.16383/j.aas.2002.04.023

2002-01-01

Abstract:This paper proposes a generic approach based on genetic algorithms to solve the multi-objective optimal control problems in hybrid dynamic systems. A case study based on the control problem of a multi-tank blending system illustrates that this approach deals effectively with such kind of complicated problems which could not be solved in classical optimal control theories.
Optimal control of a class of hybrid systems with states and inputs constraints

张聚,李平

DOI: https://doi.org/10.3785/j.issn.1008-973X.2003.02.003

2003-01-01

Abstract:Based on the discrete time mixed Logical Dynamical (MLD) model of hybrid systems, the Mixed Integer quadratic Programming (MIQP) approach for optimal control of hybrid systems with inputs and states constraints is studied. MIQP approach can easily deal with constraints which had to be satisfied because of the physical restrictions or for operation safety. A quadratic performance criterion is used, and the problem of constrained optimal control of hybrid systems is transformed into an MIQP problem. Different from that of the usual Quadratic Programming (QP) problem, some decision variables of the MIQP problem had to be constrained to be 0 or 1. Based on the Branch and Bound strategy and genetic algorithm respectively, procedures for solving an MIQP problem are discussed and a MATLAB program is developed as an MIQP solver. At the end of this paper, a numerical example is presented and computerized simulations show that the MIQP approach is an effective and promising method for optimal control of hybrid systems.
Near Optimal Control for a Class of Stochastic Hybrid Systems.

Chunyue Song,Ping Li

DOI: https://doi.org/10.1016/j.automatica.2010.05.024

IF: 6.4

2010-01-01

Automatica

Abstract:To address a computationally intractable optimal control problem for a class of stochastic hybrid systems, this paper proposes a near optimal state feedback control scheme, which is constructed by using a statistical prediction method based on approximate numerical solution that samples over the entire state space. A numerical example illustrates the potential of the approach.
Mixed Integer Programming Based Optimization of a Class of Hybrid Systems

张聚,李平

DOI: https://doi.org/10.3321/j.issn:1006-2467.2002.z1.003

2002-01-01

Shanghai Jiaotong Daxue Xuebao/Journal of Shanghai Jiaotong University

Abstract:Based on the discrete time mixed logical dynamical (MLD) model of hybrid systems and on the mixed integer quadratic programming (MIQP), the optimization of a class of hybrid systems with states and inputs constraints was studied. The problem of constrained optimization of hybrid systems is transformed into an MIQP problem. Based on the Branch and Bound strategy, an MATLAB program is developed as MIQP solver. Finally, computer simulations are made. The simulation results show that the MIQP approach is an effective and promising method for the optimization of hybrid systems.
A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Chunyue Song,Bing Wu,Ping Li

DOI: https://doi.org/10.1016/j.jprocont.2012.02.013

IF: 3.951

2012-01-01

Journal of Process Control

Abstract:To improve the overall control performance of nonlinear systems, an optimal control method, based on the framework of hybrid systems, is proposed. Firstly, the nonlinear systems are approximated by a number of piecewise affine models which are produced by the nonlinear systems at the specified operating points, then the piecewise affine models are synthesized under the framework of hybrid systems, and an associated optimal control problem, in which decision variables involve not only admissible continuous control but also the scheduling of subsystem modes, is established. Secondly, the optimal control problem is transformed into a MIQP problem by discretization over the whole state space and admissible control space to obtain the numerical optimal solution. For speeding up the algorithm, the simultaneous method on finite elements is used to lower the dimensions of the MIQP problem. Consequently, a hybrid model-based MPC for nonlinear systems is designed, and the adverse effects of model mismatch resulted from simultaneous method is weakened by MPC strategy. Simulations and comparisons with soft-switching method, hard-switching method and MLD method, confirm that a satisfactory performance can be obtained using the presented approach. © 2012 Elsevier Ltd. All rights reserved.
Optimal Control Of A Class Of Non-Linear Discrete-Continuous Hybrid Systems

Minghui Hu,Qian Gao,Huihe Shao

DOI: https://doi.org/10.1109/WCICA.2006.1712460

2006-01-01

Abstract:This work is derived for solving a non-linear discrete-continuous systems optimal control problem by iterating on a sequence of simplified problems in linear quadratic form. A mixed approach with a discrete performance index and continuous state variable system description is used as the basis of the design, and it is shown how the global problem can be decomposed into local sub-system problems and a coordinator within a hierarchical framework, so the problem become simple. The correct optimal solution of a real system in which model-reality difference exists can be obtained from system model by interconnected costate prediction iterative solution. An analysis of the optimality and convergence properties of the algorithm is presented and the efficiency of the technique is demonstrated.
Varying Receding-Horizon Based Production Control for Hybrid Production Systems

SONG Chun-yue,LI Ping

DOI: https://doi.org/10.3785/j.issn.1008-973x.2005.12.025

2005-01-01

Abstract:To simplify the complexities of the production optimum control problem,a new policy composed of dynamic programming and varying receding-horizon optimization was initiated.After analyzing the model of the hybrid production system involving setup cost and stochastic failure-prone,the original problem was simplified into a stochastic dynamic programming problem over finite horizon,and the associated Bellman equation was also proposed.By introducing the idea of just-in-time,the stochastic optimal production control problem of multi-dimension vector was reduced to a limited sub-problem of the system under expectation meaning,and the solutions of the simplified problem were made close to that of the original proposition by varying receding-horizon optimization on one dimension.The simulation results show that the proposed policy can avoid solving partial differential equations directly,and makes the receding algorithm feasible,more accurate and realtime.
Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Zhongpu Xia,Dongbin Zhao,Huajin Tang

DOI: https://doi.org/10.3182/20140824-6-za-1003.01368

2014-01-01

IFAC Proceedings Volumes

Abstract:Abstract In this paper, a model-free and effective approach is proposed to solve infinite horizon optimal control problem for affine nonlinear systems based on adaptive dynamic programming technique. The developed approach, referred to as the actor-critic structure, employs two multilayer perceptron neural networks to approximate the state-action value function and the control policy, respectively. It uses data collected arbitrarily from any reasonable sampling distribution for policy iteration. In the policy evaluation phase, a novel objective function is defined for updating the critic network, and thus makes the critic network converge to the Bellman equation directly rather than iteratively. In the policy improvement phase, the action network is updated to minimize the outputs of the critic network. The two phases alternate until no more improvement of the control policy is observed, such that the optimal control policy is achieved. Two simulation examples are provided to show the effectiveness of the approach.
Optimal Control of Switched Hybrid Systems

Jian Xu,Qijun Chen

2011-01-01

Abstract:This paper considers the optimal control of switched hybrid systems in which the switching sequence is not specified. First, a depth first search is used to find all the possible paths between the initial subsystem and the terminal subsystem. Then, the switching points are fixed through a nonlinear programming method, and hence the optimal control of switched hybrid systems is reduced to the conventional optimal control problem. Finally, An estimation of the lower bound and the upper bound of cost in each path is given in order to reduce the computational complexity.
On the Relation between the Minimum Principle and Dynamic Programming for Classical and Hybrid Control Systems

Ali Pakniyat,Peter E. Caines

DOI: https://doi.org/10.1109/TAC.2017.2667043

2016-09-11

Abstract:Hybrid optimal control problems are studied for a general class of hybrid systems where autonomous and controlled state jumps are allowed at the switching instants and in addition to terminal and running costs switching between discrete states incurs costs. The statements of the Hybrid Minimum Principle and Hybrid Dynamic Programming are presented in this framework and it is shown that under certain assumptions the adjoint process in the Hybrid Minimum Principle and the gradient of the value function in Hybrid Dynamic Programming are governed by the same set of differential equations and have the same boundary conditions and hence are almost everywhere identical to each other along optimal trajectories. Analytic examples are provided to illustrate the results and, in particular, a Riccati formalism for linear quadratic hybrid tracking problems is presented.

Optimization and Control
Intelligent Optimal Control of Constrained Nonlinear Systems Via Receding-Horizon Heuristic Dynamic Programming

Ding Wang,Peng Xin,Mingming Zhao,Junfei Qiao

DOI: https://doi.org/10.1109/tsmc.2023.3306338

2024-01-01

Abstract:For addressing the approximate optimal control problem of nonlinear affine systems with the terminal state constraint and asymmetric control constraints, the constrained receding-horizon heuristic dynamic programming (RH-HDP) algorithm is established in this article. In consideration of the RH mechanism of model predictive control (MPC), the approximate optimal control problem based on the HDP algorithm is transformed into a battery of subproblems. Then, the terminal state constraint related with the current prediction horizon is considered such that the terminal state is forced into the neighborhood of the system equilibrium point. In addition, the asymmetric control constraints are introduced to release the pressure of actuator saturation, so that the control input is well confined within the given constraint range. Meanwhile, relevant results of the stability proof are also displayed based on the Lyapunov approach. Finally, the constrained RH-HDP algorithm has been applied in two kinds of systems to verify its effectiveness. Comparative experiments with the traditional HDP algorithm have been carried out to verify the superiority of the present algorithm.
Optimal control for a class of hybrid systems in manufacturing

Jian Xu,Qijun Chen

DOI: https://doi.org/10.1109/CCDC.2009.5191723

2009-01-01

Abstract:This paper deals with the optimal control problem arising in manufacturing environment with multi-stages, in which the dynamics of the process take the form of piecewise differential equations with control constraints, and a combined time-energy cost functional. The necessary conditions of optimality are presented by variational methods, and a simple example of kinetic system is shown to illustrate the results in the paper.
Optimization-Based Design and Control of Dynamic Systems

Vediyappan Govindan,J. Jayaprakash,Choonkil Park,Jung Rye Lee,Ismail Naci Cangul

DOI: https://doi.org/10.58496/bjm/2023/006

2023-06-28

Abstract:Dynamic systems such as robots, autonomous vehicles, and process plants require careful design and control to achieve optimal performance. This paper presents an integrated framework for the simultaneous optimization of system design parameters and control policies. The underlying mathematical formulation utilizes optimization techniques to find the best system configuration and control strategy according to specified objectives. These objectives may include metrics such as tracking error, energy consumption, safety margins, and cost. Both model-based and data-driven techniques are explored for learning the system dynamics and synthesizing optimal controllers. The optimization algorithms leveraged include gradient-based methods, evolutionary algorithms, and reinforcement learning. The benefits of the joint optimization approach are demonstrated through case studies on representative dynamic systems. Results show that the integrated design and control framework outperforms sequential optimization, leading to improved efficiency, responsiveness, and robustness. This underscores the importance of co-optimizing design and control parameters, especially for complex, uncertain systems. The proposed methods provide an effective tool for next-generation automated design of smart, adaptable systems.
A "Hybrid" Approach for Synthesizing Optimal Controllers of Hybrid Systems: A Case Study of the Oil Pump Industrial Example

Hengjun Zhao,Naijun Zhan,Deepak Kapur,Kim G. Larsen

DOI: https://doi.org/10.48550/arXiv.1203.6025

2012-06-02

Abstract:In this paper, we propose an approach to reduce the optimal controller synthesis problem of hybrid systems to quantifier elimination; furthermore, we also show how to combine quantifier elimination with numerical computation in order to make it more scalable but at the same time, keep arising errors due to discretization manageable and within bounds. A major advantage of our approach is not only that it avoids errors due to numerical computation, but it also gives a better optimal controller. In order to illustrate our approach, we use the real industrial example of an oil pump provided by the German company HYDAC within the European project Quasimodo as a case study throughout this paper, and show that our method improves (up to 7.5%) the results reported in [3] based on game theory and model checking.

Systems and Control,Symbolic Computation
Costate prediction based optimal control for non-linear hybrid systems.

Minghui Hu,Yongshan Wang,Huihe Shao

DOI: https://doi.org/10.1016/j.isatra.2007.06.001

IF: 7.3

2008-01-01

ISA Transactions

Abstract:This paper is derived for solving a non-linear discrete-continuous systems optimal control problem by iterating on a sequence of simplified problems in discrete form. A mixed approach with a discrete cost function and continuous state variable system description is used as the basis of the design, and it is shown how the global problem can be decomposed into local subsystem problems and a coordinator within a hierarchical framework. The correct optimal solution to a real system in which model-reality difference exists can be obtained from the system model by interconnected costate prediction iterative solution. The algorithm efficiency and convergence properties are demonstrated by simulation study.
Dynamic Energy Management for Hybrid Electric Vehicle Based on Adaptive Dynamic Programming

Weimin Li,Guoqing Xu,Zhancheng Wang,Yangsheng Xu

DOI: https://doi.org/10.1109/wcica.2008.4594156

2008-01-01

Abstract:In this paper, an adaptive dynamic programming (ADP) based strategy for real-time energy control of parallel hybrid electric vehicles is presented. The aim is to develop a fuel-optimal control which is not relying on the priori knowledge of the future driving conditions (global optimal control), but only upon the current system operation. Adaptive dynamic programming is an on-line tuning method, which controls the system while simultaneously learning its characteristics in real time. A suboptimal energy control is then obtained with a proper definition of a cost function to be minimized at each time instant. The instantaneous cost function includes the fuel economy, emissions and the deviation of battery soc. Our approach guarantees an optimization of vehicle performance and an adaptation to driving conditions. Simulation results over standard driving cycles are presented to demonstrate the effectiveness of the proposed stochastic approach. It was found that the obtained ADP control algorithm outperforms a traditional rule-based control strategy.
Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems

C. Russell Philbrick,Peter K. Kitanidis

DOI: https://doi.org/10.1287/opre.49.3.398.11219

IF: 2.7

2001-06-01

Operations Research

Abstract:New dynamic programming methods are developed to solve stochastic control problems with a larger number of state variables than previously possible. These methods apply accurate interpolation to numerical approximation of continuous cost-to-go functions, greatly reducing the number of discrete states that must be evaluated. By efficiently incorporating information on first and second derivatives, the approximation reduces computational effort by several orders of magnitude over traditional methods. Consequently, it is practical to apply dynamic programming to complex stochastic problems with a larger number of state variables than traditionally possible. Results are presented for hypothetical reservoir control problems with up to seven state variables and two random inputs.

management,operations research & management science

Optimal Control of Hybrid Systems Based on Dynamic Programming

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

Study on optimal control strategy for a class of piecewise linear hybrid systems

On Modeling and Optimal Control of a Class of Hybrid Systems

AN APPROACH TO MULTI-OBJECTIVE OPTIMAL CONTROL PROBLEMS FOR HYBRID DYNAMIC SYSTEMS

Optimal control of a class of hybrid systems with states and inputs constraints

Near Optimal Control for a Class of Stochastic Hybrid Systems.

Mixed Integer Programming Based Optimization of a Class of Hybrid Systems

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Optimal Control Of A Class Of Non-Linear Discrete-Continuous Hybrid Systems

Varying Receding-Horizon Based Production Control for Hybrid Production Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Optimal Control of Switched Hybrid Systems

On the Relation between the Minimum Principle and Dynamic Programming for Classical and Hybrid Control Systems

Intelligent Optimal Control of Constrained Nonlinear Systems Via Receding-Horizon Heuristic Dynamic Programming

Optimal control for a class of hybrid systems in manufacturing

Optimization-Based Design and Control of Dynamic Systems

A "Hybrid" Approach for Synthesizing Optimal Controllers of Hybrid Systems: A Case Study of the Oil Pump Industrial Example

Costate prediction based optimal control for non-linear hybrid systems.

Dynamic Energy Management for Hybrid Electric Vehicle Based on Adaptive Dynamic Programming

Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems