From Bacteriology to Biochemistry: Albert Jan Kluyver and Chester Werkman at Iowa State

R. Singleton

DOI: https://doi.org/10.1023/A:1004775817881

Journal of the History of Biology

Abstract:

What problem does this paper attempt to address?

Epigenetics of colorectal cancer: emerging circulating diagnostic and prognostic biomarkers.

E. Danese,M. Montagnana

DOI: https://doi.org/10.21037/atm.2017.04.45

IF: 3.616

2017-05-06

Annals of Translational Medicine

Abstract:Colorectal cancer (CRC) is one of the most common cancers worldwide and the second leading cause of cancer-related mortality in western countries. Despite the high incidence, treatment options for advanced CRC remain limited and unsuccessful, resulting in a poor prognosis. Therefore, novel accurate diagnostic, prognostic and predictive biomarkers are clearly and urgently needed to detect advanced colon polyps and early stage CRC and to identify the most effective treatments for specific CRC patients. CRC is known to develop from early premalignant lesions to full blown cancer via a multi-step process involving a series of genetic mutations that accumulate over time. Recent improvement of our understanding of CRC biology and advances in genomic technologies has led to the identification of a variety of epigenetic alterations strongly involved in cancer initiation and progression. Among the epigenetic marks implicated in CRC the most widely studied are the global DNA hypomethylation, the promoter hypermethylation and the miRNAs dysregulations. Many evidence exist that such tumour associated alterations may serve as new potential biomarkers. Moreover, due the non-invasive, objective, and potential reproducible assessment, circulating epigenetic biomarkers have reached increasing attentions in the last few years. In this review, we attempt to analyze the existing most recent literature on the role of circulating DNA methylations and miRNAs alterations in CRC diagnosis and prognosis.
Neural-Based Decentralized Adaptive Finite-Time Control for Nonlinear Large-Scale Systems With Time-Varying Output Constraints

Peihao Du,Hongjing Liang,Shiyi Zhao,Choon Ki Ahn

DOI: https://doi.org/10.1109/tsmc.2019.2918351

2021-05-01

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Abstract:This paper addresses the adaptive finite-time decentralized control problem for time-varying output-constrained nonlinear large-scale systems preceded by input saturation. The intermediate control functions designed are approximated by neural networks. Time-varying barrier Lyapunov functions are used to ensure that the system output constraints are never breached. An adaptive finite-time decentralized control scheme is devised by combining the backstepping approach with Lyapunov function theory. Under the action of the proposed approach, the system stability and desired control performance can be obtained in finite time. The feasibility of this control strategy is demonstrated by using simulation results.

English Else
Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Qinmin Yang,S. Jagannathan

DOI: https://doi.org/10.1109/acc.2007.4282709

2007-01-01

Abstract:In this paper, a novel reinforcement learning neural network (NN)-based controller, referred to adaptive critic controller, is proposed for general multi-input and multi- output affine unknown nonlinear discrete-time systems in the presence of bounded disturbances. Adaptive critic designs consist of two entities, an action network that produces optimal solution and a critic that evaluates the performance of the action network. The critic is termed adaptive as it adapts itself to output the optimal cost-to-go function and the action network is adapted simultaneously based on the information from the critic. In our online learning method, one NN is designated as the critic NN, which approximates the Bellman equation. An action NN is employed to derive the control signal to track a desired system trajectory while minimizing the cost function. Online updating weight tuning schemes for these two NNs are also derived and uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is shown. The effectiveness of the controller is evaluated on a two-link robotic arm system.
A Novel Stabilizing Control for Neural Nonlinear Systems with Time Delays by State and Dynamic Output Feedback

Mei-Qin Liu,Hui-Fang Wang

IF: 2.964

2008-01-01

International Journal of Control Automation and Systems

Abstract:A novel neural network model, termed the standard neural network model (SNNM), similar to the nominal model in linear robust control theory, is suggested to facilitate the synthesis of controllers for delayed (or non-delayed) nonlinear systems composed of neural networks. The model is composed of a linear dynamic system and a bounded static delayed (or non-delayed) nonlinear operator. Based on the global asymptotic stability analysis of SNNMs, Static state-feedback controller and dynamic output feedback controller are designed for the SNNMs to stabilize the closed-loop systems, respectively. The control design equations are shown to be a set of linear matrix inequalities (LMIs) which can be easily solved by various convex optimization algorithms to determine the control signals. Most neural-network-based nonlinear systems with time delays or without time delays can be transformed into the SNNMs for controller synthesis in a unified way. Two application examples are given where the SNNMs are employed to synthesize the feedback stabilizing controllers for an SISO nonlinear system modeled by the neural network, and for a chaotic neural network, respectively. Through these examples, it is demonstrated that the SNNM not only makes controller synthesis of neural-network-based systems much easier, but also provides a new approach to the synthesis of the controllers for the other type of nonlinear systems.
Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Qinmin Yang,S. Jagannathan

DOI: https://doi.org/10.1109/adprl.2007.368192

2007-01-01

Abstract:In this paper, a novel reinforcement learning neural network (NN)-based controller, referred to adaptive critic controller, is proposed for affine nonlinear discrete-time systems with applications to nanomanipulation. In the online NN reinforcement learning method, one NN is designated as the critic NN, which approximates the long-term cost function by assuming that the states of the nonlinear systems is available for measurement. An action NN is employed to derive an optimal control signal to track a desired system trajectory while minimizing the cost function. Online updating weight tuning schemes for these two NNs are also derived. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is shown. Nanomanipulation implies manipulating objects with nanometer size. It takes several hours to perform a simple task in the nanoscale world. To accomplish the task automatically the proposed online learning control design is evaluated for the task of nanomanipulation and verified in the simulation environment.
Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Qinmin Yang,Jonathan Blake Vance,S. Jagannathan

DOI: https://doi.org/10.1109/tsmcb.2008.926607

2008-01-01

Abstract:A nonaffine discrete-time system represented by the nonlinear autoregressive moving average with eXogenous input (NARMAX) representation with unknown nonlinear system dynamics is considered. An equivalent affinelike representation in terms of the tracking error dynamics is first obtained from the original nonaffine nonlinear discrete-time system so that reinforcement-learning-based near-optimal neural network (NN) controller can be developed. The control scheme consists of two linearly parameterized NNs. One NN is designated as the critic NN, which approximates a predefined long-term cost function, and an action NN is employed to derive a near-optimal control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. The NN weights are tuned online. By using the standard Lyapunov approach, the stability of the closed-loop system is shown. The net result is a supervised actor-critic NN controller scheme which can be applied to a general nonaffine nonlinear discrete-time system without needing the affinelike representation. Simulation results demonstrate satisfactory performance of the controller.
Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Qinmin Yang,Sarangapani Jagannathan

DOI: https://doi.org/10.1109/isic.2007.4450950

2007-01-01

Abstract:In this paper, a novel online reinforcement learning neural network (NN)-based optimal output feedback controller, referred to as adaptive critic controller, is proposed for affine nonlinear discrete-time systems, to deliver a desired tracking performance. The adaptive critic design consist of three entities, an observer to estimate the system states, an action network that produces optimal control input and a critic that evaluates the performance of the action network. The critic is termed adaptive as it adapts itself to output the optimal cost-to-go function which is based on the standard Bellman equation. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the estimation and tracking errors and weight estimates is demonstrated. The effectiveness of the controller is evaluated for the task of nanomanipulation in a simulation environment.
Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Qinmin Yang,Jagannathan, S.

DOI: https://doi.org/10.1109/CDC.2007.4434959

2007-01-01

Abstract:In this paper, a novel neural network (NN) based online reinforcement learning controller is designed for nonaffine nonlinear discrete-time systems with bounded disturbances. The nonaffine systems are represented by nonlinear auto regressive moving average with exogenous input (NARMAX) model with unknown nonlinear functions. An equivalent affine-like representation for the tracking error dynamics is developed first from the original nonaffine system. Subsequently, a reinforcement learning-based neural network (NN) controller is proposed for the affine-like nonlinear error dynamic system. The control scheme consists of two NNs. One NN is designated as the critic, which approximates a predefined long-term cost function, whereas an action NN is employed to derive a control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. Offline NN training is not required and online NN weight tuning rules are derived. By using the standard Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is demonstrated.
Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Miao Huang,Cong Liu,Xiaoqi He,Longhua Ma,Zheming Lu,Hongye Su

DOI: https://doi.org/10.1016/j.neucom.2020.03.061

IF: 6

2020-01-01

Neurocomputing

Abstract:In this work, output-feedback control problems for a class of discrete-time non-affine nonlinear systems with unknown control directions and input constraints are considered by using reinforcement learning (RL) method. Two neural networks (NNs) implement the control: 1) a critic NN that estimates a non-quadratic strategic utility function (SUF) and 2) an action NN that generates optimized control input and minimizes the SUF. The implicit function theorem is applied to obtain the optimal control law since the control is appeared in a non-affine form. For the first time, the discrete Nussbaum gain is introduced to overcome the difficulty that the control directions are unknown and a non-quadratic SUF is used to deal with the control constraints in the RL-based control. The theoretical derivation of the uniformly ultimately boundedness of the NN weights and the closed-loop output tracking error is given. And two numerical examples have been supplied to valid the proposed method.
Decentralized Adaptive Neural Inverse Optimal Control of Nonlinear Interconnected Systems

Kaixin Lu,Zhi Liu,Haoyong Yu,C. L. Philip Chen,Yun Zhang

DOI: https://doi.org/10.1109/tnnls.2022.3153360

IF: 14.255

2022-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Existing methods on decentralized optimal control of continuous-time nonlinear interconnected systems require a complicated and time-consuming iteration on finding the solution of Hamilton-Jacobi-Bellman (HJB) equations. In order to overcome this limitation, in this article, a decentralized adaptive neural inverse approach is proposed, which ensures the optimized performance but avoids solving HJB equations. Specifically, a new criterion of inverse optimal practical stabilization is proposed, based on which a new direct adaptive neural strategy and a modified tuning functions method are proposed to design a decentralized inverse optimal controller. It is proven that all the closed-loop signals are bounded and the goal of inverse optimality with respect to the cost functional is achieved. Illustrative examples validate the performance of the methods presented.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
A Neural Network Approach for Stochastic Optimal Control

Xingjian Li,Deepanshu Verma,Lars Ruthotto

DOI: https://doi.org/10.1137/23m155832x

2024-09-05

SIAM Journal of Scientific Computing

Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).
Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Yongfeng Lv,Jing Na,Qinmin Yang,Xing Wu,Yu Guo

DOI: https://doi.org/10.1080/00207179.2015.1060362

IF: 2.102

2015-01-01

International Journal of Control

Abstract:An online adaptive optimal control is proposed for continuous-time nonlinear systems with completely unknown dynamics, which is achieved by developing a novel identifier-critic-based approximate dynamic programming algorithm with a dual neural network (NN) approximation structure. First, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is employed to approximate the optimal value function. Then, the optimal control law is computed based on the information from the identifier NN and the critic NN, so that the actor NN is not needed. In particular, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simultaneously, which converge to small neighbourhoods around their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.
A Fast Finite-Time Neural Network Control of Stochastic Nonlinear Systems

Fang Wang,Zhaoyang You,Zhi Liu,C. L. Philip Chen

DOI: https://doi.org/10.1109/tnnls.2022.3143655

IF: 14.255

2022-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:This article takes a fast finite-time control of stochastic nonlinear systems into account. The presence of unknown stochastic disturbance terms makes the traditional fast finite-time control approaches unavailable. To deal with this difficulty, by establishing an auxiliary function and using Jensen's inequality, in Lemma 6, a new criterion of fast finite-time stability is first established for the uncertain stochastic system. Based on the approximation ability of neural networks (NNs), an innovative fast finite-time strategy is put forward for stochastic nonlinear systems. Furthermore, by adopting the presented fast finite-time stability criterion, the stability of the stochastic systems is confirmed. Finally, two simulations are implemented to validate the feasibility of the presented NN control strategy.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
A novel adaptive neural predefined time control for a class of nonlinear systems

Wei Jiang,Huanqing Wang,Xudong Zhao

DOI: https://doi.org/10.1002/asjc.3344

IF: 2.4

2024-02-20

Asian Journal of Control

Abstract:In this article, a novel adaptive neural predefined time control scheme is designed for a class of uncertain nonlinear systems. In the design procedure, the unknown smooth nonlinear functions are approximated by the neural networks (NNs). A neural adaptive predefined time controller is proposed based on NNs, adaptive backstepping technology, and predefined time stability theory. The designed control strategy includes a new adaptive updating law which ensures that the derivative of the Lyapunov function satisfies the predefined time stable form. The proposed control scheme ensures that all the signals in the closed‐loop system are predefined time stability. A practical simulation example validates the effectiveness of the designed controller.

automation & control systems
Adaptive Neural Networks Finite-Time Optimal Control for a Class of Nonlinear Systems

Yongming Li,Tingting Yang,Shaocheng Tong

DOI: https://doi.org/10.1109/tnnls.2019.2955438

IF: 14.255

2020-11-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:This article addresses the finite-time optimal control problem for a class of nonlinear systems whose powers are positive odd rational numbers. First of all, a finite-time controller, which is capable of ensuring the semiglobal practical finite-time stability for the closed-loop systems, is developed using the adaptive neural networks (NNs) control method, adding one power integrator technique and backstepping scheme. Second, the corresponding design parameters are optimized, and the finite-time optimal control property is obtained by means of minimizing the well-defined and designed cost function. Finally, a numerical simulation example is given to further validate the feasibility and effectiveness of the proposed optimal control strategy.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Observer-Based Adaptive Optimized Control for Stochastic Nonlinear Systems With Input and State Constraints

Yongming Li,Jiaxin Zhang,Wei Liu,Shaocheng Tong

DOI: https://doi.org/10.1109/tnnls.2021.3087796

IF: 14.255

2021-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:In this work, an adaptive neural network (NN) optimized output-feedback control problem is studied for a class of stochastic nonlinear systems with unknown nonlinear dynamics, input saturation, and state constraints. A nonlinear state observer is designed to estimate the unmeasured states, and the NNs are used to approximate the unknown nonlinear functions. Under the framework of the backstepping technique, the virtual and actual optimal controllers are developed by employing the actor–critic architecture. Meanwhile, the tan-type Barrier optimal performance index functions are developed to prevent the nonlinear systems from the state constraints, and all the states are confined within the preselected compact sets all the time. It is worth mentioning that the proposed optimized control is clearly simple since the reinforcement learning (RL) algorithm is derived based on the negative gradient of a simple positive function. Furthermore, the proposed optimal control strategy ensures that all the signals in the closed-loop system are bounded. Finally, a practical simulation example is carried out to further illustrate the effectiveness of the proposed optimal control method.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Finite-Time Adaptive Neural Control for a Class of Nonlinear Systems With Asymmetric Time-Varying Full-State Constraints

Yan Zhang,Jian Guo,Zhengrong Xiang

DOI: https://doi.org/10.1109/tnnls.2022.3164948

IF: 14.255

2022-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:In this article, an adaptive finite-time tracking control scheme is developed for a category of uncertain nonlinear systems with asymmetric time-varying full-state constraints and actuator failures. First, in the control design process, the original constrained nonlinear system is transformed into an equivalent ``unconstrained'' one by using the uniform barrier function (UBF). Then, by introducing a new coordinate transformation and incorporating it into each recursive step of adaptive finite-time control design based on the backstepping technique, more general state constraints can be handled. In addition, since the nonlinear function in the system is unknown, neural network is employed to approximate it. Considering singularity, the virtual control signal is designed as a piecewise function to guarantee the performance of the system within a finite time. The developed finite-time control method ensures that all signals in the closed-loop system are bounded, and the output tracking error converges to a small neighborhood of the origin. At last, the simulation example illustrates the feasibility and superiority of the presented control method.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
A Novel Approximate Dynamic Programming Structure for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Jiayue Sun,Zhiming Xu

DOI: https://doi.org/10.1109/tcsii.2024.3371997

2024-01-01

Abstract:In this brief, for time-varying nonlinear systems, a novel approximate dynamic programming structure is first proposed to obtain the approximate optimal solution of the control system. In addition, considering that the system is time-varying and it is difficult to obtain the system dynamics, we propose the idea of time normalization to identify the time-varying system dynamics through a model network. Based on the traditional heuristic dynamic programming structure, an action network is added to approximate the next time step action. With the increase of feedback channels, the convergence speed of the action network is accelerated. And we also prove that the weight errors of the three improved neural network structure are uniformly ultimately bounded. The effectiveness of our proposed new neural network structure is demonstrated by numerical simulation results.

engineering, electrical & electronic
Adaptive neural fault‐tolerant optimal control for nonlinear uncertain systems with dynamic state constraints

Yan Wei,Jun Fu,Huaicheng Yan,Minrui Fei,Yueying Wang

DOI: https://doi.org/10.1002/rnc.6826

IF: 3.8973

2023-06-09

International Journal of Robust and Nonlinear Control

Abstract:This article investigates the issue of adaptive neural optimal fault‐tolerant control for a class of nonlinear uncertain systems subject to dynamic state constraints and external disturbances. To handle more general dynamic constraints, a unified tangent‐type nonlinear mapping is first proposed to transform the state‐constrained system into one free of constraints. To solve the problem of actuator faults and external disturbances, a single network adaptive dynamic program method is designed, which consists of a feed‐forward fault‐tolerant control scheme and a feedback differential game control strategy. Neural networks are employed to approximate the uncertainties and cost function, respectively. To handle the issue of "explosion of complexity," a finite‐time convergent differentiator is established to estimate the derivative of virtual control signals in the backstepping design. Via Lyapunov stability analysis, asymptotic stability of the original and transformed nonlinear system is theoretically guaranteed. Two comparative simulation examples are provided to evaluate the efficacy of the proposed control approach.

automation & control systems,engineering, electrical & electronic,mathematics, applied

From Bacteriology to Biochemistry: Albert Jan Kluyver and Chester Werkman at Iowa State

Epigenetics of colorectal cancer: emerging circulating diagnostic and prognostic biomarkers.

Neural-Based Decentralized Adaptive Finite-Time Control for Nonlinear Large-Scale Systems With Time-Varying Output Constraints

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

A Novel Stabilizing Control for Neural Nonlinear Systems with Time Delays by State and Dynamic Output Feedback

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Decentralized Adaptive Neural Inverse Optimal Control of Nonlinear Interconnected Systems

A Neural Network Approach for Stochastic Optimal Control

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

A Fast Finite-Time Neural Network Control of Stochastic Nonlinear Systems

A novel adaptive neural predefined time control for a class of nonlinear systems

Adaptive Neural Networks Finite-Time Optimal Control for a Class of Nonlinear Systems

Observer-Based Adaptive Optimized Control for Stochastic Nonlinear Systems With Input and State Constraints

Finite-Time Adaptive Neural Control for a Class of Nonlinear Systems With Asymmetric Time-Varying Full-State Constraints

A Novel Approximate Dynamic Programming Structure for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Adaptive neural fault‐tolerant optimal control for nonlinear uncertain systems with dynamic state constraints