Abstract:This paper presents a new hybrid learning and control method that can tune their parameters based on reinforcement learning. In the new proposed method, nonlinear controllers are considered multi-input multi-output functions and then the functions are replaced with SNNs with reinforcement learning algorithms. Dopamine-modulated spike-timing-dependent plasticity (STDP) is used for reinforcement learning and manipulating the synaptic weights between the input and output of neuronal groups (for parameter adjustment). Details of the method are presented and some case studies are done on nonlinear controllers such as Fractional Order PID (FOPID) and Feedback Linearization. The structure and the dynamic equations for learning are presented, and the proposed algorithm is tested on robots and results are compared with other works. Moreover, to demonstrate the effectiveness of SNNFOPID, we conducted rigorous testing on a variety of systems including a two-wheel mobile robot, a double inverted pendulum, and a four-link manipulator robot. The results revealed impressively low errors of 0.01 m, 0.03 rad, and 0.03 rad for each system, respectively. The method is tested on another controller named Feedback Linearization, which provides acceptable results. Results show that the new method has better performance in terms of Integral Absolute Error (IAE) and is highly useful in hardware implementation due to its low energy consumption, high speed, and accuracy. The duration necessary for achieving full and stable proficiency in the control of various robotic systems using SNNFOPD, and SNNFL on an Asus Core i5 system within Simulink's Simscape environment is as follows: - Two-link robot manipulator with SNNFOPID: 19.85656 hours - Two-link robot manipulator with SNNFL: 0.45828 hours - Double inverted pendulum with SNNFOPID: 3.455 hours - Mobile robot with SNNFOPID: 3.71948 hours - Four-link robot manipulator with SNNFOPID: 16.6789 hours. This method can be generalized to other controllers and systems like robots.

Learning Feedforward and Recurrent Deterministic Spiking Neuron Network Feedback Controllers

Evolving Spiking Neural Networks for Nonlinear Control Problems

Non-linear motor control by local learning in spiking neural networks

A new hybrid learning control system for robots based on spiking neural networks

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Neural Learning Control for Discrete-Time Nonlinear Systems in Pure-Feedback Form

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Online Adaptation of Kernel Learning Adaptive Predictive Controller

Deterministic learning and neural control of a class of nonlinear systems toward improved performance

Learning from Adaptive Neural Control of SISO Strict-Feedback Nonlinear Systems

Learning from Adaptive Neural Dynamic Surface Control of Strict-Feedback Systems

Dynamic Learning from Neural Network‐based Control for Sampled‐data Strict‐feedback Nonlinear Systems

Pattern-based NN Control for Uncertain Pure-Feedback Nonlinear Systems.

Deterministic Learning-Based Adaptive Neural Control for Nonlinear Full-State Constrained Systems.

Learning Precise Spike Train to Spike Train Transformations in Multilayer Feedforward Neuronal Networks

Learning from adaptive neural control of electrically-driven mechanical systems

Learning from Neural Control of Strict-feedback Systems

Data-Driven Stable Neural Feedback Loop Design

Learning from neural control for a class of discrete-time nonlinear systems

Dynamic Learning from Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems

Iterative Learning Control Algorithm for Spiking Behavior of Neuron Model