Abstract:This paper proposes a comprehensive hierarchical control framework for autonomous decision-making arising in robotics and autonomous systems. In a typical hierarchical control architecture, high-level decision making is often characterised by discrete state and decision/control sets. However, a rational decision is usually affected by not only the discrete states of the autonomous system, but also the underlying continuous dynamics even the evolution of its operational environment. This paper proposes a holistic and comprehensive design process and framework for this type of challenging problems, from new modelling and design problem formulation to control design and stability analysis. It addresses the intricate interplay between traditional continuous systems dynamics utilized at the low levels for control design and discrete Markov decision processes (MDP) for facilitating high-level decision making. We model the decision making system in complex environments as a hybrid system consisting of a controlled MDP and autonomous (i.e. uncontrolled) continuous dynamics. Consequently, the new formulation is called as hybrid Markov decision process (HMDP). The design problem is formulated with a focus on ensuring both safety and optimality while taking into account the influence of both the discrete and continuous state variables of different levels. With the help of the model predictive control (MPC) concept, a decision maker design scheme is proposed for the proposed hybrid decision making model. By carefully designing key ingredients involved in this scheme, it is shown that the recursive feasibility and stability of the proposed autonomous decision making scheme are guaranteed. The proposed framework is applied to develop an autonomous lane changing system for intelligent vehicles.

Design of a Hierarchical-Type Control System Based on Smart MBD Approach and its Application to Hydraulic Excavator

Research on the Design and Control Strategy for a Flow-Coupling-based Hydraulic Hybrid Excavator

Modeling and Control of General Hydraulic Excavator for Human-in-the-loop Automation

Design and Control of Hydraulic Hybrid System for Excavators

Precision Versus Intelligence: Autonomous Supporting Pressure Balance Control for Slurry Shield Tunnel Boring Machines

Dynamic Modeling and Trajectory Tracking Control of a Novel Muck-Removal Hydraulic Manipulator

Research and Application of Intelligent Control Method of Small Type Loader Actuator

Preparation of Papers for Ifac Conferences & Symposia: Integration of Process Design and Control Using Hierarchical Control Structure

Design of Database-Driven Model Predictive Control System for Digging of an Autonomous Excavator

Hierarchical Decoupling Controller with Cylinder Separated Model of Hydraulic Manipulators for Contact Force/Motion Control

A hierarchical control framework for autonomous decision-making systems: Integrating HMDP and MPC

Modeling and Design of a Series Hydraulic Hybrid Powertrain for Compact Wheel Loaders

A Multi-mode Electronic Load Sensing Control Scheme with Power Limitation and Pressure Cut-off for Mobile Machinery

Pump/valves coordinate control of the independent metering system for mobile machinery

A novel cutterhead cascade control strategy for hybrid-driving tunnel boring machines

APPLICATION OF ADAPTIVE INTERACTION PID CONTROLLER IN HYDRAULIC SERVO SYSTEM

A Study on Trajectory Control System of Hydraulic Excavators Based on Multi-Domain Physical Model*

Sdm Hybrid Control Approach for Nonlinear Systems and Application to Power Systems

Database-driven model predictive control system for online adaptation of an autonomous excavator to environmental conditions

Conceptual Design of a Flexible Hydraulic System of the Excavator Robot

A Data-driven Hierarchical Control Structure for Systems with Uncertainty