Abstract:This paper proposes an online predictive control method for fixed-wing unmanned aerial vehicles (UAVs) with a pan-tilt camera in target tracking. It aims to achieve long-term tracking while concurrently maintaining the target near the image center. Particularly, this work takes the UAV and pan-tilt camera as an overall system and deals with the target tracking problem via joint optimization, so that the tracking ability of the UAV can be improved. The image captured by the pan-tilt camera is the unique input associated with the target, and model predictive control (MPC) is used to solve the optimization problem with constraints that cannot be performed by the classic image-based visual servoing (IBVS). In addition to the dynamic constraint of the UAV, the perception constraint of the camera is also taken into consideration, which is described by the maximum distance between the target and the camera. The accurate detection of the target depends on the amount of its feature information contained in the image, which is highly related to the relative distance between the target and the camera. Moreover, considering the real-time requirements of practical applications, an MPC strategy based on soft constraints and a warm start is presented. Furthermore, a switching-based approach is proposed to return the target back to the perception range quickly once it exceeds the range, and the exponential asymptotic stability of the switched controller is proven as well. Both numerical and hardware-in-the-loop (HITL) simulations are conducted to verify the effectiveness and superiority of the proposed method compared with the existing method.

Monocular Vision-based Sense and Avoid of UAV Using Nonlinear Model Predictive Control

Low Altitude Sense and Avoid for Muav Based on Stereo Vision

Vision-Based Sense and Avoid with Monocular Vision and Real-Time Object Detection for UAVs

Vision-based Sense and Avoid of UAV with 2D Dynamic Safety Envelop

A Vision Based Sense and Avoid System for Small Unmanned Helicopter

A Vision-based UAV Tracker Aiming at Aerial Targets

Vision Based UAV Formation Sense and Avoid System

Monocular Vision-Based Obstacle Detection/Avoidance For Unmanned Aerial Vehicles

Perception-Aware Model Predictive Control for Target Tracking with UAVs

A detection and relative direction estimation method for UAV in sense-and-avoid

A Perception-Aware NMPC for Vision-Based Target Tracking and Collision Avoidance with a Multi-Rotor UAV

Perception-Aware Model Predictive Visual Servoing for Fixed-Wing Unmanned Aerial Vehicles in Target Tracking

Online Predictive Visual Servo Control for Constrained Target Tracking of Fixed-Wing Unmanned Aerial Vehicles

Object Detection and Obstacle Avoidance Based on Dynamic Vision Sensor for UAV

Monocular Vision-Based Dynamic Moving Obstacles Detection and Avoidance

Vision-based finite-time prescribed performance control for uncooperative UAV target-tracking subject to field of view constraints

A Cascaded and Adaptive Visual Predictive Control Approach for Real-Time Dynamic Visual Servoing

Vision-based UAV adaptive tracking control for moving targets with velocity observation

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

Intent Prediction-Driven Model Predictive Control for UAV Planning and Navigation in Dynamic Environments

Vision-Based Mid-Air Object Detection and Avoidance Approach for Small Unmanned Aerial Vehicles s with Deep Learning and Risk Assessment