Abstract:Autonomous systems, such as self-driving cars and drones, have made significant strides in recent years by leveraging visual inputs and machine learning for decision-making and control. Despite their impressive performance, these vision-based controllers can make erroneous predictions when faced with novel or out-of-distribution inputs. Such errors can cascade into catastrophic system failures and compromise system safety. In this work, we compute Neural Reachable Tubes, which act as parameterized approximations of Backward Reachable Tubes to stress-test the vision-based controllers and mine their failure modes. The identified failures are then used to enhance the system safety through both offline and online methods. The online approach involves training a classifier as a run-time failure monitor to detect closed-loop, system-level failures, subsequently triggering a fallback controller that robustly handles these detected failures to preserve system safety. For the offline approach, we improve the original controller via incremental training using a carefully augmented failure dataset, resulting in a more robust controller that is resistant to the known failure modes. In either approach, the system is safeguarded against shortcomings that transcend the vision-based controller and pertain to the closed-loop safety of the overall system. We validate the proposed approaches on an autonomous aircraft taxiing task that involves using a vision-based controller to guide the aircraft towards the centerline of the runway. Our results show the efficacy of the proposed algorithms in identifying and handling system-level failures, outperforming methods that rely on controller prediction error or uncertainty quantification for identifying system failures.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to improve the safety and robustness of vision - based controllers (VBCs) in autonomous systems. Specifically, the paper focuses on the following aspects:
1. **Identifying visual failure modes**: Autonomous systems, such as self - driving cars and drones, have made significant progress by using visual input and machine learning for decision - making and control. However, these vision - based controllers may make incorrect predictions when faced with new or out - of - distribution inputs, and these errors may lead to catastrophic system - level failures and affect system safety. The paper proposes a method to calculate neural reachable tubes (NRTs), which is a method of parameterized approximate backward reachable tubes (BRTs), to test vision - based controllers and mine their failure modes.
2. **Enhancing system safety**: Through the identified failure modes, the paper proposes two methods, online and offline, to enhance system safety. The online method involves training a classifier as a runtime failure detector (FD) to detect closed - loop system - level failures and trigger a backup controller to handle these detected failures, thereby maintaining system safety. The offline method improves the original controller through incremental training using carefully augmented failure datasets, making the controller more robust and able to resist known failure modes.
3. **Verifying the effectiveness of the method**: The paper verifies the proposed algorithm in an autonomous aircraft taxiing task, which uses a vision - based controller to guide the aircraft along the runway centerline. The experimental results show that the proposed algorithm is superior to traditional methods that rely on controller prediction error or uncertainty quantification in identifying and handling system - level failures.
### Formula Explanation
- **State Space and Control Input**:
\[
f(x, u)=\begin{bmatrix} \dot{p_x} \\ \dot{p_y} \\ \dot{\theta} \end{bmatrix}=\begin{bmatrix} v \sin(\theta) \\ v \cos(\theta) \\ u \end{bmatrix}
\]
where \(x\) is the system state composed of \(p_x\) (lateral error), \(p_y\) (longitudinal position), and \(\theta\) (heading error), \(v\) is the linear velocity of the aircraft, and \(u\) is the control input (angular velocity).
- **Value Function and HJB Equation**:
\[
V(x, t)=\min_{\tau \in [t, T]} l(\zeta^\pi_x(\tau))
\]
\[
\min\{D_t V(x, t)+H(x, t), l(x)-V(x, t)\}=0
\]
where \(D_t\) represents the time gradient, \(H(x, t)=\langle \nabla_x V(x, t), f(x, \pi(x)) \rangle\) is the Hamiltonian, and \(l(x)\) is the distance function from the state to the failure set.
- **Loss Function**:
\[
\text{Loss}(x, d, \tau):=L_{\text{ham}}(x, d, \tau)+\lambda L_{\text{init}}(x, d)
\]
\[
L_{\text{ham}}(x, d, \tau):=\min\{D_t V_\theta(x, d, \tau)+H(x, d, \tau), l(x)-V_\theta(x, d, \tau)\}
\]
\[
L_{\text{init}}(x, d):=|V_\theta(x, d,