Abstract:This paper introduces Test-time Correction (TTC) system, a novel online 3D detection system designated for online correction of test-time errors via human feedback, to guarantee the safety of deployed autonomous driving systems. Unlike well-studied offline 3D detectors frozen at inference, TTC explores the capability of instant online error rectification. By leveraging user feedback with interactive prompts at a frame, e.g., a simple click or draw of boxes, TTC could immediately update the corresponding detection results for future streaming inputs, even though the model is deployed with fixed parameters. This enables autonomous driving systems to adapt to new scenarios immediately and decrease deployment risks reliably without additional expensive training. To achieve such TTC system, we equip existing 3D detectors with Online Adapter (OA) module, a prompt-driven query generator for online correction. At the core of OA module are visual prompts, images of missed object-of-interest for guiding the corresponding detection and subsequent tracking. Those visual prompts, belonging to missed objects through online inference, are maintained by the visual prompt buffer for continuous error correction in subsequent frames. By doing so, TTC consistently detects online missed objects and immediately lowers driving risks. It achieves reliable, versatile, and adaptive driving autonomy. Extensive experiments demonstrate significant gain on instant error rectification over pre-trained 3D detectors, even in challenging scenarios with limited labels, zero-shot detection, and adverse conditions. We hope this work would inspire the community to investigate online rectification systems for autonomous driving post-deployment. Code would be publicly shared.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the problem that 3D object detectors in autonomous driving systems cannot correct errors in real - time after deployment. Specifically, existing 3D object detectors are usually deployed after offline training. Once the model is deployed on an autonomous vehicle, it is difficult to update it online to correct new errors or adapt to new scenarios. This limitation may lead to the system's failure to identify important objects, thus causing potential safety risks, such as inappropriate lane changes, turns, or even collisions.
To solve these problems, the authors propose a new system named **Test - time Correction (TTC)**. The TTC system uses human feedback (such as clicking, drawing boxes, etc.) to achieve real - time online error correction, enabling the deployed 3D detector to immediately correct errors during the test without retraining or updating the model parameters. This not only improves the system's safety but also enhances its adaptability and flexibility, allowing it to quickly respond to new driving environments and challenges.
### Main contributions of the TTC system
1. **Real - time error correction**: The TTC system can immediately correct the errors of 3D detectors during testing through human feedback, ensuring the safety and reliability of the autonomous driving system.
2. **Visual prompt mechanism**: It introduces "Visual Prompts", that is, the target object images extracted from user feedback, to help the detector identify and track previously undetected objects.
3. **Dynamic visual prompt buffer**: A dynamic visual prompt buffer is designed to store all previously unrecognized object prompts, ensuring continuous error correction.
4. **Extensive experimental verification**: Through a large number of experiments on the nuScenes dataset, the effectiveness and robustness of the TTC system in different scenarios are verified. In particular, its performance in handling challenging tasks such as long - distance object detection, vehicle detection, zero - sample detection, and domain transfer is significantly better than that of traditional methods.
### Experimental results
The experimental results show that the TTC system can significantly improve the performance during testing on a variety of offline - trained 3D detectors. For example, when handling tasks such as long - distance object detection, vehicle detection, zero - sample detection, and domain transfer, the TTC system achieves 14.4%, 21.6%, 13.6%, and 4.7% EDS improvements respectively. These results prove the potential of the TTC system in enhancing the safety and adaptability of autonomous driving systems.
### Summary
In conclusion, this paper solves the problem that 3D detectors in autonomous driving systems cannot correct errors in real - time after deployment by introducing the TTC system, significantly improving the system's safety and adaptability. This innovation is expected to provide more reliable and flexible solutions for future autonomous driving technologies.