Abstract:For autonomous driving, it is imperative to perform various high-computation image recognition tasks with high accuracy, utilizing diverse sensors to perceive the surrounding environment. Specifically, cameras are used to perform lane detection, object detection, and segmentation, and, in the absence of lidar, tasks extend to inferring 3D information through depth estimation, 3D object detection, 3D reconstruction, and SLAM. However, accurately processing all these image recognition operations in real-time for autonomous driving under constrained hardware conditions is practically unfeasible. In this study, considering the characteristics of image recognition tasks performed by these sensors and the given hardware conditions, we investigated MTL (multi-task learning), which enables parallel execution of various image recognition tasks to maximize their processing speed, accuracy, and memory efficiency. Particularly, this study analyzes the combinations of image recognition tasks for autonomous driving and proposes the MDO (multi-task decision and optimization) algorithm, consisting of three steps, as a means for optimization. In the initial step, a MTS (multi-task set) is selected to minimize overall latency while meeting minimum accuracy requirements. Subsequently, additional training of the shared backbone and individual subnets is conducted to enhance accuracy with the predefined MTS. Finally, both the shared backbone and each subnet undergo compression while maintaining the already secured accuracy and latency performance. The experimental results indicate that integrated accuracy performance is critically important in the configuration and optimization of MTL, and this integrated accuracy is determined by the ITC (inter-task correlation). The MDO algorithm was designed to consider these characteristics and construct multi-task sets with tasks that exhibit high ITC. Furthermore, the implementation of the proposed MDO algorithm, coupled with additional SSL (semi-supervised learning) based training, resulted in a significant performance enhancement. This advancement manifested as approximately a 12% increase in object detection mAP performance, a 15% improvement in lane detection accuracy, and a 27% reduction in latency, surpassing the results of previous three-task learning techniques like YOLOP and HybridNet.

A Multi-Task Effectiveness Metric and an Adaptive Co-Training Method for Enhancing Learning Performance with Few Samples

Component Identification and Depth Estimation for Structural Images Based on Multi-Scale Task Interaction Network

Post-earthquake Inspection of High-Speed Railway Viaducts with Multi-Scale Task Interaction Deep Learning Strategy

A Deep Meta-Metric Learning Method for Few-Shot Weld Seam Visual Detection

Adept: Annotation-denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretraining

A Task-Aware Attention-Based Method for Improved Meta-Learning.

A lightweight multi-task learning network based on key area guidance for counterfeit detection

Multi-Task Consistency for Active Learning

A Dynamic Feature Interaction Framework for Multi-task Visual Perception

Joint Perceptual Learning for Enhancement and Object Detection in Underwater Scenarios

Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization

A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline

An Efficient End-to-End Multitask Network Architecture for Defect Inspection

A new tool wear condition monitoring method based on deep learning under small samples

A difficulty-aware and task-augmentation method based on meta-learning model for few-shot diabetic retinopathy classification

Optimal Configuration of Multi-Task Learning for Autonomous Driving

Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement

A Small Target Detection Method Based on Deep Learning With Considerate Feature and Effectively Expanded Sample Size

Efficiently Identifying Task Groupings for Multi-Task Learning

A Point-Based Approach to Efficient LiDAR Multi-Task Perception

A Good Data Augmentation Policy Is Not All You Need: A Multi-Task Learning Perspective