Abstract:Recently, scene coordinate regression (SCoRe) approaches for visual localization have been extensively investigated. However, current SCoRe methods are scene-specific and necessitate retraining when generalizing new scenarios, leaving a consistent rise in model capacity as the number of scenes increases. To this end, we develop UFVL-Net, a unifying framework that integrates localization tasks of multiple indoor scenarios into a manageable network and optimizes these tasks collectively under diversified scene domains, where the localization of each scenario domain is considered a separate task. UFVL-Net is storage-efficient since multiple models with shared parameters can be consolidated into a single one. Specifically, we introduce two parameter sharing policies, that is, channel-wise sharing policy (CSP) and kernel-wise sharing policy, which offer fine-grained parameter sharing within each layer of the backbone for efficient storage while providing task-specific parameters to tackle the inherent hurdles associated with multidomain learning for visual localization, that is, gradient conflict due to a skewed competition among tasks for the shared parameters. The key insight lies in that leveraging task-sharing parameters can learn a generic feature representation across scenes while utilizing task-specific parameters can learn task-related features for alleviating gradient conflict. Moreover, we develop a sign-based gradient normalization (SIGGrad) technique applied to task-sharing parameters to promote the training of UFVL-Net by further mitigating gradient conflict, thus emphasizing the utilization of task-sharing parameters and ensuring that each task is thoroughly optimized. We undertake extensive experiments across numerous datasets and complex real-world scenarios, showing that UFVL-Net families significantly outperform the cutting-edge methods with much less storage space. We demonstrate that UFVL-Net can be generalized to new scenarios using a few task-specific parameters, further highlighting the superiority of UFVL-Net. The code is available at here.

A Vanilla Multi-Task Framework for Dense Visual Prediction Solution to 1st VCL Challenge -- Multi-Task Robustness Track

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Up-to-Down Network: Fusing Multi-Scale Context for 3D Semantic Scene Completion

UniVision: A Unified Framework for Vision-Centric 3D Perception

A Dynamic Feature Interaction Framework for Multi-task Visual Perception

Multi-attention Associate Prediction Network for Visual Tracking

A Multi-Task Framework for Infrared Small Target Detection and Segmentation

UniOcc: Unifying Vision-Centric 3D Occupancy Prediction with Geometric and Semantic Rendering

All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment

UAMD-Net: A Unified Adaptive Multimodal Neural Network for Dense Depth Completion

Adaptive Context-Aware Multi-Modal Network for Depth Completion

LidarMultiNet: Unifying LiDAR Semantic Segmentation, 3D Object Detection, and Panoptic Segmentation in a Single Multi-task Network

RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception

Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios

Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving

Improving Multiple Dense Prediction Performances by Exploiting Inter-Task Synergies for Neuromorphic Vision Sensors

ViT-MVT: A Unified Vision Transformer Network for Multiple Vision Tasks.

A Multi-Task Network Based on Dual-Neck Structure for Autonomous Driving Perception

UFVL-Net: A Unified Framework for Visual Localization Across Multiple Indoor Scenes

MLVSNet: Multi-level Voting Siamese Network for 3D Visual Tracking