SEKD: Self-Evolving Keypoint Detection and Description

Yafei Song,Ling Cai,Jia Li,Yonghong Tian,Mingyang Li

DOI: https://doi.org/10.48550/arxiv.2006.05077

2020-01-01

Abstract:Researchers have attempted utilizing deep neural network (DNN) to learn novellocal features from images inspired by its recent successes on a variety ofvision tasks. However, existing DNN-based algorithms have not achieved suchremarkable progress that could be partly attributed to insufficient utilizationof the interactive characters between local feature detector and descriptor. Toalleviate these difficulties, we emphasize two desired properties, i.e.,repeatability and reliability, to simultaneously summarize the inherent andinteractive characters of local feature detector and descriptor. Guided bythese properties, a self-supervised framework, namely self-evolving keypointdetection and description (SEKD), is proposed to learn an advanced localfeature model from unlabeled natural images. Additionally, to have performanceguarantees, novel training strategies have also been dedicatedly designed tominimize the gap between the learned feature and its properties. We benchmarkthe proposed method on homography estimation, relative pose estimation, andstructure-from-motion tasks. Extensive experimental results demonstrate thatthe proposed method outperforms popular hand-crafted and DNN-based methods byremarkable margins. Ablation studies also verify the effectiveness of eachcritical training strategy. We will release our code along with the trainedmodel publicly.