A Hyper-network Based End-to-end Visual Servoing with Arbitrary Desired Poses

Hongxiang Yu,Anzhe Chen,Kechun Xu,Zhongxiang Zhou,Wei Jing,Yue Wang,Rong Xiong
DOI: https://doi.org/10.48550/arXiv.2304.08952
2023-04-18
Abstract:Recently, several works achieve end-to-end visual servoing (VS) for robotic manipulation by replacing traditional controller with differentiable neural networks, but lose the ability to servo arbitrary desired poses. This letter proposes a differentiable architecture for arbitrary pose servoing: a hyper-network based neural controller (HPN-NC). To achieve this, HPN-NC consists of a hyper net and a low-level controller, where the hyper net learns to generate the parameters of the low-level controller and the controller uses the 2D keypoints error for control like traditional image-based visual servoing (IBVS). HPN-NC can complete 6 degree of freedom visual servoing with large initial offset. Taking advantage of the fully differentiable nature of HPN-NC, we provide a three-stage training procedure to servo real world objects. With self-supervised end-to-end training, the performance of the integrated model can be further improved in unseen scenes and the amount of manual annotations can be significantly reduced.
Robotics
What problem does this paper attempt to address?