Abstract:This work introduces a novel bio-inspired reconfigurable stereo vision system for robotics, leveraging omnidirectional cameras and a novel algorithm to achieve flexible visual capabilities. Inspired by the adaptive vision of various species, our visual system addresses traditional stereo vision limitations, i.e., immutable camera alignment with narrow fields of view, by introducing a reconfigurable stereo vision system to robotics. Our key innovations include the reconfigurable stereo vision strategy that allows dynamic camera alignment, a robust depth measurement system utilizing a nonrectified geometrical method combined with a deep neural network for feature matching, and a geometrical compensation technique to enhance visual accuracy. Implemented on a metamorphic robot, this vision system demonstrates its great adaptability to various scenarios by switching its configurations of 316° monocular with 79° binocular field for fast target seeking and 242° monocular with 150° binocular field for detailed close inspection.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the hardware limitations of existing stereo vision systems, especially the narrow field - of - view (FOV) caused by the fixed arrangement of cameras, which restricts the visual performance of robots in practical applications. To overcome these problems, the authors seek inspiration from nature and propose a bio - inspired reconfigurable stereo vision system that uses omnidirectional cameras and innovative algorithms to achieve flexible visual capabilities. Specifically, this system aims to: 1. **Solve the limitations of traditional stereo vision systems**: Traditional stereo vision systems usually use two planar - arranged pinhole cameras, which results in a limited field - of - view (about 60 - 90°), limiting their applicability and flexibility in various application scenarios. 2. **Introduce a reconfigurable stereo vision strategy**: By dynamically adjusting the arrangement of cameras, this system can switch between different fields - of - view and binocular fields, thus adapting to different task requirements. For example, it can provide a monocular field - of - view of 316° and a binocular field - of - view of 79° during rapid target search, and provide a monocular field - of - view of 242° and a binocular field - of - view of 150° during detailed close - up inspection. 3. **Develop a robust depth measurement system**: This system combines an improved uncalibrated geometric method and a deep neural network for feature matching, and uses geometric compensation techniques to improve visual accuracy. These techniques enable the system to perform accurate depth measurement in the case of non - planar camera arrangements. 4. **Deploy on a deformable robot**: This system is deployed on a deformable robot (Origaker), demonstrating its adaptability and versatility in different configurations. By changing the shape of the robot, the relative positions of the cameras also change, thereby obtaining different visual characteristics. In summary, this paper aims to enhance the visual perception ability and adaptability of robots in various application scenarios by introducing a new, bio - inspired reconfigurable stereo vision system.

Bio-inspired reconfigurable stereo vision for robotics using omnidirectional cameras

Stereo Calibration and Rectification for Omnidirectional Multi-Camera Systems

Human-Computer Interaction System Based on Binocular Stereo Vision

Heterogeneous stereo: A human vision inspired method for general robotics sensing

3D Perception of Biomimetic Eye Based on Motion Vision and Stereo Vision

A Target Tracking and Location Robot System Based on Omnistereo Vision

Calibration and Implementation of a Novel Omnidirectional Vision System for Robot Perception

An active vision system for navigation in a natural environment

A Flexible Framework for Virtual Omnidirectional Vision to Improve Operator Situation Awareness

Variable Baseline and Flexible Configuration Stereo Vision Using Two Aerial Robots

Omnidirectional Stereo Vision Sensor Based on Single Camera and Catoptric System.

The Design and Development of Active Stereovision System for Mobile Robot Navigation

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

Design and implementation of an omnidirectional vision system for robot perception

Active Perception with A Monocular Camera for Multiscopic Vision

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio‐inspired Electronic Eyes and Neuromorphic Image Sensors

Vision System of Mobile Robot Combining Binocular and Depth Cameras

3D perception from binocular vision for a low cost humanoid robot NAO

Omnidirectional Stereo Video Using a Hybrid Representation.

Autonomous Aerial Robot Using Dual-Fisheye Cameras.

Research and Development of Multi-Freedom Stereovision Navigating System with Biological Vision Mechanisms