Abstract:The manufacturing industry is currently witnessing a paradigm shift with the unprecedented adoption of industrial robots, and machine vision is a key perception technology that enables these robots to perform precise operations in unstructured environments. However, the sensitivity of conventional vision sensors to lighting conditions and high-speed motion sets a limitation on the reliability and work-rate of production lines. Neuromorphic vision is a recent technology with the potential to address the challenges of conventional vision with its high temporal resolution, low latency, and wide dynamic range. In this paper and for the first time, we propose a novel neuromorphic vision based controller for robotic machining applications to enable faster and more reliable operation, and present a complete robotic system capable of performing drilling tasks with sub-millimeter accuracy. Our proposed system localizes the target workpiece in 3D using two perception stages that we developed specifically for the asynchronous output of neuromorphic cameras. The first stage performs multi-view reconstruction for an initial estimate of the workpiece’s pose, and the second stage refines this estimate for a local region of the workpiece using circular hole detection. The robot then precisely positions the drilling end-effector and drills the target holes on the workpiece using a combined position-based and image-based visual servoing approach. The proposed solution is validated experimentally for drilling nutplate holes on workpieces placed arbitrarily in an unstructured environment with uncontrolled lighting. Experimental results prove the effectiveness of our solution with maximum positional errors of less than 0.2 mm, and demonstrate that the use of neuromorphic vision overcomes the lighting and speed limitations of conventional cameras. The findings of this paper identify neuromorphic vision as a promising technology that can expedite and robustify robotic manufacturing processes in line with the requirements of the fourth industrial revolution.

A novel vision-based multi-functional sensor for normality and position measurements in precise robotic manufacturing

Elastomer-Based Visuotactile Sensor for Normality of Robotic Manufacturing Systems

Development of a Monocular Vision System for Robotic Drilling.

Neuromorphic vision based control for the precise positioning of robotic drilling systems

A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction

Design and Evaluation of a Rapid Monolithic Manufacturing Technique for a Novel Vision-Based Tactile Sensor: C-Sight

Soft Robotic Perception System with Ultrasonic Auto-Positioning and Multimodal Sensory Intelligence

Large-scale Deployment of Vision-based Tactile Sensors on Multi-fingered Grippers

A Novel Bioinspired Neuromorphic Vision-based Tactile Sensor for Fast Tactile Perception

Visuo-Tactile Exploration of Unknown Rigid 3D Curvatures by Vision-Augmented Unified Force-Impedance Control

A novel vision-based calibration framework for industrial robotic manipulators

A NOVEL VISION-BASED TACTILE SENSOR USING PARTICLE IMAGE VELOCIMETRY FOR MULTI-MODAL OBJECT DETECTION AND FORCE SENSING

Augmented Tactile Perception of Robotic Fingers Enabled by AI‐Enhanced Triboelectric Multimodal Sensors

A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception

CompdVision: Combining Near-Field 3D Visual and Tactile Sensing Using a Compact Compound-Eye Imaging System

Mechanical Behaviour, Contact Pose Estimation, and Finite Element Analysis of Vision Based Tactile Sensors Fabricated by Molding and Direct Ink Writing: A Comparative Study

Robot Tactile Sensing: Vision Based Tactile Sensor for Force Perception

A Multi-Sensor System For Optical Precise Assembly

Subtle Vibration Sensing and Dimension Measurement with A Bio-inspired Optical Tactile Sensor

A Vision-Based Path Planning and Object Tracking Framework for 6-DOF Robotic Manipulator

Vision- and tactile-based continuous multimodal intention and attention recognition for safer physical human-robot interaction