Abstract:The International Journal of Robotics Research, Ahead of Print. This study proposes a modularized soft robotic arm with integrated sensing of human touches for physical human–robot interactions. The proposed robotic arm is constructed by connecting multiple soft manipulator modules, each of which consists of three bellow-type soft actuators, pneumatic valves, and an on-board sensing and control circuit. By employing stereolithography three-dimensional (3D) printing technique, the bellow actuator is capable of incorporating embedded organogel channels in the thin wall of its body that are used for detecting human touches. The organogel thus serves as a soft interface for recognizing the intentions of the human operators, enabling the robot to interact with them while generating desired motions of the manipulator. In addition to the touch sensors, each manipulator module has compact, soft string sensors for detecting the displacements of the bellow actuators. When combined with an inertial measurement unit (IMU), the manipulator module has a capability of estimating its own pose or orientation internally. We also propose a localization method that allows us to estimate the location of the manipulator module and to acquire the 3D information of the target point in an uncontrolled environment. The proposed method uses only a single depth camera combined with a deep learning model and is thus much simpler than those of conventional motion capture systems that usually require multiple cameras in a controlled environment. Using the feedback information from the internal sensors and camera, we implemented closed-loop control algorithms to carry out tasks of reaching and grasping objects. The manipulator module shows structural robustness and the performance reliability over 5,000 cycles of repeated actuation. It shows a steady-state error and a standard deviation of 0.8 mm and 0.3 mm, respectively, using the proposed localization method and the string sensor data. We demonstrate an application example of human–robot interaction that uses human touches as triggers to pick up and manipulate target objects. The proposed soft robotic arm can be easily installed in a variety of human workspaces, since it has the ability to interact safely with humans, eliminating the need for strict control of the environments for visual perception. We believe that the proposed system has the potential to integrate soft robots into our daily lives.

A Vision-Based Sensing Approach for a Spherical Soft Robotic Arm

Vision-based Motion Control of a Flexible Robot for Surgical Applications

Proprioception and Exteroception of a Soft Robotic Finger Using Neuromorphic Vision-Based Sensing

Sensing Expectation Enables Simultaneous Proprioception and Contact Detection in an Intelligent Soft Continuum Robot

SoPrA: Fabrication & Dynamical Modeling of a Scalable Soft Continuum Robotic Arm with Integrated Proprioceptive Sensing

Exoskeleton-covered soft finger with vision-based proprioception and tactile sensing

Soft robot perception using embedded soft sensors and recurrent neural networks

Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment

Sensing and Control of a Multi-Joint Soft Wearable Robot for Upper-Limb Assistance and Rehabilitation

Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

Proprioceptive and Exteroceptive Information Perception in a Fabric Soft Robotic Arm via Physical Reservoir Computing with minimal training data

Vision-Based Tactile Perception for Soft Robotic Hand

A soft thumb-sized vision-based sensor with accurate all-round force perception

Sensorless Proprioception in Multi-DoF Dielectric Elastomer Soft Robots via System-Level Self-Sensing

Vision-based reinforcement learning control of soft robot manipulators

Unsupervised Sim-to-Real Adaptation of Soft Robot Proprioception using a Dual Cross-modal Autoencoder

ViSE: Vision-Based 3D Online Shape Estimation of Continuously Deformable Robots

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

Adaptive Tracking Control of Soft Robots using Integrated Sensing Skin and Recurrent Neural Networks

Soft modularized robotic arm for safe human–robot interaction based on visual and proprioceptive feedback

Self-Sensing for Proprioception and Contact Detection in Soft Robots Using Shape Memory Alloy Artificial Muscles