Abstract:Abstract The redundant kinematic structure of the human upper body provides sufficient dexterity in performing daily living activities, such as drinking, eating, and other manipulation tasks. Most of these Activities of Daily Living (ADL) tasks can be performed using various trajectories; however, healthy humans employ some possible trajectories to accomplish these tasks. Assistive robots can explore these Cartesian trajectories and joint angles’ trajectories from human demonstrations and utilize artificial intelligence techniques to recognize, learn, and perform ADLs using robotic arms. This paper aims to develop a platform for Assistive Robots to identify and perform ADLs through human demonstration. In this paper, two Mask Regional Convolutional Neural Networks (Mask R-CNN) were developed to predict human intention for a given task, and the Reinforcement Learning approach was implemented for optimum trajectory planning. The First Mask R-CNN network, which was used for the detection of an object, was trained on a novel dataset. This dataset comprises images captured by the research team as well as gathered images from the internet. The first Mask R-CNN is expected to achieve two main objectives. The first objective is related to structure learning, which detects an object in a cluttered environment. The second objective is to estimate parameters that maximize the weight and detection rate of diverse types of similar objects. The first Mask R-CNN tries to detect the object with its location in the image and provides a soft mask on it. The second Mask R-CNN, which tries to detect human posture, was trained on a pre-existing dataset comprising human posture images. The output of these two Mask R-CNN’s then feed-forward to recognize human intensions that can be achieved through human-pose key points. Prediction of human intention was performed if the detected object and human pose match a range of pre-stored values specific to ADLs. Once the human intention has been identified via Mask R-CNN networks, the system starts planning for an optimal trajectory. For the trajectory planning, the Q-learning approach was implemented, which uses the concept of reward and penalty while exploring the unstructured environment in order to find an optimal trajectory. Experiments were conducted to determine the level of accuracy and reliability provided by our assistive platform in predicting human actions and performing trajectory planning. Results show that our assistive platform successfully recognized human actions such as water pouring, cereal making, and opening the door, as well as finding optimized trajectories to perform those actions.

Feature Learning in Assistive Rehabilitation Robotic Systems.

Evaluating Rehabilitation Progress Using Motion Features Identified by Machine Learning

Learning Deep Features for Robotic Inference from Physical Interactions.

Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors

Improved Transfer Learning for Detecting Upper-Limb Movement Intention Using Mechanical Sensors in an Exoskeletal Rehabilitation System

Learning-Based Motion-Intention Prediction for End-Point Control of Upper-Limb-Assistive Robots

A Machine Learning Approach for Predicting Upper Limb Motion Intentions with Multimodal Data in Virtual Reality

Adaptive Learning based Upper-Limb Rehabilitation Training System with Collaborative Robot

Upper Limb Movement Recognition utilising EEG and EMG Signals for Rehabilitative Robotics

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Perception-enhancement Based Task Learning and Action Scheduling for Robotic Limb in CPS Environment

Detection of Participation and Training Task Difficulty Applied to the Multi-Sensor Systems of Rehabilitation Robots

Feature Analysis for Classification of Physical Actions Using Surface EMG Data

A Systematic Study on Electromyography-Based Hand Gesture Recognition for Assistive Robots Using Deep Learning and Machine Learning Models

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

Federated Joint Learning of Robot Networks in Stroke Rehabilitation

Creation and Evaluation of Human Models with Varied Walking Ability from Motion Capture for Assistive Device Development

Toward Hand Pattern Recognition in Assistive and Rehabilitation Robotics Using EMG and Kinematics

A Novel and Efficient Feature Extraction Method for Deep Learning Based Continuous Estimation

Robot Learning From Human Demonstration of Activities of Daily Living (ADL) Tasks

Research on the method of identifying upper and lower limb coordinated movement intentions based on surface EMG signals