Robotic hand augmentation drives changes in neural body representation

Paulina Kieliba,Danielle Clode,Roni O. Maimon-Mor,Tamar R. Makin

DOI: https://doi.org/10.1126/scirobotics.abd7935

IF: 25

2021-05-12

Science Robotics

Abstract:Skillful Third Thumb hand augmentation can be readily learned but may influence user’s body representation and motor control.

robotics

What problem does this paper attempt to address?

Enhancing human bodies with extra robotic arms and fingers: The Neural Resource Allocation Problem

Giulia Dominijanni,Solaiman Shokur,Gionata Salvietti,Sarah Buehler,Erica Palmerini,Simone Rossi,Frederique De Vignemont,Andrea D'Avella,Tamar R. Makin,Domenico Prattichizzo,Silvestro Micera

DOI: https://doi.org/10.48550/arXiv.2103.17252

IF: 3.7

2021-03-31

Robotics

Abstract:The emergence of robot-based body augmentation promises exciting innovations that will inform robotics, human-machine interaction, and wearable electronics. Even though augmentative devices like extra robotic arms and fingers in many ways build on restorative technologies, they introduce unique challenges for bidirectional human-machine collaboration. Can humans adapt and learn to operate a new limb collaboratively with their biological limbs without sacrificing their physical abilities? To successfully achieve robotic body augmentation, we need to ensure that by giving a person an additional (artificial) limb, we are not in fact trading off an existing (biological) one. In this manuscript, we introduce the "Neural Resource Allocation" problem, which distinguishes body augmentation from existing robotics paradigms such as teleoperation and prosthetics. We discuss how to allow the effective and effortless voluntary control of augmentative devices without compromising the voluntary control of the biological body. In reviewing the relevant literature on extra robotic fingers and limbs we critically assess the range of potential solutions available for the "Neural Resource Allocation" problem. For this purpose, we combine multiple perspectives from engineering and neuroscience with considerations from human-machine interaction, sensory-motor integration, ethics and law. Altogether we aim to define common foundations and operating principles for the successful implementation of motor augmentation.
The Supernumerary Robotic 3rd Thumb for Skilled Music Tasks

James Cunningham,Anita Hapsari,Pierre Guilleminot,Ali Shafti,A. Aldo Faisal

DOI: https://doi.org/10.48550/arXiv.1807.08274

2018-08-01

Abstract:Wearable robotics bring the opportunity to augment human capability and performance, be it through prosthetics, exoskeletons, or supernumerary robotic limbs. The latter concept allows enhancing human performance and assisting them in daily tasks. An important research question is, however, whether the use of such devices can lead to their eventual cognitive embodiment, allowing the user to adapt to them and use them seamlessly as any other limb of their own. This paper describes the creation of a platform to investigate this. Our supernumerary robotic 3rd thumb was created to augment piano playing, allowing a pianist to press piano keys beyond their natural hand-span; thus leading to functional augmentation of their skills and the technical feasibility to play with 11 fingers. The robotic finger employs sensors, motors, and a human interfacing algorithm to control its movement in real-time. A proof of concept validation experiment has been conducted to show the effectiveness of the robotic finger in playing musical pieces on a grand piano, showing that naive users were able to use it for 11 finger play within a few hours.

Robotics
Robotic hand illusion with tactile feedback: Unravelling the relative contribution of visuotactile and visuomotor input to the representation of body parts in space

The Vu Huynh,Robin Bekrater-Bodmann,Jakob Fröhner,Joachim Vogt,Philipp Beckerle

DOI: https://doi.org/10.1371/journal.pone.0210058

IF: 3.7

2019-01-23

PLoS ONE

Abstract:The rubber hand illusion describes a phenomenon in which participants experience a rubber hand as being part of their body by the synchronous application of visuotactile stimulation to the real and the artificial limb. In the recently introduced robotic hand illusion (RobHI), a robotic hand is incorporated into one's body representation due to the integration of synchronous visuomotor information. However, there are no setups so far that combine visuotactile and visuomotor feedback, which is expected to unravel mechanisms that cannot be detected in experimental designs applying this information in isolation. We developed a robotic hand, controlled by a sensor glove and equipped with pressure sensors, and varied systematically and separately the synchrony for motor feedback (MF) and tactile feedback (TF). In Experiment 1, we implemented a ball-grasping task and assessed the perceived proprioceptive drift of one's own hand as a behavioral measure of the spatial calibration of body coordinates as well as explicit embodiment experiences by a questionnaire. Results revealed significant main effects of both MF and TF for proprioceptive drift data, but we only observed main effects for MF on perceived embodiment. Furthermore, for the proprioceptive drift we found that synchronous feedback in one factor compensates for asynchronous feedback in the other. In Experiment 2, including a new sample of naïve participants, we further explored this finding by adding unimodal conditions, in which we manipulated the presence or absence of MF and/or TF. These findings replicated the results from Experiment 1 and we further found evidence for a supper-additive multisensory effect on spatial body representation caused by the presence of both factors. Results on conscious body perception were less consistent across both experiments. The findings indicate that sensory and motor input equally contribute to the representation of spatial body coordinates which for their part are subject to multisensory enhancing effects. The results outline the potential of human-in-the-loop approaches and might have important implications for clinical applications such as for the future design of robotic prostheses.

multidisciplinary sciences
Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Ang Ke,Jian Huang,Jing Wang,Jiping He

DOI: https://doi.org/10.3389/fnbot.2022.853773

IF: 3.493

2022-06-08

Frontiers in Neurorobotics

Abstract:Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the "hardware" and "algorithm" components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.

robotics,computer science, artificial intelligence,neurosciences
Embodied Manipulation with Past and Future Morphologies through an Open Parametric Hand Design

Kieran Gilday,Chapa Sirithunge,Fumiya Iida,Josie Hughes

2024-10-24

Abstract:A human-shaped robotic hand offers unparalleled versatility and fine motor skills, enabling it to perform a broad spectrum of tasks with precision, power and robustness. Across the paleontological record and animal kingdom we see a wide range of alternative hand and actuation designs. Understanding the morphological design space and the resulting emergent behaviors can not only aid our understanding of dexterous manipulation and its evolution, but also assist design optimization, achieving, and eventually surpassing human capabilities. Exploration of hand embodiment has to date been limited by inaccessibility of customizable hands in the real-world, and by the reality gap in simulation of complex interactions. We introduce an open parametric design which integrates techniques for simplified customization, fabrication, and control with design features to maximize behavioral diversity. Non-linear rolling joints, anatomical tendon routing, and a low degree-of-freedom, modulating, actuation system, enable rapid production of single-piece 3D printable hands without compromising dexterous behaviors. To demonstrate this, we evaluated the design's low-level behavior range and stability, showing variable stiffness over two orders of magnitude. Additionally, we fabricated three hand designs: human, mirrored human with two thumbs, and aye-aye hands. Manipulation tests evaluate the variation in each hand's proficiency at handling diverse objects, and demonstrate emergent behaviors unique to each design. Overall, we shed light on new possible designs for robotic hands, provide a design space to compare and contrast different hand morphologies and structures, and share a practical and open-source design for exploring embodied manipulation.

Robotics
Evaluating initial usability of a hand augmentation device across a large and diverse sample

Dani Clode,Lucy Dowdall,Edmund da Silva,Klara Selén,Dorothy Cowie,Giulia Dominijanni,Tamar R. Makin

DOI: https://doi.org/10.1126/scirobotics.adk5183

IF: 25

2024-05-31

Science Robotics

Abstract:The advancement of motor augmentation and the broader domain of human-machine interaction rely on a seamless integration with users' physical and cognitive capabilities. These considerations may markedly fluctuate among individuals on the basis of their age, form, and abilities. There is a need to develop a standard for considering these diversity needs and preferences to guide technological development, and large-scale testing can provide us with evidence for such considerations. Public engagement events provide an important opportunity to build a bidirectional discourse with potential users for the codevelopment of inclusive and accessible technologies. We exhibited the Third Thumb, a hand augmentation device, at a public engagement event and tested participants from the general public, who are often not involved in such early technological development of wearable robotic technology. We focused on wearability (fit and control), ability to successfully operate the device, and ability levels across diversity factors relevant for physical technologies (gender, handedness, and age). Our inclusive design was successful in 99.3% of our diverse sample of 596 individuals tested (age range from 3 to 96 years). Ninety-eight percent of participants were further able to successfully manipulate objects using the extra thumb during the first minute of use, with no significant influences of gender, handedness, or affinity for hobbies involving the hands. Performance was generally poorer among younger children (aged ≤11 years). Although older and younger adults performed the task comparably, we identified age costs with the older adults. Our findings offer tangible demonstration of the initial usability of the Third Thumb for a broad demographic.

robotics
Neural signatures of motor imagery for a supernumerary thumb in VR: an EEG analysis

Haneen Alsuradi,Joseph Hong,Alireza Sarmadi,Robert Volcic,Hanan Salam,S Farokh Atashzar,Farshad Khorrami,Mohamad Eid

DOI: https://doi.org/10.1038/s41598-024-72358-3

2024-09-16

Abstract:Human movement augmentation is a rising field of research. A promising control strategy for augmented effectors involves utilizing electroencephalography through motor imagery (MI) functions. However, performing MI of a supernumerary effector is challenging, to which MI training is one potential solution. In this study, we investigate the validity of a virtual reality (VR) environment as a medium for eliciting MI neural activations for a supernumerary thumb. Specifically, we assess whether it is possible to induce a distinct neural signature for MI of a supernumerary thumb in VR. Twenty participants underwent a two-fold experiment in which they observed movements of natural and supernumerary thumbs, then engaged in MI of the observed movements. Spectral power and event related desynchronization (ERD) analyses at the group level showed that the MI signature associated with the supernumerary thumb was indeed distinct, significantly different from both the baseline and the MI signature associated with the natural thumb, while single-trial classification showed that it is distinguishable with a 78% and 69% classification accuracy, respectively. Furthermore, spectral power and ERD analyses at the group level showed that the MI signatures associated with directional movement of the supernumerary thumb, flexion and extension, were also significantly different, and single-trial classification demonstrated that these movements could be distinguished with 60% accuracy. Fine-tuning the models further increased the respective classification accuracies, indicating the potential presence of personalized features across subjects.
NeuralFeels with neural fields: Visuotactile perception for in-hand manipulation.

Sudharshan Suresh,Tingfan Wu,Luis Pineda,Mrinal Kalakrishnan,Mustafa Mukadam,Roberto Calandra,Joseph Ortiz,Mike Lambeta,Taosha Fan,Haozhi Qi,Michael Kaess,Jitendra Malik

DOI: https://doi.org/10.1126/scirobotics.adl0628

IF: 25

2024-11-13

Science Robotics

Abstract:To achieve human-level dexterity, robots must infer spatial awareness from multimodal sensing to reason over contact interactions. During in-hand manipulation of novel objects, such spatial awareness involves estimating the object's pose and shape. The status quo for in-hand perception primarily uses vision and is restricted to tracking a priori known objects. Moreover, visual occlusion of objects in hand is imminent during manipulation, preventing current systems from pushing beyond tasks without occlusion. We combined vision and touch sensing on a multifingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. We studied multimodal in-hand perception in simulation and the real world, interacting with different objects via a proprioception-driven policy. Our experiments showed final reconstruction F scores of 81% and average pose drifts of 4.7 millimeters, which was further reduced to 2.3 millimeters with known object models. In addition, we observed that, under heavy visual occlusion, we could achieve improvements in tracking up to 94% compared with vision-only methods. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation. We release our evaluation dataset of 70 experiments, FeelSight, as a step toward benchmarking in this domain. Our neural representation driven by multimodal sensing can serve as a perception backbone toward advancing robot dexterity.

Medicine,Computer Science,Engineering
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

Sudharshan Suresh,Haozhi Qi,Tingfan Wu,Taosha Fan,Luis Pineda,Mike Lambeta,Jitendra Malik,Mrinal Kalakrishnan,Roberto Calandra,Michael Kaess,Joseph Ortiz,Mustafa Mukadam

2023-12-21

Abstract:To achieve human-level dexterity, robots must infer spatial awareness from multimodal sensing to reason over contact interactions. During in-hand manipulation of novel objects, such spatial awareness involves estimating the object's pose and shape. The status quo for in-hand perception primarily employs vision, and restricts to tracking a priori known objects. Moreover, visual occlusion of objects in-hand is imminent during manipulation, preventing current systems to push beyond tasks without occlusion. We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. We study multimodal in-hand perception in simulation and the real-world, interacting with different objects via a proprioception-driven policy. Our experiments show final reconstruction F-scores of $81$% and average pose drifts of $4.7\,\text{mm}$, further reduced to $2.3\,\text{mm}$ with known CAD models. Additionally, we observe that under heavy visual occlusion we can achieve up to $94$% improvements in tracking compared to vision-only methods. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation. We release our evaluation dataset of 70 experiments, FeelSight, as a step towards benchmarking in this domain. Our neural representation driven by multimodal sensing can serve as a perception backbone towards advancing robot dexterity. Videos can be found on our project website <a class="link-external link-https" href="https://suddhu.github.io/neural-feels/" rel="external noopener nofollow">this https URL</a>

Robotics,Computer Vision and Pattern Recognition,Machine Learning
A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Yinlin Li,Peng Wang,Rui Li,Mo Tao,Zhiyong Liu,Hong Qiao

DOI: https://doi.org/10.3389/fnbot.2022.843267

IF: 3.493

2022-04-27

Frontiers in Neurorobotics

Abstract:Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.

robotics,computer science, artificial intelligence,neurosciences
Robust neural decoding for dexterous control of robotic hand kinematics

Jiahao Fan,Luis Vargas,Derek G Kamper,Xiaogang Hu

DOI: https://doi.org/10.1016/j.compbiomed.2023.107139

Abstract:Background: Manual dexterity is a fundamental motor skill that allows us to perform complex daily tasks. Neuromuscular injuries, however, can lead to the loss of hand dexterity. Although numerous advanced assistive robotic hands have been developed, we still lack dexterous and continuous control of multiple degrees of freedom in real-time. In this study, we developed an efficient and robust neural decoding approach that can continuously decode intended finger dynamic movements for real-time control of a prosthetic hand. Methods: High-density electromyogram (HD-EMG) signals were obtained from the extrinsic finger flexor and extensor muscles, while participants performed either single-finger or multi-finger flexion-extension movements. We implemented a deep learning-based neural network approach to learn the mapping from HD-EMG features to finger-specific population motoneuron firing frequency (i.e., neural-drive signals). The neural-drive signals reflected motor commands specific to individual fingers. The predicted neural-drive signals were then used to continuously control the fingers (index, middle, and ring) of a prosthetic hand in real-time. Results: Our developed neural-drive decoder could consistently and accurately predict joint angles with significantly lower prediction errors across single-finger and multi-finger tasks, compared with a deep learning model directly trained on finger force signals and the conventional EMG-amplitude estimate. The decoder performance was stable over time and was robust to variations of the EMG signals. The decoder also demonstrated a substantially better finger separation with minimal predicted error of joint angle in the unintended fingers. Conclusions: This neural decoding technique offers a novel and efficient neural-machine interface that can consistently predict robotic finger kinematics with high accuracy, which can enable dexterous control of assistive robotic hands.
Tactile Model O: Fabrication and testing of a 3d-printed, three-fingered tactile robot hand

Jasper W. James,Alex Church,Luke Cramphorn,Nathan F. Lepora

DOI: https://doi.org/10.48550/arXiv.1907.07535

2020-08-14

Abstract:Bringing tactile sensation to robotic hands will allow for more effective grasping, along with the wide range of benefits of human-like touch. Here we present a 3D-printed, three-fingered tactile robot hand comprising an OpenHand Model O customized to house a TacTip soft biomimetic tactile sensor in the distal phalanx of each finger. We expect that combining the grasping capabilities of this underactuated hand with sophisticated tactile sensing will result in an effective platform for robot hand research -- the Tactile Model O (T-MO). The design uses three JeVois machine vision systems, each comprising a miniature camera in the tactile fingertip with a processing module in the base of the hand. To evaluate the capabilities of the T-MO, we benchmark its grasping performance using the Gripper Assessment Benchmark on the YCB object set. Tactile sensing capabilities are evaluated by performing tactile object classification on 26 objects and predicting whether a grasp will successfully lift each object. Results are consistent with the state of the art, taking advantage of advances in deep learning applied to tactile image outputs. Overall, this work demonstrates that the T-MO is an effective platform for robot hand research and we expect it to open-up a range of applications in autonomous object handling. Supplemental video: <a class="link-external link-https" href="https://youtu.be/RTcCpgffCrQ" rel="external noopener nofollow">this https URL</a>.

Robotics
Adaptive Grasp Control through Multi-Modal Interactions for Assistive Prosthetic Devices

Michelle Esponda,Thomas M. Howard

DOI: https://doi.org/10.48550/arXiv.1810.07899

2018-10-18

Abstract:The hand is one of the most complex and important parts of the human body. The dexterity provided by its multiple degrees of freedom enables us to perform many of the tasks of daily living which involve grasping and manipulating objects of interest. Contemporary prosthetic devices for people with transradial amputations or wrist disarticulation vary in complexity, from passive prosthetics to complex devices that are body or electrically driven. One of the important challenges in developing smart prosthetic hands is to create devices which are able to mimic all activities that a person might perform and address the needs of a wide variety of users. The approach explored here is to develop algorithms that permit a device to adapt its behavior to the preferences of the operator through interactions with the wearer. This device uses multiple sensing modalities including muscle activity from a myoelectric armband, visual information from an on-board camera, tactile input through a touchscreen interface, and speech input from an embedded microphone. Presented within this paper are the design, software and controls of a platform used to evaluate this architecture as well as results from experiments deigned to quantify the performance.

Robotics
Sensory- and Action-Oriented Embodiment of Neurally-Interfaced Robotic Hand Prostheses

Giovanni Di Pino,Daniele Romano,Chiara Spaccasassi,Alessandro Mioli,Marco D’Alonzo,Rinaldo Sacchetti,Eugenio Guglielmelli,Loredana Zollo,Vincenzo Di Lazzaro,Vincenzo Denaro,Angelo Maravita

DOI: https://doi.org/10.3389/fnins.2020.00389

IF: 4.3

2020-05-07

Frontiers in Neuroscience

Abstract:Embodiment is the percept that something not originally belonging to the self becomes part of the body. Feeling embodiment for a prosthesis may counteract amputees' altered image of the body and increase prosthesis acceptability. Prosthesis embodiment has been studied longitudinally in an amputee receiving feedback through intraneural and perineural multichannel electrodes implanted in her stump. Three factors—invasive (vs non-invasive) stimulation, training, and anthropomorphism—have been tested through two multisensory integration tasks: visuo-tactile integration (VTI) and crossing-hand effect in temporal order judgment (TOJ), the former more sensible to an extension of a safe margin around the body and the latter to action-oriented remapping. Results from the amputee participant were compared with the ones from healthy controls. Testing the participant with intraneural stimulation produced an extension of peripersonal space, a sign of prosthesis embodiment. One-month training extended the peripersonal space selectively on the side wearing the prostheses. More and less-anthropomorphic prostheses benefited of intraneural feedback and extended the peripersonal space. However, the worsening of TOJ performance following arm crossing was present only wearing the more trained, despite less anthropomorphic, prosthesis, suggesting that training was critical for our participant to achieve operative tool-like embodiment.

neurosciences
An Anthropomorphic Design Guideline for the Thumb of the Dexterous Hand

Hairong Wang,Shaowei Fan,Hong Liu

DOI: https://doi.org/10.1109/icma.2012.6283241

2012-01-01

Abstract:The preeminent manipulation capabilities of the human hand are undoubtedly related to the thumb. In view of this, this paper presents an anthropomorphic design guideline for the thumb of the dexterous hand. Anatomy analysis of the human thumb represents an excellent base from which a robotic design can be inferred. With simplification of human thumb's joint motion model, five configurations of thumb are given for the design of dexterous robotic hand. A new method for determining the position of thumb on the palm is proposed, according to the Euler Theorem about Finite Rotation of Rigid Body. A numeric method for computing the opposability of the thumb is described based on Monte Carlo methods, to evaluate dexterity of the hand in object manipulation. In order to verify the effectiveness of the proposed method, it is applied to the revised design of the HIT/DLR II hand. Performance indexes of both the original and the revised hand are calculated for comparison. The opposability of the thumb on revised dexterous robotic hand is 2.92 times of the original, which indicates the effectiveness of the proposed method.
Voluntary phantom hand and finger movements in transhumeral amputees could be used to naturally control polydigital prostheses

Nathanael Jarrasse,Caroline Nicol,Florian Richer,Amelie Touillet,Noël Martinet,Jean Paysant,Jozina de Graaf

DOI: https://doi.org/10.48550/arXiv.1904.11702

2019-04-26

Neurons and Cognition

Abstract:An arm amputation is extremely invalidating since many of our daily tasks require bi-manual and precise control of hand movements. Perfect hand prostheses should therefore offer a natural, intuitive and cognitively simple control over their numerous biomimetic active degrees of freedom. While efficient polydigital prostheses are commercially available, their control remains complex to master and offers limited possibilities, especially for high amputation levels. In this pilot study, we demonstrate the possibility for upper-arm amputees to intuitively control a polydigital hand prosthesis by using surface myoelectric activities of residual limb muscles (sEMG) associated with phantom limb movements, even if these residual arm muscles on which the phantom activity is measured were not naturally associated with hand movements before amputation. Using pattern recognition methods, three arm amputees were able, without training, to initiate 5-8 movements of a robotic hand (including individual finger movements) by simply mobilizing their phantom limb while the robotic hand was mimicking the action in real time. This innovative control approach could offer to numerous upper-limb amputees an access to recent biomimetic prostheses with multiple controllable joints, without requiring surgery or complex training; and might deeply change the way the phantom limb is apprehended by both patients and clinicians.
Optimal Control and Grasping for a Robotic Hand with a Non-linked Double Tendon Arrangement

Erick J. Sánchez-Garnica,Liliam Rodríguez-Guerrero,Rocío Ortega-Palacios,Omar Jacobo Santos-Sánchez

DOI: https://doi.org/10.1109/tla.2024.10431547

IF: 0.967

2024-02-13

IEEE Latin America Transactions

Abstract:After comparing different robotic hand projects, a problem is identified: when a finger has a degree of freedom, the hand is unable to grasp irregularly shaped objects. This article proposes a solution. The use of a non-linked doubletendon arrangement in the fingers allows them to have free movement; coupled with the use of Inertial Measurement Units to determine its position, ensures that, despite having one degree of freedom per finger, the hand can effectively grasp irregular objects. Additionally, a web application is developed to control hand movements through voice commands. Finally, due to the necessity for these types of devices to be mobile, an optimal control law is used to minimize energy consumption, thereby increasing autonomy when the hand is powered by batteries. As an additional note, the conducted experiments reveal that the movement of all fingers occurs simultaneously, demonstrating that parallel multitasking programming techniques effectively fulfill that purpose.

engineering, electrical & electronic,computer science, information systems
Biomimetic versus arbitrary motor control strategies for bionic hand skill learning

Hunter R. Schone,Malcolm Udeozor,Mae Moninghoff,Beth Rispoli,James Vandersea,Blair Lock,Levi Hargrove,Tamar R. Makin,Chris I. Baker

DOI: https://doi.org/10.1038/s41562-023-01811-6

IF: 24.252

2024-03-18

Nature Human Behaviour

Abstract:A long-standing engineering ambition has been to design anthropomorphic bionic limbs: devices that look like and are controlled in the same way as the biological body (biomimetic). The untested assumption is that biomimetic motor control enhances device embodiment, learning, generalization and automaticity. To test this, we compared biomimetic and non-biomimetic control strategies for non-disabled participants when learning to control a wearable myoelectric bionic hand operated by an eight-channel electromyography pattern-recognition system. We compared motor learning across days and behavioural tasks for two training groups: biomimetic (mimicking the desired bionic hand gesture with biological hand) and arbitrary control (mapping an unrelated biological hand gesture with the desired bionic gesture). For both trained groups, training improved bionic limb control, reduced cognitive reliance and increased embodiment over the bionic hand. Biomimetic users had more intuitive and faster control early in training. Arbitrary users matched biomimetic performance later in training. Furthermore, arbitrary users showed increased generalization to a new control strategy. Collectively, our findings suggest that biomimetic and arbitrary control strategies provide different benefits. The optimal strategy is probably not strictly biomimetic, but rather a flexible strategy within the biomimetic-to-arbitrary spectrum, depending on the user, available training opportunities and user requirements.

psychology, experimental,neurosciences,multidisciplinary sciences
Biohybrid Robotic Hand to Investigate Tactile Encoding and Sensorimotor Integration

Craig Ades,Moaed A Abd,Douglas T Hutchinson,Emmanuelle Tognoli,E Du,Jianning Wei,Erik D Engeberg

DOI: https://doi.org/10.3390/biomimetics9020078

2024-01-27

Abstract:For people who have experienced a spinal cord injury or an amputation, the recovery of sensation and motor control could be incomplete despite noteworthy advances with invasive neural interfaces. Our objective is to explore the feasibility of a novel biohybrid robotic hand model to investigate aspects of tactile sensation and sensorimotor integration with a pre-clinical research platform. Our new biohybrid model couples an artificial hand with biological neural networks (BNN) cultured in a multichannel microelectrode array (MEA). We decoded neural activity to control a finger of the artificial hand that was outfitted with a tactile sensor. The fingertip sensations were encoded into rapidly adapting (RA) or slowly adapting (SA) mechanoreceptor firing patterns that were used to electrically stimulate the BNN. We classified the coherence between afferent and efferent electrodes in the MEA with a convolutional neural network (CNN) using a transfer learning approach. The BNN exhibited the capacity for functional specialization with the RA and SA patterns, represented by significantly different robotic behavior of the biohybrid hand with respect to the tactile encoding method. Furthermore, the CNN was able to distinguish between RA and SA encoding methods with 97.84% ± 0.65% accuracy when the BNN was provided tactile feedback, averaged across three days in vitro (DIV). This novel biohybrid research platform demonstrates that BNNs are sensitive to tactile encoding methods and can integrate robotic tactile sensations with the motor control of an artificial hand. This opens the possibility of using biohybrid research platforms in the future to study aspects of neural interfaces with minimal human risk.
Improving internal model strength and performance of prosthetic hands using augmented feedback

Ahmed W Shehata,Leonard F Engels,Marco Controzzi,Christian Cipriani,Erik J Scheme,Jonathon W Sensinger

DOI: https://doi.org/10.1186/s12984-018-0417-4

2018-07-31

Abstract:Background: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. Methods: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. Results: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. Conclusions: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

Robotic hand augmentation drives changes in neural body representation

Enhancing human bodies with extra robotic arms and fingers: The Neural Resource Allocation Problem

The Supernumerary Robotic 3rd Thumb for Skilled Music Tasks

Robotic hand illusion with tactile feedback: Unravelling the relative contribution of visuotactile and visuomotor input to the representation of body parts in space

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Embodied Manipulation with Past and Future Morphologies through an Open Parametric Hand Design

Evaluating initial usability of a hand augmentation device across a large and diverse sample

Neural signatures of motor imagery for a supernumerary thumb in VR: an EEG analysis

NeuralFeels with neural fields: Visuotactile perception for in-hand manipulation.

Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Robust neural decoding for dexterous control of robotic hand kinematics

Tactile Model O: Fabrication and testing of a 3d-printed, three-fingered tactile robot hand

Adaptive Grasp Control through Multi-Modal Interactions for Assistive Prosthetic Devices

Sensory- and Action-Oriented Embodiment of Neurally-Interfaced Robotic Hand Prostheses

An Anthropomorphic Design Guideline for the Thumb of the Dexterous Hand

Voluntary phantom hand and finger movements in transhumeral amputees could be used to naturally control polydigital prostheses

Optimal Control and Grasping for a Robotic Hand with a Non-linked Double Tendon Arrangement

Biomimetic versus arbitrary motor control strategies for bionic hand skill learning

Biohybrid Robotic Hand to Investigate Tactile Encoding and Sensorimotor Integration

Improving internal model strength and performance of prosthetic hands using augmented feedback