Technical Problems in the Operative Correction of Pectus Excavatum

M. Ravitch

DOI: https://doi.org/10.1097/00000658-196507000-00004

IF: 13.787

1965-07-01

Annals of Surgery

Abstract:

What problem does this paper attempt to address?

Control of a Wheelchair-Mounted 6DOF Assistive Robot With Chin and Finger Joysticks

Ivan Rulik,Md Samiul Haque Sunny,Javier Dario Sanjuan De Caro,Md Ishrak Islam Zarif,Brahim Brahmi,Sheikh Iqbal Ahamed,Katie Schultz,Inga Wang,Tony Leheng,Jason Peng Longxiang,Mohammad H Rahman

DOI: https://doi.org/10.3389/frobt.2022.885610

2022-07-22

Abstract:Throughout the last decade, many assistive robots for people with disabilities have been developed; however, researchers have not fully utilized these robotic technologies to entirely create independent living conditions for people with disabilities, particularly in relation to activities of daily living (ADLs). An assistive system can help satisfy the demands of regular ADLs for people with disabilities. With an increasing shortage of caregivers and a growing number of individuals with impairments and the elderly, assistive robots can help meet future healthcare demands. One of the critical aspects of designing these assistive devices is to improve functional independence while providing an excellent human-machine interface. People with limited upper limb function due to stroke, spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis, and other conditions find the controls of assistive devices such as power wheelchairs difficult to use. Thus, the objective of this research was to design a multimodal control method for robotic self-assistance that could assist individuals with disabilities in performing self-care tasks on a daily basis. In this research, a control framework for two interchangeable operating modes with a finger joystick and a chin joystick is developed where joysticks seamlessly control a wheelchair and a wheelchair-mounted robotic arm. Custom circuitry was developed to complete the control architecture. A user study was conducted to test the robotic system. Ten healthy individuals agreed to perform three tasks using both (chin and finger) joysticks for a total of six tasks with 10 repetitions each. The control method has been tested rigorously, maneuvering the robot at different velocities and under varying payload (1-3.5 lb) conditions. The absolute position accuracy was experimentally found to be approximately 5 mm. The round-trip delay we observed between the commands while controlling the xArm was 4 ms. Tests performed showed that the proposed control system allowed individuals to perform some ADLs such as picking up and placing items with a completion time of less than 1 min for each task and 100% success.
Diegetic Graphical User Interfaces and Intuitive Control of Assistive Robots via Eye-gaze

Emanuel Nunez Sardinha,Marcela Munera,Nancy Zook,David Western,Virginia Ruiz Garate

2024-01-08

Abstract:Individuals with tetraplegia and similar forms of paralysis suffer physically and emotionally due to a lack of autonomy. To help regain part of this autonomy, assistive robotic arms have been shown to increase living independence. However, users with paralysis pose unique challenging conditions for the control of these devices. In this article, we present the use of Diegetic Graphical User Interfaces, a novel, intuitive, and computationally inexpensive approach for gaze-controlled interfaces applied to robots. By using symbols paired with fiducial markers, interactive buttons can be defined in the real world which the user can trigger via gaze, and which can be embedded easily into the environment. We apply this system to pilot a 3-degree-of-freedom robotic arm for precision pick-and-place tasks. The interface is placed directly on the robot to allow intuitive and direct interaction, eliminating the need for context-switching between external screens, menus, and the robot. After calibration and a brief habituation period, twenty-one participants from multiple backgrounds, ages and eye-sight conditions completed the Yale-CMU-Berkeley (YCB) Block Pick and Place Protocol to benchmark the system, achieving a mean score of 13.71 out of the maximum 16.00 points. Good usability and user experience were reported (System Usability Score of 75.36) while achieving a low task workload measure (NASA-TLX of 44.76). Results show that users can employ multiple interface elements to perform actions with minimal practice and with a small cognitive load. To our knowledge, this is the first easily reconfigurable screenless system that enables robot control entirely via gaze for Cartesian robot control without the need for eye or face gestures.

Robotics,Systems and Control
Development of a Human-Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM

Jianlong Li,Zhijun Li,Ying Feng,Yiliang Liu,Guangming Shi

DOI: https://doi.org/10.1109/tase.2019.2911667

IF: 6.636

2019-01-01

IEEE Transactions on Automation Science and Engineering

Abstract:To achieve the better navigation performance of a mobile robot in the unknown environments, a novel human–robot hybrid system incorporating a motor-imagery (MI)-based brain teleoperation control is presented in this paper, where a deep-learning-based active perception is developed in the simultaneous localization and mapping (SLAM) framework. Using the deep-learning-based object recognition in the red–green–blue-depth (RGB-D) data acquisition process, the designed SLAM approach can select the valid feature points effectively, and the speed of displacement tracking can be improved by combining the oriented FAST and rotated BRIEF (ORB) SLAM algorithm with the optical flow method. The global trajectory map can also be mended using graph-based nonlinear error optimization. In addition, to build the connection between human intentions and the robot control commands flexibly in the developed mobile robot, a common spatial pattern (CSP)-based support vector machine (SVM) classification algorithm is proposed so that the control commands can be obtained directly from the human electroencephalograph (EEG) signals, which are preanalyzed and classified using the phenomena of event-related synchronization/desynchronization (ERS/ERD). Experiments involving several operators have verified the effectiveness of the proposed framework in the actual unstructured environments. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</italic> —This paper is motivated by the exploration of the mobile robots remotely controlled by the disables in an unstructured environment that includes some unknown moving objects in the background. In the conventional approaches, the image feedback and the brain–computer interface-based EEG signals invoked by the visual stimulus are used to guide the motion of the robots. In this paper, the MI of left or right hand is adopted as an input of the brain teleoperation system, and the EEG signals are classified into two categories representing two robot control commands through the designed algorithm. Besides, a deep-learning-based SLAM is designed to analyze the image and depth of the environmental information provided by the robot RGB-D sensor, and the results can be transferred into a 3-D map locating the robot and assisting the operator to understand the operating environment. It has been verified that the deep-learning-based SLAM presented is more efficient and robust than traditional SLAM. The feasibility of the system is demonstrated by a set of experiments in the corridor environment.
An Intelligent and Low-cost Eye-tracking System for Motorized Wheelchair Control

Mahmoud Dahmani,Muhammad E. H. Chowdhury,Amith Khandakar,Tawsifur Rahman,Khaled Al-Jayyousi,Abdalla Hefny,Serkan Kiranyaz

DOI: https://doi.org/10.3390/s20143936

2020-05-03

Abstract:In the 34 developed and 156 developing countries, there are about 132 million disabled people who need a wheelchair constituting 1.86% of the world population. Moreover, there are millions of people suffering from diseases related to motor disabilities, which cause inability to produce controlled movement in any of the limbs or even <a class="link-external link-http" href="http://head.The" rel="external noopener nofollow">this http URL</a> paper proposes a system to aid people with motor disabilities by restoring their ability to move effectively and effortlessly without having to rely on others utilizing an eye-controlled electric wheelchair. The system input was images of the users eye that were processed to estimate the gaze direction and the wheelchair was moved accordingly. To accomplish such a feat, four user-specific methods were developed, implemented and tested; all of which were based on a benchmark database created by the <a class="link-external link-http" href="http://authors.The" rel="external noopener nofollow">this http URL</a> first three techniques were automatic, employ correlation and were variants of template matching, while the last one uses convolutional neural networks (CNNs). Different metrics to quantitatively evaluate the performance of each algorithm in terms of accuracy and latency were computed and overall comparison is presented. CNN exhibited the best performance (i.e. 99.3% classification accuracy), and thus it was the model of choice for the gaze estimator, which commands the wheelchair motion. The system was evaluated carefully on 8 subjects achieving 99% accuracy in changing illumination conditions outdoor and indoor. This required modifying a motorized wheelchair to adapt it to the predictions output by the gaze estimation algorithm. The wheelchair control can bypass any decision made by the gaze estimator and immediately halt its motion with the help of an array of proximity sensors, if the measured distance goes below a well-defined safety margin.

Human-Computer Interaction,Computer Vision and Pattern Recognition
A Portable Wireless Eye Movement-Controlled Human-Computer Interface for the Disabled

Xiaoxiang Zheng,Xin Li,Jun Liu,Weidong Chen,Yaoyao Hao

DOI: https://doi.org/10.1109/iccme.2009.4906647

2009-01-01

Abstract:Human-Computer Interface (HCI) has become an important area of research and development for the disabled. A portable wireless eye movement-controlled Human-Computer Interface which can be used for the disabled who have motor paralysis and who cannot speak in multiple applications (such as communication aid and smart home applications) is described here. This Interface consists of four major parts: (1) surface electrodes, (2) a two-channel amplifier, (3) a laptop (or a micro-processor), and (4) a ZigBee wireless module. Horizontal and vertical Electro-Oculography (EOG) signals are measured using five surface electrodes placed on the head .The vertical electrodes are placed about 1.0 cm above the right eyebrow and 2.0 cm below the lower lid of the right eye, the horizontal electrodes are placed 2.0 cm lateral to the each side of outer canthi and the last electrode is placed on user's forehead to serve as a ground. The two-channel amplifier is comprised of instrumentation amplifiers, band-pass filters and shift circuits. The EOG signals are sampled at the rate of 250Hz and then sent to a laptop or a micro-processor for signal processing which is based on the method of mathematical morphology to recognize the direction of eye movements and voluntary eye blink. The ZigBee wireless communication technology, which is proved to be reliable, low-power and cost-efficient, is used in the portable interface. The subjects can control the wireless device or move a cursor over a screen by using this interface. The delay of this interface is less than 0.5s and errors are very limited. This interface provides a flexible method for the disabled to improve the life quality.
Eye Gaze Controlled Robotic Arm for Persons with SSMI

Vinay Krishna Sharma,L. R. D. Murthy,KamalPreet Singh Saluja,Vimal Mollyn,Gourav Sharma,Pradipta Biswas,L.R.D. Murthy

DOI: https://doi.org/10.48550/arXiv.2005.11994

IF: 6.4588

2020-05-25

Human-Computer Interaction

Abstract:Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker. Objective: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye gaze tracker is used to develop a video see through based AR (augmented reality) display that controls a robotic device with eye gaze and deployed for a fabric printing task. Methodology: We undertook a user centred design process and separately evaluated the web cam based gaze tracker and the video see through based human robot interaction involving users with SSMI. We also reported a user study on manipulating a robotic arm with webcam based eye gaze tracker. Results: Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker.
User Evaluation of a Shared Robot Control System Combining BCI and Eye Tracking in a Portable Augmented Reality User Interface

Arnau Dillen,Mohsen Omidi,Fakhreddine Ghaffari,Olivier Romain,Bram Vanderborght,Bart Roelands,Ann Nowé,Kevin De Pauw

DOI: https://doi.org/10.3390/s24165253

2024-08-14

Abstract:This study evaluates an innovative control approach to assistive robotics by integrating brain-computer interface (BCI) technology and eye tracking into a shared control system for a mobile augmented reality user interface. Aimed at enhancing the autonomy of individuals with physical disabilities, particularly those with impaired motor function due to conditions such as stroke, the system utilizes BCI to interpret user intentions from electroencephalography signals and eye tracking to identify the object of focus, thus refining control commands. This integration seeks to create a more intuitive and responsive assistive robot control strategy. The real-world usability was evaluated, demonstrating significant potential to improve autonomy for individuals with severe motor impairments. The control system was compared with an eye-tracking-based alternative to identify areas needing improvement. Although BCI achieved an acceptable success rate of 0.83 in the final phase, eye tracking was more effective with a perfect success rate and consistently lower completion times (p<0.001). The user experience responses favored eye tracking in 11 out of 26 questions, with no significant differences in the remaining questions, and subjective fatigue was higher with BCI use (p=0.04). While BCI performance lagged behind eye tracking, the user evaluation supports the validity of our control strategy, showing that it could be deployed in real-world conditions and suggesting a pathway for further advancements.
Assistive Technology: Design and Implementation of an Eye Tracking Based Electric Wheelchair Control System for Children with Cerebral Palsy

Katherine Hernandez,Veronica Salinas,Hugo Torres,Fernando Estevez,Diego Almeida- Galárraga,Omar Alvarado Cando

DOI: https://doi.org/10.54941/ahfe1001027

2022-01-01

Abstract:Cerebral palsy (CP) is a set of non-progressive alterations which affect muscle tone, posture, and movement. CP is produced by a brain injury or malformation in one or more brain areas that control movement. It is the most common motor disability in childhood. It affects the autonomy and socialization of children and generates great pressure on the family; therefore, children with CP require the aid of a wheelchair, which is usually driven by the user or pushed by another person. Even using a wheelchair controlled with a joystick becomes complicated because it depends on the control of gross and fine motor skills. Then, there is necessary a more amicable and accessible way to control a wheelchair. Since the ocular muscles are one of the few that still fulfill their function in people with CP, in this paper, we present the design and implementation of an eye-tracking-based electric wheelchair control system for children with spastic and mixed CP. This system includes a low-cost electric wheelchair built with PCV tubes, a Microsoft Surface Pro Tablet where the software is running, and a PCEye Mini device used to monitor the gaze. The PCEeye Mini sends the signs to the mainboard that controls the electric motors, allowing the children to steer the wheelchair by themselves using their gaze. The software has three components: the first manages the application data, the second represents the model graphically displayed on the screen, and the third receives and manages user inputs. The application has three interfaces for the start of the system that children can access since the home screen, and each one gives the user different control system parameters options. Eight children with CP participated in three experiments to validate the efficiency and safety of the system. Each experiment was developed in an internal and external environment with three different floor soils: cement, ceramic, and wood. The results showed a 100% efficiency and safety of the system for children use. The calibration process was fast and simple, the activation of directional arrows on the screen with the gaze gave 100% performance, the power consumption did not show a difference between the type of floor, and the electric wheelchair did not slip on any floor. The software maintained stable communication with the hardware through a serial communication protocol, allowing a correct execution of the commands. This work shows that the implementation of the eye-tracking technology in the electric wheelchair works successfully. This system operates effectively and is easy to control for children with CP, improving the children’s autonomy in terms of mobility. This software can be implemented on any PC, and more functions can be added to the same software without affecting its performance. This developed system offers an intelligent and different option to help children with motor deficits, which is a great contribution to society.
Human brain control of electric wheelchair with eye-blink electrooculogram signal

Bo Ning,Ming-jie Li,Tong Liu,Hui-min Shen,Liang Hu,Xin Fu

DOI: https://doi.org/10.1007/978-3-642-33509-9_58

2012-01-01

Abstract:In this paper, a human brain-based control method of electric wheelchair is presented for individuals with motor disabilities. In this method, intended eye-blink electrooculogram (EOG) signal measured by a single dry electrode is utilized as communication channel for random direction control of wheelchair. Here, a close-loop wheelchair control system with human-machine interface (HMI) based on this method is introduced, and the feedback is realized by human vision. To validate feasibility of this brain-controlled electric wheelchair system with eye-blink EOG signal, user is required to operate wheelchair with the proposed control system to move along three kinds of designated routes. The results show good performance of this brain-controlled wheelchair system with eye-blink EOG signal. Application of the proposed hand-free control system is expected to help people with motor disabilities live an improved lifestyle with more autonomy.
Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping

Ali Shafti,Pavel Orlov,A. Aldo Faisal

DOI: https://doi.org/10.1109/icra.2019.8793804

2019-05-01

Abstract:Assistive robotic systems endeavour to support those with movement disabilities, enabling them to move again and regain functionality. Main issue with these systems is the complexity of their low-level control, and how to translate this to simpler, higher level commands that are easy and intuitive for a human user to interact with. We have created a multi-modal system, consisting of different sensing, decision making and actuating modalities, leading to intuitive, human-in-the-loop assistive robotics. The system takes its cue from the user’s gaze, to decode their intentions and implement low-level motion actions to achieve high-level tasks. This results in the user simply having to look at the objects of interest, for the robotic system to assist them in reaching for those objects, grasping them, and using them to interact with other objects. We present our method for 3D gaze estimation, and grammars-based implementation of sequences of action with the robotic system. The 3D gaze estimation is evaluated with 8 subjects, showing an overall accuracy of 4.68pm 0.14cm. The full system is tested with 5 subjects, showing successful implementation of 100% of reach to gaze point actions and full implementation of pick and place tasks in 96%, and pick and pour tasks in 76% of cases. Finally we present a discussion on our results and what future work is needed to improve the system.
A Novel Eye-Gaze-Controlled Wheelchair System for Navigating Unknown Environments: Case Study With a Person With ALS

Mohamad A. Eid,Nikolas Giakoumidis,Abdulmotaleb El Saddik

DOI: https://doi.org/10.1109/access.2016.2520093

IF: 3.9

2016-01-01

IEEE Access

Abstract:Thanks to advances in electric wheelchair design, persons with motor impairments due to diseases, such as amyotrophic lateral sclerosis (ALS), have tools to become more independent and mobile. However, an electric wheelchair generally requires considerable skill to learn how to use and operate. Moreover, some persons with motor disabilities cannot drive an electric wheelchair manually (even with a joystick), because they lack the physical ability to control their hand movement (such is the case with people with ALS). In this paper, we propose a novel system that enables a person with motor disability to control a wheelchair via eye-gaze and to provide a continuous, real-time navigation in unknown environments. The system comprises a Permobile M400 wheelchair, eye tracking glasses, a depth camera to capture the geometry of the ambient space, a set of ultrasound and infrared sensors to detect obstacles with low proximity that are out of the field of view for the depth camera, a laptop placed on a flexible mount for maximized comfort, and a safety off switch to turn off the system whenever needed. First, a novel algorithm is proposed to support continuous, real-time target identification, path planning, and navigation in unknown environments. Second, the system utilizes a novel N-cell grid-based graphical user interface that adapts to input/output interfaces specifications. Third, a calibration method for the eye tracking system is implemented to minimize the calibration overheads. A case study with a person with ALS is presented, and interesting findings are discussed. The participant showed improved performance in terms of calibration time, task completion time, and navigation speed for a navigation trips between office, dining room, and bedroom. Furthermore, debriefing the caregiver has also shown promising results: the participant enjoyed higher level of confidence driving the wheelchair and experienced no collisions through all the experiment.

computer science, information systems,telecommunications,engineering, electrical & electronic
Non-invasive Cognitive-level Human Interfacing for the Robotic Restoration of Reaching & Grasping

Ali Shafti,A. Aldo Faisal

DOI: https://doi.org/10.1109/ner49283.2021.9441453

2021-05-04

Abstract:Assistive and Wearable Robotics have the potential to support humans with different types of motor impairments to become independent and fulfil their activities of daily living successfully. The success of these robot systems, however, relies on the ability to meaningfully decode human action intentions and carry them out appropriately. Neural interfaces have been explored for use in such system with several successes, however, they tend to be invasive and require training periods in the order of months. We present a robotic system for human augmentation, capable of actuating the user's arm and fingers for them, effectively restoring the capability of reaching, grasping and manipulating objects; controlled solely through the user's eye movements. We combine wearable eye tracking, the visual context of the environment and the structural grammar of human actions to create a cognitive-level assistive robotic setup that enables the users in fulfilling activities of daily living, while conserving interpretability, and the agency of the user. The interface is worn, calibrated and ready to use within 5 minutes. Users learn to control and make successful use of the system with an additional 5 minutes of interaction. The system is tested with 5 healthy participants, showing an average success rate of 96.6% on first attempt across 6 tasks.
A Wearable Human–Machine Interactive Instrument for Controlling a Wheelchair Robotic Arm System

Zilin Lu,Yajun Zhou,Li Hu,Junbiao Zhu,Songhan Liu,Qiyun Huang,Yuanqing Li

DOI: https://doi.org/10.1109/tim.2024.3376685

IF: 5.6

2024-03-27

IEEE Transactions on Instrumentation and Measurement

Abstract:The limitations of traditional manual human–machine interactive instruments (HMIs) prevent individuals with severe motor disabilities from effectively controlling wheelchair robotic arm systems, thereby impacting their independence and quality of life. Addressing this issue, this study developed a wearable hybrid HMI to meet the practical needs of individuals with severe motor disabilities. By recruiting ten healthy participants to complete three experiments—a blink detection test, wheelchair control test, and wheelchair robotic arm system test—the effectiveness of the proposed HMI was validated. In the first experiment, the average accuracy of blink detection in the system reached 97.29%, with an average response time of 1.47 s, and the system generated 0.07 errors per minute in idle state. The second experiment demonstrated an average response time of 1.03 s for wheelchair turning and 1.48 s for wheelchair stopping. In the hybrid control condition, the wheelchair navigated obstacles along the specified route in an average time of 1.14 min, and participants executed a minimum of 25.20 commands. In the third experiment, all participants successfully completed the mobile self-drinking task by controlling the wheelchair robotic arm system, with an average workload reported on the NASA task load index (NASA-TLX) scale of 30.2. The study revealed that the proposed HMI offers a promising solution for nonmanual control in complex rehabilitation assistive systems. It has the potential to assist a wider range of individuals with motor disabilities, improving their daily life experiences.

engineering, electrical & electronic,instruments & instrumentation
Smart Wheelchair Controlled Through a Vision-Based Autonomous System

Usman Masud,Nawaf Abdualaziz Almolhis,Ali Alhazmi,Jayabrabu Ramakrishnan,Fezan Ul Islam,Abdul Razzaq Farooqi

DOI: https://doi.org/10.1109/access.2024.3395656

IF: 3.9

2024-05-15

IEEE Access

Abstract:People are preoccupied with their own duties in today's environment, making it tough to oversee and care for disabled people. Paralyzed and disabled people, on the other hand, have a powerful desire to move around freely. Different initiatives have been taken in the past to improve and preserve the self-esteem of such people, and various technologies have also been created to assist them in a better way. The primary advantage of a vision-based autonomous wheelchair is that it can provide greater independence to users with limited mobility. The methodology of other smart wheelchairs varies depending on the specific technology used. Some smart wheelchairs may use sensors to detect obstacles and provide feedback to the user, while others may incorporate GPS and other navigation technologies to assist with indoor and outdoor navigation. Our study proposes a revolutionary implementation of an autonomous system for fully handicapped people, allowing them to drive wheelchairs using just their eyes. The wheelchair's orientation will be determined by the orientation of the eye. The camera will follow the movement of the eyes. Furthermore, sensors will be installed on the front side of the wheelchair to identify any obstacles along its path. The calibration of the camera with the human eye, without obstructing its vision, is the most challenging task in this research, besides controlling the wheels of the wheelchair.

computer science, information systems,telecommunications,engineering, electrical & electronic
An EOG-based wheelchair robotic arm system for assisting patients with severe spinal cord injuries

Qiyun Huang,Yang Chen,Zhijun Zhang,Shenghong He,Rui Zhang,Jun Liu,Yuandong Zhang,Ming Shao,Yuanqing Li

DOI: https://doi.org/10.1088/1741-2552/aafc88

Abstract:Objective: In this study, we combine a wheelchair and an intelligent robotic arm based on an electrooculogram (EOG) signal to help patients with spinal cord injuries (SCIs) accomplish a self-drinking task. The main challenge is to accurately control the wheelchair to ensure that the randomly located object is within a limited reachable space of the robotic arm (length: 0.8 m; width: 0.4 m; height: 0.6 m), which requires decimeter-level precision, and is still undemonstrated for EOG-based systems as well as EEG-based systems. Approach: A novel high-precision EOG-based human machine interface (HMI) is proposed which can effectively translate two kinds of eye movements (i.e. blinking and eyebrow raising) into various commands. For the wheelchair, positional precision can reach decimeter-level and the minimal steering angle is [Formula: see text]. For the intelligent robotic arm, shared control is implemented based on an EOG-based HMI, two cameras and the arm's own intelligence. Main results: After brief training, five healthy subjects and five paralyzed patients with severe SCIs successfully completed three experiments. For the healthy subjects/patients with SCIs, the system achieved an average accuracy of 99.3%/97.3%, an average response time of 1.91 s/2.02 s per command and an average stop-response time of 1.30 s/1.36 s with a 0 false operation rate. Significance: The EOG-based HMI can provide sufficient precision control to integrate a wheelchair and a robotic arm into a system which can help patients with SCIs to accomplish a self-drinking task. (ChiCTR1800019764).
Multi-motion Robots Control Based on Bioelectric Signals from Single-Channel Dry Electrode

Hui-Min Shen,Liang Hu,Kok-Meng Lee,Xin Fu

DOI: https://doi.org/10.1177/0954411915570079

2015-01-01

Proceedings of the Institution of Mechanical Engineers Part H Journal of Engineering in Medicine

Abstract:This article presents a multi-motion control system to help severe disabled people operate an auxiliary appliance using neck-up bioelectric signals measured by a single-channel dry electrode on the forehead. The single-channel dry-electrode multi-motion control system exhibits several practical advantages over its conventional counterparts that use multi-channel wet-electrodes; among the challenges is an effective technique to extract bioelectric features for reliable implementation of multi degrees-of-freedom motion control. Using both time and frequency characteristics of the single-channel dry-electrode measurements, motion commands are derived from multiple feature signals associated with concentration demands and different eye-blink actions for use in a two-level control strategy that has been developed to control predefined multi degrees-of-freedom motion trajectories. Test paradigms were designed to pre-calibrate the users' feature signals to statistically account for individual variances. Experimental trials were then carried out on able-bodied and disabled volunteers to validate the universal applicability of the algorithms. The classification success rates for two different eye-blink feature signals were approximately 95% with an average time of 2.4 s for executing a concentration feature signal. The single-channel dry-electrode-based technique has been validated on a 6-degree-of-freedom robot arm demonstrating its significant potentials to help patients suffering severe motor dysfunctions operate a multi-motion auxiliary appliance in everyday living where the ease of use is a priority.
An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System

Qiyun Huang,Zhijun Zhang,Tianyou Yu,Shenghong He,Yuanqing Li

DOI: https://doi.org/10.3389/fnins.2019.01243

IF: 4.3

2019-11-22

Frontiers in Neuroscience

Abstract:Most existing brain-computer Interfaces (BCIs) are designed to control a single assistive device, such as a wheelchair, a robotic arm or a prosthetic limb. However, many daily tasks require combined functions which can only be realized by integrating multiple robotic devices. Such integration raises the requirement of the control accuracy and is more challenging to achieve a reliable control compared with the single device case. In this study, we propose a novel hybrid BCI with high accuracy based on electroencephalogram (EEG) and electrooculogram (EOG) to control an integrated wheelchair robotic arm system. The user turns the wheelchair left/right by performing left/right hand motor imagery (MI), and generates other commands for the wheelchair and the robotic arm by performing eye blinks and eyebrow raising movements. Twenty-two subjects participated in a MI training session and five of them completed a mobile self-drinking experiment, which was designed purposely with high accuracy requirements. The results demonstrated that the proposed hBCI could provide satisfied control accuracy for a system that consists of multiple robotic devices, and showed the potential of BCI-controlled systems to be applied in complex daily tasks.

neurosciences
Assistive Mobile Robot with Shared Control of Brain-Machine Interface and Computer Vision

Yonghao Song,Weifeng Wu,Chengqi Lin,Gengliang Lin,Guofeng Li,Longhan Xie

DOI: https://doi.org/10.1109/itnec48623.2020.9085096

2020-01-01

Abstract:The daily life of stroke patients may be severely limited. In this paper, we proposed a shared control based assistive mobile robot system to improve their ability. Brain-machine interface (BMI) was used for obtaining the user's intention to give a command to the robot. The computer vision technology with laser radar and camera was equipped to detect the environment for obstacle avoidance and target recognition. With the spatial coordinate provided by the vision part, the robot would grasp the target back to the user eventually. The test was carried out on both the BMI based console part and the robot executive. The results showed good performance with satisfied BMI accuracy, quite small route planning error, recognition error and motion planning error, which make us convinced that this assistive mobile robot can help the disabled people.
Eye-Gaze Controlled Wheelchair Based on Deep Learning

Jun Xu,Zuning Huang,Liangyuan Liu,Xinghua Li,Kai Wei

DOI: https://doi.org/10.3390/s23136239

IF: 3.9

2023-07-07

Sensors

Abstract:In this paper, we design a technologically intelligent wheelchair with eye-movement control for patients with ALS in a natural environment. The system consists of an electric wheelchair, a vision system, a two-dimensional robotic arm, and a main control system. The smart wheelchair obtains the eye image of the controller through a monocular camera and uses deep learning and an attention mechanism to calculate the eye-movement direction. In addition, starting from the relationship between the trajectory of the joystick and the wheelchair speed, we establish a motion acceleration model of the smart wheelchair, which reduces the sudden acceleration of the smart wheelchair during rapid motion and improves the smoothness of the motion of the smart wheelchair. The lightweight eye-movement recognition model is transplanted into an embedded AI controller. The test results show that the accuracy of eye-movement direction recognition is 98.49%, the wheelchair movement speed is up to 1 m/s, and the movement trajectory is smooth, without sudden changes.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Persistent Human–Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking

Seunghyeb Ban,Yoon Jae Lee,Ki Jun Yu,Jae Won Chang,Jong-Hoon Kim,Woon-Hong Yeo

DOI: https://doi.org/10.1002/aisy.202200408

IF: 7.298

2023-04-08

Advanced Intelligent Systems

Abstract:Herein, a persistent human–machine interface to control robotic arms via gaze and eye direction tracking is presented. Recent advances in sensors and electronics have enabled electrooculogram (EOG) detection systems for capturing eye movements. However, EOG signals are susceptible to the sensor's skin‐contact quality, limiting the precise detection of eye angles and gaze. Herein, a two‐camera eye‐tracking system and a data classification method for persistent human–machine interfaces (HMIs) are introduced. Machine‐learning technology is used for a continuous real‐time classification of gaze and eye directions, to precisely control a robotic arm. In addition, a deep‐learning algorithm for classifying eye directions is developed and the pupil center‐corneal reflection method of an eye tracker for gaze tracking is utilized. A supervisory control and data acquisition architecture that can be universally applied to any screen‐based HMI task are used by the system. It is shown in the study that the classification algorithm using deep learning enables exceptional accuracy (99.99%) with the number of actions per command (≥64), the highest performance compared to other HMI systems. Demonstrating real‐time control of a robotic arm captures the unique advantages of the precise eye‐tracking system for playing chess and manipulating dice. Overall, this paper shows the HMI system's potential for remote control of surgery robots, warehouse systems, and construction tools.

Technical Problems in the Operative Correction of Pectus Excavatum

Control of a Wheelchair-Mounted 6DOF Assistive Robot With Chin and Finger Joysticks

Diegetic Graphical User Interfaces and Intuitive Control of Assistive Robots via Eye-gaze

Development of a Human-Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM

An Intelligent and Low-cost Eye-tracking System for Motorized Wheelchair Control

A Portable Wireless Eye Movement-Controlled Human-Computer Interface for the Disabled

Eye Gaze Controlled Robotic Arm for Persons with SSMI

User Evaluation of a Shared Robot Control System Combining BCI and Eye Tracking in a Portable Augmented Reality User Interface

Assistive Technology: Design and Implementation of an Eye Tracking Based Electric Wheelchair Control System for Children with Cerebral Palsy

Human brain control of electric wheelchair with eye-blink electrooculogram signal

Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping

A Novel Eye-Gaze-Controlled Wheelchair System for Navigating Unknown Environments: Case Study With a Person With ALS

Non-invasive Cognitive-level Human Interfacing for the Robotic Restoration of Reaching & Grasping

A Wearable Human–Machine Interactive Instrument for Controlling a Wheelchair Robotic Arm System

Smart Wheelchair Controlled Through a Vision-Based Autonomous System

An EOG-based wheelchair robotic arm system for assisting patients with severe spinal cord injuries

Multi-motion Robots Control Based on Bioelectric Signals from Single-Channel Dry Electrode

An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System

Assistive Mobile Robot with Shared Control of Brain-Machine Interface and Computer Vision

Eye-Gaze Controlled Wheelchair Based on Deep Learning

Persistent Human–Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking