Abstract:Significant challenges in perception, prediction, and decision-making within self-driving systems remain inadequately addressed. Concurrently, the advancement of autonomous driving technologies reduces driver engagement, inadvertently eroding their proficiency. Integrating human cognitive flexibility and experiential insight with the machine's precision and reliability offers a promising approach for the transitional phase towards fully automated driving. This study presents a human-machine collaboration approach to enhance the highly automated vehicles' high-level flexibility and personalization attribute without the need for passengers' prior driving experience. Firstly, we propose a tactical human–vehicle collaboration framework leveraging the hand-landmark extraction algorithm and augmented visual feedback. The proposed vision-based interface projects the gesture onto the ground and feeds it back to the driver through the augmented reality head-up display (AR-HUD) for intuitive interaction. The projection offers strategic decision-making guidance and planning recommendations for the vehicle. Utilizing these suggestions, the automation algorithm efficiently manages the remaining tasks, including collision avoidance and adherence to traffic regulations. This approach minimizes the driver's engagement in routine driving tasks and negates the need for driving skills. Incorporating cooperative game theory, the methodology optimally balances personalization with system robustness. Finally, we compare our approach with conventional manual driving schemes that both can assist the self-driving car in avoiding unknown obstacles and reaching the personalized goal. Results demonstrate that the proposed decision-making and planning collaboration scheme significantly reduces human physical burdens without compromising driving performance and driver mental workloads.

Human-Machine Collaboration for Automated Vehicles via an Intelligent Two-Phase Haptic Interface

Human–Machine Collaboration for Automated Driving Using an Intelligent Two‐Phase Haptic Interface

A Novel Haptic Takeover Method Based on Human-Machine Collaboration States

Shared and traded control for human-automation interaction: a haptic steering controller and a visual interface

Design of steering wheel torque with human-machine interaction for uncoupled shared steering

Sharing control with haptics: seamless driver support from manual to automatic control

An intelligent human-machine interaction-based longitudinal control strategy for autonomous vehicles

Human-Automation Interaction Through Shared and Traded Control Applications

A Workload Adaptive Haptic Shared Control Scheme for Semi-Autonomous Driving

Pattern Recognition and Characterization of Upper Limb Neuromuscular Dynamics during Driver-Vehicle Interactions

Auxiliary Steering Control of Vehicle Driving with Force/Haptic Guidance

A Human-Machine Shared Control Framework Considering Time-Varying Driver Characteristics

Human–machine cooperative decision-making and planning for automated vehicles using spatial projection of hand gestures

Human-Machine Shared Control for Semi-Autonomous Vehicles Using Level of Cooperativeness

Adaptive Driver-Automation Shared Steering Control via Forearm Surface Electromyography Measurement

Human-vehicle dynamic model with driver’s neuromuscular characteristic for shared control of autonomous vehicle

Research on the Steering Torque Control for Intelligent Vehicles Co-Driving with the Penalty Factor of Human–Machine Intervention

Variable Impedance-based Human-machine Interaction Method Using Reinforcement Learning for Shared Steering Control of Intelligent Vehicle

Haptic Feedback for the Transfer of Control in Autonomous Vehicles

Toward human-vehicle collaboration: Review and perspectives on human-centered collaborative automated driving

Acceptance and Usability of a Soft Robotic, Haptic Feedback Seat for Autonomy Level Transitions in Highly Automated Vehicles