Abstract:With the increasing demand for product customization, the exponential increase in large-scale and small-batch production orders has yielded new challenges for the capacity of traditional production lines. Due to the information redundancy and complex production process on the production site of the factory, the traditional monitoring system is incapable of meeting the information interaction requirements between the factory management level and the executive level. Meanwhile, the traditional augmented reality (AR) based on image recognition is not suitable in the complex industrial environments. To address the gap, we propose a novel mobile production monitoring system (AIMPMs) with human-in-the-loop control by leveraging the cutting-edge AR and indoor positioning technique. In the proposed system, Ultra-wideband (UWB) and Inertial Measurement Unit (IMU) fusion indoor positioning technology is proposed, which provides accurate indoor positioning information for the production factors in the factory. Subsequently, we build the lightweight indoor map for positioning that can serve as the location reference and path planning, and a nearest-neighbor decision algorithm with double rejection decision (NN-DRD) is proposed to match the positioning features to trigger virtual monitoring information. Finally, the AIMPMs is applied in a hydraulic cylinder factory to verify its enforceability and effectiveness, and the human factors evaluation model and index system are constructed to evaluate two systems, (1) the mobile monitoring system based on positioning information, and (2) the mobile monitoring system based on image recognition, respectively. The experimental results indicate that after the application of AIMPMs, the physiological and mental fatigue of the production personnel is immensely decreased. Therefore, the system realizes a smarter and highly humanized human-machine interaction mode with human-in-the-loop control.

Face-to-Machine Proximity Estimation for Mobile Industrial Human Machine Interaction

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Design and Evaluation of a Wireless Magnetic-Based Proximity Detection Platform for Indoor Applications

Intention recognition-based human–machine interaction for mixed flow assembly

Human-centric Application in Cyber-Physical System: an Inertial-based Motion Capture and Recognition System

Mechanical Acoustic Signal Assisted Translational Model for Industrial Human-Machine Interaction

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

The investigation and implementation of real-time face pose and direction estimation on mobile computing devices

A Real Time Face Detection Method in Human-Machine Interaction

Augmented reality and indoor positioning based mobile production monitoring system to support workers with human-in-the-loop

Human-computer interaction based on machine vision of a smart assembly workbench

Development of human-machine interface for teleoperation of a mobile manipulator

Fast 3D Hand Estimation for Mobile Interactions

Detection and Estimation of Mental Fatigue in Manual Assembly Process of Complex Products

Research on the Three-Machines Perception System and Information Fusion Technology for Intelligent Work Faces

Dynamic Hazardous Proximity Zone Design for Excavator Based on 3D Mechanical Arm Pose Estimation Via Computer Vision

It's the Human that Matters: Accurate User Orientation Estimation for Mobile Computing Applications

Look into My Eyes: Fine-grained Detection of Face-screen Distance on Smartphones

Robust Proximity Detection using On-Device Gait Monitoring

Human-machine interfaces based on EMG and Kinect applied to teleoperation of a mobile humanoid robot