Abstract:To improve the efficiency of manual assembly process,a gesture recognition and error prevention method to guarantee process sequence was proposed.An Augmented Reality (AR) assembly assistant system controlled by vision recognized signal was presented to guide error correction,which reduced the time cost of interaction and rework.An example assembly video was captured by an ego-centric camera,and time measurement method was used to analyze the assembly operation time and sequence.The assembly process simulation was performed to construct the assembly guidance information model.Hand region on each frame of sampled video was extracted and then processed into geometric features.Gesture recognition constraints for each operation step was proposed based on probability distribution fitting.After adding the assembly time and sequential constraints,the recognized gesture features were further processed to assembly sequence signals that update the user's assembly progress and control the display of AR guidance information.The proposed methods were verified by a computer motherboard assembly case study,and the results showed an increment on assembly efficiency and one-time completion rate.

Sequential dynamic gesture recognition controlled poka-yoke system for manual assembly