Abstract:In view of the low recognition rate of lip recognition, this paper uses the interframe difference method to process the dataset, uses 3D Convolutional Neural Network and ResNetl8 residual network structure to extract image features, and uses Temporal Convolutional Network to extract semantic information between images to realize lip recognition. In order to retain more lip movement details of the original feature map and reduce the complexity of the overall network, dual attention mechanisms of space and channel are added to ResNet residual unit. The experiment was carried out on the published English lip dataset LRW, and the accuracy of the method reached 89.88%. Through the comparison experiment, the results show that the method can effectively improve the accuracy of lip recognition.

Lip Recognition Based on 3D Convolutional Neural Network