Gesture Recognition Technology of VR Piano Playing Teaching Game based on Hidden Markov Model
Min Zeng
DOI: https://doi.org/10.34028/iajit/21/4/15
2024-01-01
The International Arab Journal of Information Technology
Abstract:Virtual Reality (VR) is a kind of simulation environment generated by computer simulation. It is a kind of isolation of users from the physical environment, and users are fully engaged in the simulation environment. In music teaching, especially in the learning of piano performance, VR technology can simulate the real playing experience without physical piano, so that learners can learn and practice piano anywhere. At present, the traditional way of learning piano often needs the guidance of professional piano teachers, and learners must study and practice beside the piano, which undoubtedly brings limitations to the time and space of learners. VR technology provides a more convenient way for learners to learn and practice without a physical piano, making learning more flexible and free. Firstly, a piano playing teaching game platform based on VR technology is constructed to provide a virtual piano playing scene. Then, by collecting a large number of piano players' gesture data, they are accurately marked. Finally, the Hidden Markov Model (HMM) is used to train and model the gesture data, and the key features of the gesture sequence are extracted to realize the recognition and real-time feedback of the piano player's gesture. According to the findings, the recognition accuracy of the Principal Component Analysis-HMM (PCA-HMM) without playing correction is very low after playing correction. Compared to the logarithmic likelihood ratio test, the recognition rate of the quadratic statistical model is only 15%. The results show that this method has good iterative convergence and convergence performance, which can effectively avoid falling into local optima. Compared with the traditional hidden Markov algorithm, it reduces 96.05% and 89.48% respectively. Compared with the fuzzy hidden Markov algorithm, it reduces 90.20% and 73.35% respectively. The average time and standard deviation for real-time positioning and map construction are 0.251s and 0.152s, respectively. The fitness of HMM based on fuzzy Particle Swarm Optimization (PSO) is 6.8% and 2.6% higher than that of the hidden Markov algorithm and the fuzzy hidden Markov algorithm, respectively. At the same time, in the comparison between the research model and other algorithm models, the accuracy and other parameter performance of the research model are better, with the highest error value of only 3.5, which is 5.9 lower than the lowest Convolutional Neural Network (CNN) algorithm model. It can be seen that the error value of the model used in the study is lower and the performance is better. Learners can receive real-time guidance and a more immersive playing experience through this technology. This provides useful reference for further exploring the combination of music teaching and VR technology
computer science, information systems, artificial intelligence,engineering, electrical & electronic