Abstract:Sheet music recognition is a vital technology aimed at converting printed or handwritten musical scores into digital or machine-readable formats. The significance of this technology lies in making music compositions more accessible for editing, performance, learning, and sharing, thereby fostering music education, composition, and culture. It also provides a powerful tool for music analysis, research, and preservation. Our aim is to investigate a sheet music recognition method that offers a simple workflow, high recognition accuracy, and fast model convergence. Specifically, the proposed Deep Multilevel Cascade Residual Recurrent (MCRR) framework for sheet music recognition consists of the following components. Firstly, we introduce additive Gaussian white noise, additive Perlin noise, and elastic deformations such as rotation and stretching to simulate real-world noise in the sheet music images, thereby augmenting the dataset, enhancing model robustness, and mitigating overfitting. Secondly, in the feature extraction phase, we employ a residual Convolutional Neural Network (ConvNet) to address the issue of model degradation and use the multilevel cascade fusion technique to obtain comprehensive feature information, improving the model’s feature extraction capability and reducing recognition errors. For note recognition, we use a variant of RNN (Recurrent Neural Network) called SRU (Simple Recurrent Unit), which transforms most computations into parallel processing, speeding up model convergence. Finally, we combine the Connectionist Temporal Classification (CTC) loss function with SRU to eliminate the requirement for strict alignment between data and labels, enabling note classification and recognition. Extensive ablation experiments and comparative analyses, including visual analysis, intuitive illustrations, and quantitative assessments, confirm the effectiveness of the proposed method, demonstrating its superiority over various state-of-the-art methods. The proposed method achieved promising results in both the PrIMus and Camera-PrIMuS datasets. Specifically, in the PrIMus dataset, the method obtained an SeER (Symbol Error Rate) of 1.4571% and a SyER (System Error Rate) of 0.3234%. Notably, it demonstrated high accuracy in pitch, type, and note recognition, scoring approximately 97% in pitch and type accuracy and around 94% in note accuracy. The training time per epoch was relatively low, recorded at 0.56 seconds. In the case of the Camera-PrIMuS dataset, the method achieved slightly lower but still competitive results. It exhibited an SeER of 5.1488% and a SyER of 1.0612%, with pitch and type accuracies around 90%, and note accuracy at approximately 88%. The training time per epoch was slightly higher at 1.93 seconds Furthermore, we compare our method with existing commercial software, namely Capella-scan, PhotoScore, and SmartScore. Among these, Capella-scan delivers the best performance but exhibits lower robustness compared to the proposed method.

Advancing Handwritten Musical Notation Recognition Using Deep Learning: A Convolutional Neural Network-Based Approach with Improved Accuracy

Research of Numbered Musical Notation Recognition Method

Improved Feature Pyramid Convolutional Neural Network for Effective Recognition of Music Scores

Deep Multilevel Cascade Residual Recurrent Framework (MCRR) for Sheet Music Recognition

Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN)

Coordinate Embedding Transformer Model for Optical Music Recognition on Monophonic Scores

A detector for page-level handwritten music object recognition based on deep learning

Low-Data Classification of Historical Music Manuscripts: A Few-Shot Learning Approach

Enhancement of handwritten text recognition using AI-based hybrid approach

Note Detection in Music Teaching Based on Intelligent Bidirectional Recurrent Neural Network

Detecting Noteheads in Handwritten Scores with ConvNets and Bounding Box Regression

Generative Adversarial Network for Musical Notation Recognition during Music Teaching

Handwritten Digit Recognition using Machine and Deep Learning Algorithms

Improvement of Speech Recognition Technology in Piano Music Scene Based on Deep Learning of Internet of Things

NeuroWrite: Predictive Handwritten Digit Classification using Deep Neural Networks

A Deep Learning-Based Piano Music Notation Recognition Method

Pianist Identification Using Convolutional Neural Networks

Handwritten Digit Recognition of MNIST dataset using Deep Learning state-of-the-art Artificial Neural Network (ANN) and Convolutional Neural Network (CNN)

Automatic Audio Chord Recognition with MIDI-Trained Deep Feature and BLSTM-CRF Sequence Decoding Model.

Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Convolutional Neural Network

Evaluation System of Music Art Instructional Quality Based on Convolutional Neural Networks and Big Data Analysis