Abstract:Handwritten music recognition (HMR) is the technology of transcribing the content of images of music scores. The accurate detection of music objects at the page level is one of the main challenges of HMR. Thus far, the existing methods suffer from the tiny and dense nature of handwritten music notations and realize positive detection accuracy only on snippets. In this paper, we propose a detector that consists of a staff line removal model and a handwritten music object detection model for page-level handwritten music object recognition. First, an end-to-end staff line removal model R_Staff_Net based on residual learning reduces the complexity of page-level detection. Second, we developed an improved YOLO-V4 model for handwritten music object detection. The improvements mainly concern the adoption of a de-coupled detection head and visual attention module in the YOLO-V4, and an adaptive multi-scale feature fusion module AMFFM is used to enhance the textures and features of tiny music symbols in the deep convolution layers, and the gradient harmonized mechanism is utilized to address the inherent imbalance between music objects. We verified the R_Staff_Net and the improved YOLO-V4 model on the ICDAR/GREC staff line removal dataset and the MUSCIMA++ dataset, respectively. The experiments highlight that R_Staff_Net presents outstanding performance with an F-M score of 98.64%, and our improved YOLO-V4 model is superior to other handwritten music symbol detection methods with a mean average precision (mAP) of 91.8% when addressing page-level input. Although the experimental results of the detector show that the R_Staff_Net helps little to the overall mAP, the network is beneficial for symbols that are similar to staff lines or heavily overlap staff lines.

Deep Watershed Detector for Music Object Recognition

Music Waveform Analysis Based on SOM Neural Network and Big Data

State-of-the-Art Model for Music Object Recognition with Deep Learning

Application and Research on Optical Music Recognition

Knowledge Discovery in Optical Music Recognition: Enhancing Information Retrieval with Instance Segmentation

Coordinate Embedding Transformer Model for Optical Music Recognition on Monophonic Scores

Research of Numbered Musical Notation Recognition Method

M-Detr: Multi-Scale Detr for Optical Music Recognition

Toward a More Complete OMR Solution

DeepScores -- A Dataset for Segmentation, Detection and Classification of Tiny Objects

A detector for page-level handwritten music object recognition based on deep learning

TrOMR:Transformer-Based Polyphonic Optical Music Recognition

Real-Time Optical Music Recognition System for Dulcimer Musical Robot

Understanding Optical Music Recognition

A Unified Representation Framework for the Evaluation of Optical Music Recognition Systems

A New Optical Music Recognition System Based on Combined Neural Network

End-to-End Optical Music Recognition with Attention Mechanism and Memory Units Optimization.

Structure Analysis Approach of Music Notes for Optical Music Recognition

Practical End-to-End Optical Music Recognition for Pianoform Music

A Watermarking Algorithm for Embedding of MIDI Information in the Digital Music Score

In Search of a Dataset for Handwritten Optical Music Recognition: Introducing MUSCIMA++