Abstract:Developmental dysgraphia is a neurological disorder that hinders children's writing skills. In recent years, researchers have increasingly explored machine learning methods to support the diagnosis of dysgraphia based on offline and online handwriting. In most previous studies, the two types of handwriting have been analysed separately, which does not necessarily lead to promising results. In this way, the relationship between online and offline data cannot be explored. To address this limitation, we propose a novel multimodal machine learning approach utilizing both online and offline handwriting data. We created a new dataset by transforming an existing online handwritten dataset, generating corresponding offline handwriting images. We considered only different types of word data (simple word, pseudoword & difficult word) in our multimodal analysis. We trained SVM and XGBoost classifiers separately on online and offline features as well as implemented multimodal feature fusion and soft-voted ensemble. Furthermore, we proposed a novel ensemble with conditional feature fusion method which intelligently combines predictions from online and offline classifiers, selectively incorporating feature fusion when confidence scores fall below a threshold. Our novel approach achieves an accuracy of 88.8%, outperforming SVMs for single modalities by 12-14%, existing methods by 8-9%, and traditional multimodal approaches (soft-vote ensemble and feature fusion) by 3% and 5%, respectively. Our methodology contributes to the development of accurate and efficient dysgraphia diagnosis tools, requiring only a single instance of multimodal word/pseudoword data to determine the handwriting impairment. This work highlights the potential of multimodal learning in enhancing dysgraphia diagnosis, paving the way for accessible and practical diagnostic tools.

Automated dysgraphia detection by deep learning with SensoGrip

Handwriting Evaluation Using Deep Learning with SensoGrip

Deep Learning and Procrustes Analysis for Early Dysgraphia Risk Detection with a Tablet Application

SmartGe: Identifying Pen-Holding Gesture with Smartwatch.

Towards Accessible Learning: Deep Learning-Based Potential Dysgraphia Detection and OCR for Potentially Dysgraphic Handwriting

Dysgraphia detection through machine learning

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

The Dynamics of Handwriting Improves the Automated Diagnosis of Dysgraphia

Dyslexia and Dysgraphia prediction: A new machine learning approach

Automated Systems For Diagnosis of Dysgraphia in Children: A Survey and Novel Framework

A statistical procedure to assist dysgraphia detection through dynamic modelling of handwriting

Multimodal Ensemble with Conditional Feature Fusion for Dysgraphia Diagnosis in Children from Handwriting Samples

Computer-Aided Diagnosis of Graphomotor Difficulties Utilizing Direction-Based Fractional Order Derivatives

Assessment of Developmental Dysgraphia Utilising a Display Tablet

Motion-Based Handwriting Recognition

Handwriting Anomalies and Learning Disabilities through Recurrent Neural Networks and Geometric Pattern Analysis

Applying Deep Learning to Specific Learning Disorder Screening

Evaluating Handwriting Skills through Human-Machine Interaction: A New Digitalized System for Parameters Extraction

Handwriting quality analysis using online-offline models

Tools and Methods for Diagnosing Developmental Dysgraphia in the Digital Age: A State of the Art

DyGAT: Dynamic Stroke Classification of Online Handwritten Documents and Sketches