Are your apps accessible? A GCN-based accessibility checker for low vision users

Mengxi Zhang,Huaxiao Liu,Shenning Song,Chunyang Chen,Pei Huang,Jian Zhao

DOI: https://doi.org/10.1016/j.infsof.2024.107518

IF: 3.9

2024-06-27

Information and Software Technology

Abstract:Context: Accessibility issues (e.g., small size and narrow interval) in mobile applications (apps) lead to obstacles for billions of low vision users in interacting with Graphical User Interfaces (GUIs). Although GUI accessibility scanning tools exist, most of them perform rule-based check relying on complex GUI hierarchies. This might make them detect invisible redundant information, cannot handle small deviations, omit similar components, and is hard to extend. Objective: In this paper, we propose a novel approach, named ALVIN (Accessibility Checker for Low Vision), which represents the GUI as a graph and adopts the Graph Convolutional Neural Networks (GCN) to label inaccessible components. Method: ALVIN removes invisible views to prevent detecting redundancy and uses annotations from low vision users to handle small deviations. Also, the GCN model could consider the relations between GUI components, connecting similar components and reducing the possibility of omission. ALVIN only requires users to annotate the relevant dataset when detecting new kinds of issues. Results: Our experiments on 48 apps demonstrate the effectiveness of ALVIN, with precision of 83.5%, recall of 78.9%, and F1-score of 81.2%, outperforming baseline methods. In RQ2, the usefulness is verified through 20 issues submitted to open-source apps. The RQ3 also illustrates the GCN model is better than other models. Conclusion: To summarize, our proposed approach can effectively detect accessibility issues in GUIs for low vision users, thereby guiding developers in fixing them efficiently.

computer science, information systems, software engineering