MotorEase: Automated Detection of Motor Impairment Accessibility Issues in Mobile App UIs

Arun Krishnavajjala,SM Hasan Mansur,Justin Jose,Kevin Moran

DOI: https://doi.org/10.1145/3597503.3639167

2024-03-20

Abstract:Recent research has begun to examine the potential of automatically finding and fixing accessibility issues that manifest in software. However, while recent work makes important progress, it has generally been skewed toward identifying issues that affect users with certain disabilities, such as those with visual or hearing impairments. However, there are other groups of users with different types of disabilities that also need software tooling support to improve their experience. As such, this paper aims to automatically identify accessibility issues that affect users with motor-impairments.

Software Engineering,Computer Vision and Pattern Recognition,Human-Computer Interaction

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the accessibility problems in mobile application user interfaces (UIs) for users with motor disabilities. Specifically, the authors focus on automatically detecting and fixing the accessibility problems that users with motor disabilities encounter when using mobile applications.

#### Background and Motivation

Current research mainly focuses on identifying and fixing the accessibility problems of users with visual or auditory disabilities, while relatively little support is provided for users with motor disabilities. Due to hand tremors or limited motor abilities, users with motor disabilities may not be able to interact with touch - screen devices as easily as ordinary users, so they need assistive devices (such as switches) to help them operate. However, existing development tools and technologies often lack consideration and support for such users.

#### Research Objectives

To improve this situation, the paper proposes a new method named **MotorEase** for automatically detecting the accessibility problems in mobile application UIs that affect users with motor disabilities. By combining computer vision and text - processing techniques, MotorEase can understand and analyze the functional and visual properties of the application UI screens, thereby identifying violations of specific accessibility design guidelines. These guidelines mainly include:

1. **Visual Touch Target Size**: Ensure that icons and buttons are large enough for users to click.
2. **Expanding Section Closure**: Ensure that pop - up menus or dialog boxes have a clear way to close.
3. **Persistent Element Location**: Ensure that the positions of icons or buttons that appear across multiple screens are consistent.
4. **Adjacent Icon Visual Distance**: Ensure that there is sufficient spacing between adjacent clickable elements to prevent accidental touches.

#### Main Contributions

1. **MotorEase Tool**: A novel automated tool that can accurately detect the accessibility problems of users with motor disabilities.
2. **MotorCheck Benchmark Dataset**: It contains 555 screen examples that violate the accessibility guidelines and 1044 screen examples that comply with the guidelines, from 70 popular applications.
3. **Empirical Evaluation Results**: It shows the effectiveness of MotorEase in detecting motor - disability - related accessibility problems, with an average accuracy rate of approximately 90% and a false - positive rate of less than 9%.
4. **Reproducibility Package**: It provides the code of MotorEase, the benchmark dataset, and the experimental infrastructure to promote further research.

Through these contributions, the paper not only fills the gaps in existing research but also provides developers with a practical tool to help them better support the needs of users with motor disabilities.