Automated Assessment of Android Exercises with Cloud-native Technologies

Daniel Bruzual,Maria L. Montoya Freire,Mario Di Francesco

DOI: https://doi.org/10.1145/3341525.3387430

2020-06-02

Abstract:Mobile applications are very challenging to test as they usually have a complex graphical user interface and advanced functionality that involves interacting with remote services. Due to these features, student assessment in courses about mobile application development usually relies on assignments or projects that are manually checked by teaching assistants for grading. This approach clearly does not scale to large classrooms, especially for online courses. This article presents a novel system for automated assessment of Android exercises with cloud-native technologies. Different from the state of the art, the proposed solution leverages a mobile app testing framework that is largely used in the industry instead of custom libraries. Furthermore, the devised system employs software containers and scales with the availability of resources in a data center, which is essential for massive open online courses. The system design and implementation is detailed, together with the results from a deployment within a master-level course with 120 students. The received feedback demonstrates that the proposed solution was effective, as it provided insightful feedback and supported independent learning of mobile application development.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges faced in student assignment assessment in mobile application development courses. Specifically:

1. **Limitations of Manual Assessment**: Currently, most courses on mobile application development rely on teaching assistants to manually check students' assignments or projects for grading. This method has poor scalability when faced with large - scale classes, especially in online courses. Manual assessment is not only time - consuming and labor - intensive, but also prone to subjective biases, affecting the fairness and accuracy of grading.

2. **Need for Automated Assessment**: Although automated assessment has been widely used in other areas of computer science education, especially in introductory programming courses, there is less research and practice for more advanced courses, such as mobile application development. Therefore, an automated system that can effectively assess complex mobile applications (especially Android applications) is needed to support the teaching needs of large - scale online courses.

To this end, the paper proposes an automated assessment system based on cloud - native technology for evaluating Android exercises. The main features of this system include:

- **Using Industry - Standard Test Frameworks**: Unlike existing methods, this system utilizes mobile application test frameworks widely used in the industry rather than custom libraries.
- **Software Containers and Resource Scaling**: By using software containers, the system can scale according to the resource availability in the data center, which is especially important for large - scale open online courses.
- **Providing Detailed Feedback**: The system can not only automatically assess students' submissions, but also provide multi - level feedback information, including test results, debugging information, error stack traces, and application screenshots, helping students understand the submission results, especially in case of failure.

In summary, this paper aims to solve the challenges of student assignment assessment in mobile application development courses by developing an efficient, scalable, and rich - feedback automated assessment system, thereby improving teaching quality and students' learning experience.