A Comparative Assessment of Technology Acceptance and Learning Outcomes in Computer-based versus VR-based Pedagogical Agents

Aimilios Hadjiliasi,Louis Nisiotis,Irene Polycarpou

2024-10-24

Abstract:As educational technology evolves, the potential of Pedagogical Agents (PAs) in supporting education is extensively explored. Typically, research on PAs has primarily focused on computer-based learning environments, but their use in VR-based environments and integration into education is still in its infancy. To address this gap, this paper presents a mixed method comparative study that has been conducted to evaluate and examine how these computer-based PAs and VR-based PAs compare, towards their learning efficacy and technology acceptance. 92 Computing and Engineering undergraduate students were recruited and participated in an educational experience focusing on computing machinery education. The findings of this study revealed that both approaches can effectively facilitate learning acquisition, and both technologies have been positively perceived by participants toward acceptance, without any significant differences. The findings of this study shed light on the potential of utilizing intelligent PAs to support education, contributing towards the advancement of our understanding of how to integrate such technologies to develop learning interventions, and establishing the foundation for future investigations that aim to successfully integrate and use PAs in education.

Human-Computer Interaction,Emerging Technologies

What problem does this paper attempt to address?

### Problems the paper attempts to solve

This paper aims to explore and compare the differences between computer - based pedagogical agents (PAs) and virtual reality (VR) - based pedagogical agents in terms of learning effectiveness and technology acceptance. Specifically, the researchers hope to fill the following problems through this study:

1. **Technology acceptance**: Evaluate students' acceptance of VR - based PAs and computer - based PAs in terms of usage intention, attitude, and perceived ease - of - use of the technology.
2. **Learning effectiveness**: Compare the impact of these two different forms of PAs on students' knowledge acquisition and learning outcomes.

### Research background

With the development of educational technology, pedagogical agents (PAs) have received extensive attention as a tool to support teaching and learning. However, although a large number of studies have focused on computer - based PAs, research on the application of PAs in VR environments and their supporting role in education is still in its infancy. Therefore, this study aims to explore the application potential of PAs in VR environments and their impact on learning effectiveness and technology acceptance through a mixed - methods comparative experiment.

### Research questions

To achieve the above goals, the researchers proposed the following two research questions:
- **RQ1**: How do VR - based PAs affect knowledge acquisition in immersive learning activities compared to computer - based PAs?
- **RQ2**: What are the differences in participants' perceptions of VR - based PAs and computer - based PAs in terms of effort perception, performance expectation, usage intention, and attitude?

### Method

The study adopted an experimental design, randomly dividing 92 undergraduate students majoring in computing and engineering into two groups: one group used VR - based PAs for learning, and the other group used computer - based PAs for learning. Data were collected through pre - tests and post - tests (including multiple - choice questions), questionnaires (covering the four dimensions of UTAUT: EE, PE, BITU, ATUT), and focus group discussions, and descriptive statistical analysis, non - parametric statistical tests (such as Wilcoxon signed - rank test and Mann - Whitney U test) and thematic analysis were carried out.

### Results

The research results show that:
- Both technologies can effectively promote learning outcomes, and there are no significant differences in learning effectiveness.
- Students have a high acceptance of both technologies. Although VR - based PAs have a slight advantage in some aspects, these differences are not statistically significant.
- Participants generally believe that VR - based PAs can provide a more immersive and interactive learning experience, which helps to enhance learning participation and depth of understanding.

### Conclusions and prospects

This study provides valuable insights for the education field, indicating that PAs have good learning effectiveness and technology acceptance both in computer environments and VR environments. Future research can further explore long - term impacts, application effects on different populations, and other related factors to ensure the successful integration and wide application of these technologies in education.