Using Virtual Reality as a Simulation Tool for Augmented Reality Virtual Windows: Effects on Cognitive Workload and Task Performance

Tianyu Liu,Weiping He,Mark Billinghurst

2024-09-24

Abstract:Virtual content in Augmented Reality (AR) applications can be constructed according to the designer's requirements, but real environments, are difficult to be accurate control or completely reproduce. This makes it difficult to prototype AR applications for certain real environments. One way to address this issue is to use Virtual Reality (VR) to simulate an AR system, enabling the design of controlled experiments and conducting usability evaluations. However, the effectiveness of using VR to simulate AR has not been well studied. In this paper, we report on a user study (N=20) conducted to investigate the impact of using an VR simulation of AR on participants' task performance and cognitive workload (CWL). Participants performed several office tasks in an AR scene with virtual monitors and then again in the VR-simulated AR scene. While using the interfaces CWL was measured with Electroencephalography (EEG) data and a subjective questionnaire. Results showed that frequent visual checks on the keyboard resulted in decreased task performance and increased cognitive workload. This study found that using AR centered on virtual monitor can be effectively simulated using VR. However, there is more research that can be done, so we also report on the study limitations and directions for future work.

Human-Computer Interaction,Graphics

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to evaluate the effectiveness of using virtual reality (VR) to simulate augmented reality (AR) environments, especially the impact of such simulations on users' task performance and cognitive workload (CWL, Cognitive Workload). Specifically, the researchers developed a virtual reality reproduction system (VRRS) to reproduce the background environment and interactive content in AR office scenes through VR, and compared users' task performance and cognitive workload in the real AR environment and the simulated VR environment.

### Research Background

With the development of augmented reality (AR) technology, AR head - mounted displays (HMD) can superimpose virtual information onto the physical environment, allowing users to create and destroy virtual displays as needed. However, the computing recognition, positioning, and tracking technologies of AR devices have error and delay problems, and environmental conditions may affect the AR effect. In addition, when conducting experimental design and usability evaluation in complex and changeable real - world environments, it is difficult to control variables and repeat experiments. Therefore, researchers have proposed using virtual reality (VR) systems to simulate AR environments to overcome these limitations.

### Research Objectives

The main objective of this study is to verify the feasibility of using VR to simulate AR environments and evaluate its effectiveness in environments with different degrees of interference. Specifically:

1. **Evaluate similarities and differences**: Compare the cognitive workload of users when using VRSS and AR HMD.
2. **Discuss usability and other aspects**: Explore the differences between the two in terms of user experience and other aspects.
3. **Provide improvement suggestions**: Provide some suggestions for using VR systems to reproduce AR environments.

### Experimental Design

To achieve the above - mentioned objectives, the researchers carried out the following work:

- **Construct VRRS**: Developed a virtual reality reproduction system (VRRS) that can reproduce the background environment and interactive content in AR office scenes.
- **Task setting**: Designed a series of office tasks, including typing, searching, mathematical calculation, and drawing tasks.
- **User study**: Recruited 20 participants to complete the same tasks under AR and VRSS conditions respectively, and recorded their task performance and cognitive workload (measured by EEG data and subjective questionnaires).

### Results and Conclusions

Through comparative analysis, the study found that:

- In a low - interference environment, the typing speed under AR conditions is significantly higher than that under VRSS conditions, and the number of backspace operations is fewer.
- In a high - interference environment, the typing speed under AR conditions is also significantly higher than that under VRSS conditions.
- There are no significant differences in the performance of other tasks under the two conditions.
- EEG data shows that in a low - interference environment, the cognitive workload of the typing task under VRSS conditions is higher; while in a high - interference environment, the difference in cognitive workload between the two conditions is smaller.

In summary, the study shows that using VR to simulate AR environments is feasible for some tasks, but there are still certain gaps in fine - grained tasks such as typing. Future research can further optimize the effect of VR simulation to better reproduce the AR experience.

### Formula Presentation

When describing cognitive workload, the paper mentions:

\[
\text{CWL}=\frac{\text{Brain resource utilization rate}}{\text{Task requirements}}
\]

Among them, brain resource utilization rate can be measured by the changes in the frequencies of θ - waves (4 - 8Hz) and α - waves (8 - 12Hz) in EEG data. Specifically, as the cognitive workload increases, the power of θ - waves increases while the power of α - waves decreases. The formulas are expressed as follows:

\[
\Delta\theta=\theta_{\text{VRSS}}-\theta_{\text{AR}}
\]
\[
\Delta\alpha=\alpha_{\text{AR}}-\alpha_{\text{VRSS}}
\]

These formulas are used to quantify the changes in cognitive workload under different conditions.