Combining object detection and causality mining for efficient development of augmented reality-based on-the-job training systems in hotel management

Gukwon Koo,Namyeon Lee,Ohbyung Kwon

DOI: https://doi.org/10.1080/13614568.2019.1694594

2019-07-03

New Review of Hypermedia and Multimedia

Abstract:The purpose of this study is to propose a methodology for establishing an augmented reality (AR)-based model for efficient OJT through object detection and causality mining, a novel text analysis method. Articles on hotel management published in the last decade and useful for OJT were collected, information on the causal relationships between them extracted, and related rules saved to a rule base. Using the same data, we detected various objects through SSD (Single Shot Multibox Detector), a real-time object detection system. Then we matched sets of causalities and displayed them to trainees wearing AR devices. This methodology reduces development and maintenance costs required to operate OJT programmes. Trainees are immersed in the training environment, which improves the effectiveness of the training. To show the feasibility of the proposed method, we developed a prototype AR-OJT system for hotel management training, automatically extracting knowledge on hotel management from articles according to the proposed method. The results demonstrate that the AR-OJT group shows better performance in terms of learning motivation and self-regulated learning than the control OJT group. No significant difference in learning performance was found between the two groups, which implies that traditional OJT can be substituted with AR-OJT.

computer science, information systems

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to improve the effectiveness and efficiency of on - the - job training (OJT) in the field of hotel management. Specifically, the author proposes a method that combines object detection and causal relationship mining to develop an on - the - job training system based on augmented reality (AR). Traditional OJT has the problems of high cost and long time - consumption, while AR technology can provide an immersive training experience, thereby improving the training effect. In addition, by automatically generating AR content, the cost of development and maintenance can be reduced.

### Main Problems and Solutions

1. **Improve OJT Efficiency**:
- Traditional OJT requires a face - to - face teaching method, which is not only time - consuming but also depends on the experience and knowledge level of the trainer.
- Solution: Utilize AR technology so that the training content can be displayed to trainees in real - time through virtual objects, reducing the dependence on actual trainers.

2. **Reduce Development and Maintenance Costs**:
- The content update and creation of traditional OJT require a large amount of manual operation and are costly.
- Solution: Extract causal relationships and perform object detection through an automated method, and automatically generate AR training content to reduce manual intervention.

3. **Enhance Training Effect**:
- The effect of traditional OJT is often affected by multiple factors, such as the level of the trainer, the participation of trainees, etc.
- Solution: Through the immersive experience provided by AR technology, enhance the sense of participation and learning motivation of trainees, thereby improving the training effect.

### Method Overview

- **Causal Relationship Mining**: Extract causal relationships from hotel management - related literature published in the past decade and store them in a rule base.
- **Object Detection**: Use the SSD (Single Shot Multibox Detector) algorithm to achieve real - time object detection.
- **AR Content Generation**: Match the extracted causal relationships with the detected objects and display them to trainees through AR devices.
- **Experimental Verification**: Verify the effectiveness and feasibility of the AR - OJT system through comparative experiments.

### Experimental Results

The experimental results show that the AR - OJT group performs better than the traditional OJT group in terms of learning motivation and self - regulated learning, but there is no significant difference in learning performance. This indicates that AR - OJT can replace or supplement traditional OJT, especially having an advantage in improving employees' learning motivation and reducing training costs.

### Formula Example

No complex mathematical formulas are involved in the paper, but to meet the requirements, the following is a simple formula representation of the object detection algorithm SSD:

\[
L(x, c, l, g)=\frac{1}{N}\left(L_{conf}(x, c)+\alpha L_{loc}(x, l, g)\right)
\]

where:
- \(L_{conf}(x, c)\) is the classification loss function.
- \(L_{loc}(x, l, g)\) is the localization loss function.
- \(N\) is the number of matched default boxes.
- \(\alpha\) is the weight coefficient.

This formula is used to describe the total loss function in the SSD algorithm, ensuring that the model can optimize classification and localization tasks simultaneously.