A New Technique of the Virtual Reality Visualization of Complex Volume Images from the Computer Tomography and Magnetic Resonance Imaging

Iva Vasic,Roberto Pierdicca,Emanuele Frontoni,Bata Vasic

DOI: https://doi.org/10.1007/978-3-030-87595-4

2023-04-29

Abstract:This paper presents a new technique for the virtual reality (VR) visu-alization of complex volume images obtained from computer tomography (CT) and Magnetic Resonance Imaging (MRI) by combining three-dimensional (3D) mesh processing and software coding within the gaming engine. The method operates on real representations of human organs avoiding any structural ap-proximations of the real physiological shape. In order to obtain realistic repre-sentation of the mesh model, geometrical and topological corrections are per-formed on the mesh surface with preserving real shape and geometric structure. Using mathematical intervention on the 3D model and mesh triangulation the second part of our algorithm ensures an automatic construction of new two-dimensional (2D) shapes that represent vector slices along any user chosen di-rection. The final result of our algorithm is developed software application that allows to user complete visual experience and perceptual exploration of real human organs through spatial manipulation of their 3D models. Thus our pro-posed method achieves a threefold effect: i) high definition VR representation of real models of human organs, ii) the real time generated slices of such a model along any directions, and iii) almost unlimited amount of training data for machine learning that is very useful in process of diagnosis. In addition, our developed application also offers significant benefits to educational process by ensuring interactive features and quality perceptual user experience.

Multimedia

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve the visualization quality of complex volumetric images in virtual reality (VR). These images are sourced from computed tomography (CT) and magnetic resonance imaging (MRI). Specifically, the paper proposes a new technical method aiming to achieve the following three goals:

1. **High - resolution VR representation**: By combining three - dimensional (3D) mesh processing and software coding in game engines, generate high - definition virtual reality representations of real human organs.
2. **Real - time slice generation**: Allow users to generate two - dimensional slices of the model in any direction, thus providing a more flexible viewing perspective.
3. **Training data for machine learning**: Generate a large amount of high - quality training data through the above - mentioned methods for machine learning, especially in the field of medical diagnosis.

### Specific Problems and Solutions

#### 1. **Geometry and Topology Errors**
- **Problem**: 3D models generated by CT and MRI often contain geometry and topology errors, such as isolated vertices, shared geometric primitives, etc. These errors will affect the subsequent visualization effects.
- **Solution**: The paper proposes an algorithm to identify and correct these errors by calculating geometric and topology features. For example, extract the Malformed Vertex Matrix (ME) and the Boundary Vertex Matrix (MB), and repair the model through geometric and topology optimization.

#### 2. **Flexibility of Slice Generation**
- **Problem**: Traditional slicing methods are usually limited to standard axial, coronal, and sagittal planes and cannot meet the observation needs at certain specific angles.
- **Solution**: The paper proposes a new slice - generation algorithm that can generate slices in any direction. By calculating intersections and re - dividing triangular faces, generate high - quality 2D slices.

#### 3. **Optimizing the Model for VR**
- **Problem**: 3D models directly generated from CT and MRI are often not smooth and natural enough in VR applications and need further optimization.
- **Solution**: The paper proposes a series of optimization steps, including mesh simplification, smoothing, texture mapping, and materialization, to ensure that the final model has good visual effects and interactive experiences in VR.

### Experimental Results

The paper verifies the effectiveness of the proposed method through experiments. The experimental results show that the optimized and processed 3D models have higher clarity and more realistic textures in VR. In addition, users can freely explore and learn the structures of human organs through interactive functions, improving the effects of education and diagnosis.

### Conclusion

In summary, this paper solves the deficiencies of existing methods in geometry and topology errors, slice - generation flexibility, and model optimization by proposing a new VR visualization technology, providing strong support for medical diagnosis and education.