Artificial Intelligence for Biomedical Video Generation

Linyuan Li,Jianing Qiu,Anujit Saha,Lin Li,Poyuan Li,Mengxian He,Ziyu Guo,Wu Yuan

2024-11-12

Abstract:As a prominent subfield of Artificial Intelligence Generated Content (AIGC), video generation has achieved notable advancements in recent years. The introduction of Sora-alike models represents a pivotal breakthrough in video generation technologies, significantly enhancing the quality of synthesized videos. Particularly in the realm of biomedicine, video generation technology has shown immense potential such as medical concept explanation, disease simulation, and biomedical data augmentation. In this article, we thoroughly examine the latest developments in video generation models and explore their applications, challenges, and future opportunities in the biomedical sector. We have conducted an extensive review and compiled a comprehensive list of datasets from various sources to facilitate the development and evaluation of video generative models in biomedicine. Given the rapid progress in this field, we have also created a github repository to regularly update the advances of biomedical video generation at: <a class="link-external link-https" href="https://github.com/Lee728243228/Biomedical-Video-Generation" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve the quality and practicality of biomedical video generation. Specifically, the author focuses on the following aspects:

1. **Understanding physical laws**: In order to enhance the realism and accuracy of synthetic videos, it is necessary to simulate and learn the physical phenomena in the video content, such as object movement, the effect of force, and the interaction between objects. Especially in surgical operations, which involve using jointed instruments to operate on deformable tissues and organs to achieve the expected results, although existing video - generation models can create surgical scenes, they fail to well simulate the coherence of these operations. In addition, further study of physiology and pathology knowledge is required to more accurately understand the movement characteristics in biomedical videos.

2. **Establishing effective evaluation criteria and benchmarks**: Besides considering the consistency and authenticity of the generated content, it is also necessary to ensure the medical practicality and applicability of the generated content and its value to existing biomedical data. Therefore, when designing evaluation criteria, it is also necessary to consider whether the biomedical knowledge contained in the generated content is meaningful and can meet the needs of medical practice.

3. **Enhancing the controllability and interpretability of generation**: The generated videos can be used for multiple medical purposes, such as auxiliary diagnosis or education. To achieve this, the generated information needs to be precisely controlled, which means that the generation model needs to be not only controllable but also interpretable. Although control mechanisms such as ControlNet have proven their applicability in medical generation, many problems remain unsolved.

By addressing the above challenges, the paper aims to promote the development of biomedical video - generation technology and bring innovation and improvement to the biomedical and medical health fields.