Abstract:Background and Objective Tongue diagnosis is one of the characteristics of traditional Chinese medicine (TCM), but traditional tongue diagnosis is affected by many factors, and its differential diagnosis results are not widely recognized. The appearance of tongue diagnosis instruments is the product of the modernization of tongue diagnosis, and it has standard and objective advantages in clinical practice. In this study, based on standard tongue images, a tongue image dataset and detection model were constructed. And based on the deep learning convolutional neural network (CNN) algorithm and visual question answering technology, a human-computer interaction intelligent health detector for tongue image recognition is constructed. Methods In this research, 1420 tongue images were collected. After screening, experts judged them, and annotated the tongue images to form tongue image datasets. Then the artificial intelligence network framework based on deep learning convolutional neural network (CNN), that is, ResNet34, is applied to this dataset to automatically extract image features and realize tongue images classification. Finally, the VGG16 network framework is applied to the dataset to compare the classification model and compare with the classification effect. Results In this paper, relevant datasets were formed by collating the tongue images collected by annotation, which verified that the ResNet34 architecture could better perform the task of tooth mark and tongue feature recognition. Compared with similar learning tasks in existing studies, the accuracy of the teeth-printed tongue recognition model proposed in this study is more than 10% higher, which indicates that the CNN algorithm can distinguish teeth-printed tongue more accurately and effectively. At the same time, using datasets and models combined with visual question and answer technology, an AI health detector for TCM tongue image identification is designed, which can make health assessments and give suggestions to users. Conclusion This study adopts a convolutional neural network model based on deep learning, which can reduce the extraction of tongue features more quickly and conveniently. At the same time, the model architecture has excellent performance and strong generalization ability and is more accurate in judging users' health status.

Research on the model of automatic recognition and natural language question-answer system for traditional Chinese medicine tongue images based on LLMs

TCMChat: A Generative Large Language Model for Traditional Chinese Medicine

A Joint Model For Question-Answering Over Traditional Chinese Medicine

A Question-Answering System over Traditional Chinese Medicine

Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models

Intelligent quality control of traditional chinese medical tongue diagnosis images based on deep learning

MedChatZH: A tuning LLM for traditional Chinese medicine consultations

Enhancing Healthcare through Large Language Models: A Study on Medical Question Answering

A survey of artificial intelligence in tongue image for disease diagnosis and syndrome differentiation

Evaluating Large Language Models for Radiology Natural Language Processing

Human-computer interaction based health diagnostics using ResNet34 for tongue image classification

An image analysis system for tongue diagnosis in traditional chinese medicine

TCMD: A Traditional Chinese Medicine QA Dataset for Evaluating Large Language Models

An Automatic Clinical Model for Chinese Medicine Based on Interpretable Machine Learning Algorithms

Study on method of automatic diagnosis of tongue feature in traditional Chinese medicine

TCMBench: A Comprehensive Benchmark for Evaluating Large Language Models in Traditional Chinese Medicine

Understanding the concerns and choices of public when using large language models for healthcare

Towards Leveraging Large Language Models for Automated Medical Q&A Evaluation

ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models

An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models

Understanding natural language: Potential application of large language models to ophthalmology