Fine-grained food image classification and recipe extraction using a customized deep neural network and NLP
Razia Sulthana Abdul Kareem,Timothy Tilford,Stoyan Stoyanov
DOI: https://doi.org/10.1016/j.compbiomed.2024.108528
IF: 7.7
2024-05-08
Computers in Biology and Medicine
Abstract:Global eating habits cause health issues leading people to mindful eating. This has directed attention to applying deep learning to food-related data. The proposed work develops a new framework integrating neural network and natural language processing for classification of food images and automated recipe extraction. It address the challenges of intra-class variability and inter-class similarity in food images that have received shallow attention in the literature. Firstly, a customized lightweight deep convolution neural network model, MResNet-50 for classifying food images is proposed. Secondly, automated ingredient processing and recipe extraction is done using natural language processing algorithms: Word2Vec and Transformers in conjunction. Thirdly, a representational semi-structured domain ontology is built to store the relationship between cuisine, food item, and ingredients. The accuracy of the proposed framework on the Food-101 and UECFOOD256 datasets is increased by 2.4% and 7.5%, respectively, outperforming existing models in literature such as DeepFood, CNN-Food, Wiser, and other pre-trained neural networks.
engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology