An Exploratory Approach to Deriving Nutrition Information of Restaurant Food from Crowdsourced Food Images: Case of Hartford

Xiang Chen,Evelyn Johnson,Aditya Kulkarni,Caiwen Ding,Natalie Ranelli,Yanyan Chen,Ran Xu

DOI: https://doi.org/10.3390/nu13114132

IF: 5.9

2021-11-18

Nutrients

Abstract:Deep learning models can recognize the food item in an image and derive their nutrition information, including calories, macronutrients (carbohydrates, fats, and proteins), and micronutrients (vitamins and minerals). This technology has yet to be implemented for the nutrition assessment of restaurant food. In this paper, we crowdsource 15,908 food images of 470 restaurants in the Greater Hartford region on Tripadvisor and Google Place. These food images are loaded into a proprietary deep learning model (Calorie Mama) for nutrition assessment. We employ manual coding to validate the model accuracy based on the Food and Nutrient Database for Dietary Studies. The derived nutrition information is visualized at both the restaurant level and the census tract level. The deep learning model achieves 75.1% accuracy when compared with manual coding. It has more accurate labels for ethnic foods but cannot identify portion sizes, certain food items (e.g., specialty burgers and salads), and multiple food items in an image. The restaurant nutrition (RN) index is further proposed based on the derived nutrition information. By identifying the nutrition information of restaurant food through crowdsourced food images and a deep learning model, the study provides a pilot approach for large-scale nutrition assessment of the community food environment.

nutrition & dietetics

What problem does this paper attempt to address?

The main problem this paper attempts to address is the extraction of nutritional information of restaurant foods from crowdsourced food images using deep learning models, in order to assess the nutritional status of foods in community dining environments. Specifically, the researchers focus on how to efficiently and on a large scale obtain nutritional data of restaurant foods through technological means. This data includes calories, macronutrients (carbohydrates, fats, and proteins), and micronutrients (vitamins and minerals). The study not only aims to improve the efficiency of nutritional assessment but also hopes to reveal food nutrition inequalities between different communities through tools such as nutrition maps and the Restaurant Nutrition (RN) index, thereby providing a basis for health policy formulation and health promotion.

The research background points out that with the increasing frequency of dining out, nutritional assessment of restaurant foods has become particularly important, as such foods often have higher calories and lower nutritional value compared to home-cooked meals, potentially leading to an increased risk of obesity and other chronic diseases. However, traditional nutritional assessment methods face many challenges when applied on a large scale, such as high costs and time consumption. Therefore, the researchers explored a new method that combines deep learning models with crowdsourced food images to overcome these limitations and provide a new solution for large-scale nutritional assessment.