Learning to Substitute Ingredients in Recipes

Bahare Fatemi,Quentin Duval,Rohit Girdhar,Michal Drozdzal,Adriana Romero-Soriano

DOI: https://doi.org/10.48550/arXiv.2302.07960

2023-02-16

Abstract:Recipe personalization through ingredient substitution has the potential to help people meet their dietary needs and preferences, avoid potential allergens, and ease culinary exploration in everyone's kitchen. To address ingredient substitution, we build a benchmark, composed of a dataset of substitution pairs with standardized splits, evaluation metrics, and baselines. We further introduce Graph-based Ingredient Substitution Module (GISMo), a novel model that leverages the context of a recipe as well as generic ingredient relational information encoded within a graph to rank plausible substitutions. We show through comprehensive experimental validation that GISMo surpasses the best performing baseline by a large margin in terms of mean reciprocal rank. Finally, we highlight the benefits of GISMo by integrating it in an improved image-to-recipe generation pipeline, enabling recipe personalization through user intervention. Quantitative and qualitative results show the efficacy of our proposed system, paving the road towards truly personalized cooking and tasting experiences.

Machine Learning,Human-Computer Interaction

What problem does this paper attempt to address?

This paper attempts to address the problem of ingredient substitution in recipes to achieve recipe personalization. Specifically, the authors aim to meet people's dietary needs and preferences, avoid potential allergens, and simplify culinary exploration in the kitchen through ingredient substitution. To achieve this goal, the paper constructs a benchmark dataset (including ingredient substitution pairs, standardized dataset splits, evaluation metrics, and baseline models) and proposes a new model based on graph neural networks—GISMo (Graph-based Ingredient Substitution Module). This model utilizes the contextual information of recipes and general ingredient relationship information to predict reasonable ingredient substitutions.

### Main Contributions of the Paper:
1. **Creation of a Benchmark Dataset**: Constructed a benchmark dataset named Recipe1MSubs, which includes valid ingredient substitution pairs and their corresponding recipes.
2. **Proposed a New Model GISMo**: Developed a graph neural network-based ingredient substitution model GISMo, which combines recipe-specific contextual information and general ingredient relationship information, significantly improving the performance of the ingredient substitution task.
3. **Extended the Inverse Cooking Pipeline**: Integrated GISMo into the inverse cooking system, enabling the generation of personalized recipes through user intervention, and validated through extensive experiments that the generated recipes effectively incorporate the modifications suggested by GISMo.

### Problems Addressed:
- **Ingredient Substitution**: How to find suitable ingredient substitutions in a given recipe to meet individual dietary needs and preferences.
- **Recipe Personalization**: How to generate personalized recipes through ingredient substitution and adjustment of cooking instructions.
- **Dataset and Evaluation Standards**: The lack of high-quality ingredient substitution datasets and evaluation standards, which hinders the development of related research.

Through these contributions, the paper paves the way for achieving truly personalized cooking and tasting experiences.