Abstract:Tangible User Interfaces (TUI) for human--computer interaction (HCI) provide the user with physical representations of digital information with the aim to overcome the limitations of screen-based interfaces. Although many compelling demonstrations of TUIs exist in the literature, there is a lack of research on TUIs intended for daily two-handed tasks and processes, such as cooking. In response to this gap, we propose SPICE (Smart Projection Interface for Cooking Enhancement). SPICE investigates TUIs in a kitchen setting, aiming to transform the recipe following experience from simply text-based to tangibly interactive. SPICE includes a tracking system, an agent-based software, and vision large language models to create and interpret a kitchen environment where recipe information is projected directly onto the cooking surface. We conducted a comparative usability study of SPICE and text-based recipe following with 30 participants, assessing the task difficulty, total duration, and efficiency, as well as user confidence and taste perception. The results indicate that SPICE allowed participants to perform the recipe with less stops and in shorter time while also improving self-reported efficiency, confidence, and taste. Despite this, participants self-reported no change in overall difficulty, which is a direction for future research. Overall, the SPICE project demonstrates the potential of using TUIs to improve everyday activities, paving the way for future research in HCI and new computing interfaces.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problem of the insufficient application of existing user interfaces (UIs) in daily two - handed operation tasks (such as cooking). Specifically, although many studies have demonstrated the potential of Tangible User Interfaces (TUI), in - depth research on its application in daily two - handed operation tasks is still lacking. To solve this problem, the author proposes SPICE (Smart Projection Interface for Cooking Enhancement), a smart projection interface for enhancing the cooking experience. #### Main problems: 1. **Limitations of existing UIs**: Traditional screen - based user interfaces (GUIs) have limitations in tasks that require two - handed operation (such as cooking) and cannot provide an intuitive and interactive experience. 2. **Information gap in the cooking process**: During the cooking process, users usually rely on static text - or image - based recipes, which is clearly disjointed from the actual hands - on operation, resulting in problems such as inefficiency and frequent interruptions. 3. **Insufficient application of TUI in daily tasks**: Although TUI performs well in laboratory environments, its application and effectiveness in daily life, especially in tasks involving two - handed operation, have not been fully verified. #### Goals of SPICE: - **Enhance user experience**: By directly projecting digital information onto the cooking surface, users can follow the recipe more intuitively and reduce operation interruptions. - **Improve efficiency and confidence**: Experimental results show that using SPICE can reduce the number of pauses during the cooking process, shorten the total time, and increase users' confidence and satisfaction with the taste of the dish. - **Explore the application of TUI in daily life**: Through the study of the kitchen environment, explore the application potential of TUI in other high - risk environments that require two - handed operation (such as manufacturing, medical, education, etc.). In conclusion, by developing and evaluating the SPICE system, this paper aims to fill the research gap of TUI in daily two - handed operation tasks and demonstrate its potential in improving user interaction experience.

SPICE: Smart Projection Interface for Cooking Enhancement

Augmented Reality Based Interactive Cooking Guide

The Influence of User-Tracking Feedback Format on Gestural Interaction's User Experience:A Kinect-Based Usability Study

A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking Instructions

A Contextual Inquiry of People with Vision Impairments in Cooking

VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks

Characterizing Barriers and Technology Needs in the Kitchen for Blind and Low Vision People

Tangible user interface: origins, development, and future trends

SONOICE! a Sonar-Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers

MOSAIC: A Modular System for Assistive and Interactive Cooking

Learning Science in an Intuitive and Fun Way Using Multi-touch Technology

Evaluating Object Manipulation Interaction Techniques in Mixed Reality: Tangible User Interfaces and Gesture

"If you've gone straight, now, you must turn left" - Exploring the use of a tangible interface in a collaborative treasure hunt for people with visual impairments

Cooking with Conversation: Enhancing User Engagement and Learning with a Knowledge-Enhancing Assistant

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch

RecipeMind: Guiding Ingredient Choices from Food Pairing to Recipe Completion using Cascaded Set Transformer

Assessment of Application Technology of Natural User Interfaces in the Creation of a Virtual Chemical Laboratory

CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision

TactCube: An Intelligent Device to 'converse' with Smart Environments

Taste Your Emotions: An Exploration of the Relationship between Taste and Emotional Experience for HCI

OnionBot: A System for Collaborative Computational Cooking