Abstract:Artificial intelligence (AI) and human-machine interaction (HMI) are two keywords that usually do not fit embedded applications. Within the steps needed before applying AI to solve a specific task, HMI is usually missing during the AI architecture design and the training of an AI model. The human-in-the-loop concept is prevalent in all other steps of developing AI, from data analysis via data selection and cleaning to performance evaluation. During AI architecture design, HMI can immediately highlight unproductive layers of the architecture so that lightweight network architecture for embedded applications can be created easily. We show that by using this HMI, users can instantly distinguish which AI architecture should be trained and evaluated first since a high accuracy on the task could be expected. This approach reduces the resources needed for AI development by avoiding training and evaluating AI architectures with unproductive layers and leads to lightweight AI architectures. These resulting lightweight AI architectures will enable HMI while running the AI on an edge device. By enabling HMI during an AI uses inference, we will introduce the AI-in-the-loop concept that combines AI's and humans' strengths. In our AI-in-the-loop approach, the AI remains the working horse and primarily solves the task. If the AI is unsure whether its inference solves the task correctly, it asks the user to use an appropriate HMI. Consequently, AI will become available in many applications soon since HMI will make AI more reliable and explainable.

What problem does this paper attempt to address?

The problem this paper attempts to address is: With the continuous deepening of artificial intelligence (AI) architectures, especially the exponential growth in the number of parameters in convolutional neural networks (CNNs), the demand for hardware resources has increased. This makes it difficult to apply complex AI architectures on embedded devices. Additionally, during the AI architecture design phase, there is often a lack of human-machine interaction (HMI) support, leading to resource waste and inefficient AI architectures. To address these issues, the paper proposes the following solutions: 1. **Introduction of intuitive human-machine interaction (HMI)**: By introducing HMI during the AI architecture design phase, users can instantly identify which layers do not contribute to network performance (i.e., non-productive layers), thereby avoiding the training and evaluation of these ineffective layers. This helps create lightweight AI architectures and reduces resource consumption. 2. **Use of Receptive Field Analysis (RFA) tools**: An easy-to-use RFA toolbox has been developed, which graphically visualizes and marks non-productive layers and boundary layers. Users can optimize AI architectures through an intuitive color-coded interface without the need for actual training. 3. **User evaluation and feedback**: The effectiveness of the RFA toolbox was validated through user testing. Experimental results show that after using the RFA toolbox, users can more quickly and accurately identify the optimal AI architecture, reducing unnecessary training and evaluation steps. 4. **AI-in-the-loop concept**: An AI-in-the-loop concept that combines the advantages of AI and humans is proposed. In this mode, AI is primarily responsible for task execution, but in uncertain situations or safety-critical scenarios, it requests user assistance through appropriate HMI. This not only improves the reliability and interpretability of AI systems but also enables AI to be applied in more embedded applications. In summary, the paper aims to address the resource constraints of applying complex AI architectures on embedded devices and improve the reliability and interpretability of AI systems by introducing HMI and optimizing AI architecture design.

AI-in-the-Loop -- The impact of HMI in AI-based Application

The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures

An HCAI Methodological Framework: Putting It Into Action to Enable Human-Centered AI

Human-in-the-loop machine learning: a state of the art

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

The Model Mastery Lifecycle: A Framework for Designing Human-AI Interaction

Human-in-the-loop Techniques in Machine Learning.

Enabling Human-Centered AI: A Methodological Perspective

Improving the Efficiency of Human-in-the-Loop Systems: Adding Artificial to Human Experts

Human in the Loop for Machine Creativity

Human-AI Interaction in Industrial Robotics: Design and Empirical Evaluation of a User Interface for Explainable AI-Based Robot Program Optimization

A Survey of Human-in-the-loop for Machine Learning

What should I say? -- Interacting with AI and Natural Language Interfaces

Human-to-AI Coach: Improving Human Inputs to AI Systems

Designing AI to Work WITH or FOR People?

On AI-Inspired UI-Design

AI from concrete to abstract: demystifying artificial intelligence to the general public

Platform Supporting Intelligent Human–Machine Interface (HMI) Applications for Smart Machine Tools

Evaluating Human-AI Collaboration: A Review and Methodological Framework

Designing Human-Machine Interface for Autonomous Vehicles

Enhancing Human-Robot Collaboration in Industry 4.0 with AI-driven HRI