Abstract:The collaboration between humans and artificial intelligence (AI) is a significant feature in this digital age. However, humans and AI may have observation, interpretation, and action conflicts when working synchronously. This phenomenon is often masked by faults and, unfortunately, overlooked. This paper systematically introduces the human-AI conflict concept, causes, measurement methods, and risk assessment. The results highlight that there is a potential second decision-maker besides the human, which is the AI; the human-AI conflict is a unique and emerging risk in digitalized process systems; and this is an interdisciplinary field that needs to be distinguished from traditional fault and failure analysis; the conflict risk is significant and cannot be ignored.
What problem does this paper attempt to address?
The paper primarily explores the conflicts that may arise when humans and artificial intelligence (AI) work together, and systematically introduces the concept, causes, measurement methods, and risk assessment of such conflicts. Specifically:
1. **Research Background**: With the development of automation, digitalization, and AI technologies, cooperation between humans and intelligent systems has become increasingly common. However, during collaborative work, differences in observation, interpretation, or actions between humans and AI may lead to conflicts.
2. **Definitions and Concepts**: The paper defines the concept of "human-AI conflict," pointing out that such conflicts arise from different understandings of one or more variables by different participants (human operators and AI) in terms of observation, interpretation, or actions.
3. **Causes of Conflict**: The paper lists the main causes of human-AI conflicts, including sensor failures, cyber-attacks, human errors, and malicious tampering, categorizing these causes into two major types: safety issues and security vulnerabilities.
4. **Measurement Methods**: To quantify the degree of conflict, the authors introduce the variables of Observation Difference (VOD), Interpretation Difference (VID), and Action Difference (VAD), and describe through mathematical formulas how these variables are calculated.
5. **Risk Assessment**: The paper proposes methods for assessing the probability, severity, and risk of conflicts, and discusses how distance affects the likelihood and consequences of conflicts.
6. **Discussion and Conclusion**: The authors point out that human-AI conflict is an emerging research field involving multiple disciplines such as sociology, engineering, computer science, and cognitive science. Additionally, they explore the possibility of resolving or resisting conflicts through advanced control algorithms and emphasize the importance of addressing conflicts from a human perspective.
In summary, this paper aims to systematically introduce the theoretical knowledge related to human-AI conflicts and proposes a methodological framework for assessing and managing such conflicts.