AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems

Clara Punzi, Roberto Pellungrini, Mattia Setzu, Fosca Giannotti, Dino Pedreschi

2024-02-10

Abstract:Everyday we increasingly rely on machine learning models to automate and support high-stake tasks and decisions. This growing presence means that humans are now constantly interacting with machine learning-based systems, training and using models everyday. Several different techniques in computer science literature account for the human interaction with machine learning systems, but their classification is sparse and the goals varied. This survey proposes a taxonomy of Hybrid Decision Making Systems, providing both a conceptual and technical framework for understanding how current computer science literature models interaction between humans and machines.

Machine Learning,Human-Computer Interaction,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the interaction issues between artificial intelligence systems and human decision-makers and proposes a classification method for hybrid decision systems. With the widespread application of machine learning models in high-risk tasks, human interaction with these models has become increasingly frequent. However, this interaction faces issues such as trust, misuse, and abandonment. Additionally, in critical fields such as medicine, finance, and law, AI systems need to collaborate with human experts as decision support systems rather than completely replacing them. These "collaborative" decision systems face the same issues and also need to address effective communication channels and the integration between human and AI systems.

The paper proposes a classification framework for hybrid decision systems to help understand how the interaction between humans and machines is modeled in the current computer science literature. Hybrid systems typically include two different types of agents: human agents with unique reasoning abilities and common-sense knowledge, and machine agents with powerful computational capabilities. Due to the heterogeneity between these two types of agents, they often have disagreements when solving problems, and traditional evaluation metrics are difficult to measure these disagreements. Therefore, the paper introduces three different paradigms of hybrid systems (human supervision, learning to defer, and co-learning) to explore how to better integrate the capabilities of humans and machines, thereby improving the quality, reliability, and transparency of overall decision-making.