Abstract:Perception occurs when two individuals interpret the same information differently. Despite being a known phenomenon with implications for bias in decision-making, as individual experience determines interpretation, perception remains largely overlooked in machine learning (ML) research. Modern decision flows, whether partially or fully automated, involve human experts interacting with ML applications. How might we then, e.g., account for two experts that interpret differently a deferred instance or an explanation from a ML model? To account for perception, we first need to formulate it. In this work, we define perception under causal reasoning using structural causal models (SCM). Our framework formalizes individual experience as additional causal knowledge that comes with and is used by a human expert (read, decision maker). We present two kinds of causal perception, unfaithful and inconsistent, based on the SCM properties of faithfulness and consistency. Further, we motivate the importance of perception within fairness problems. We illustrate our framework through a series of decision flow examples involving ML applications and human experts.

What problem does this paper attempt to address?

This paper discusses the problem of causal perception, which mainly focuses on how different individuals interpret the same information based on their own experiences in machine learning (ML), resulting in decision biases. The paper points out that although perceptual differences play a role in decision biases, they are often overlooked in ML research. Modern decision processes, such as partially or fully automated processes, involve interaction between human experts and ML applications, making it important to understand and address these perceptual differences. The paper proposes a framework based on Structural Causal Models (SCM) to formalize individual experiences and applies it to causal reasoning for decision-makers. The authors define two types of causal perception: unfaithful and inconsistent, based on the properties of SCM - faithfulness and consistency. Additionally, they emphasize the importance of considering perception in fairness issues. The paper illustrates this framework through a series of decision process examples involving ML applications and human experts. The authors highlight the critical role of perception in building fair and transparent ML applications, as it can impact the outputs of ML applications. They also discuss the role of perception in fairness issues, viewing sensitive attributes (such as gender) as factors that trigger perception due to stereotypes. The main contributions of the paper include: 1) defining perception as a causal reasoning problem for the first time; 2) proposing an intuitive framework applicable to ML problems to structure and infer additional information carried by decision-makers (i.e., personal experiences); and 3) exploring perception as a parameter for modeling fairness and introducing the concept of sensitive attributes. The paper further discusses the motivation behind perception, citing the psychological experiment "Linda Problem" to demonstrate how perception is influenced by sensitive attributes (such as gender) and related stereotypes. Additionally, the paper reviews related work, including cognitive biases, interpretability of artificial intelligence, and the role of humans in iterative decision-making. Finally, the paper introduces the mathematical modeling background of the problem, including the use of SCM to describe causal relationships and defines the two types of perception. The entire framework aims to lay the foundation for future research on how to consider and implement perception in ML decision processes.

Causal Perception

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Causal inference in perception

Causal Machine Learning: A Survey and Open Problems

Perceived Causalities of Physical Events Are Influenced by Social Cues

Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation

Promises and Challenges of Causality for Ethical Machine Learning

Disentangled Representations for Causal Cognition

Social cues help us construct the causal perception of physical events

Subjective Causality

Explaining the Behavior of Black-Box Prediction Algorithms with Causal Learning

When Causality Meets Fairness: A Survey

Learning Perceptual Causality from Video

Causality Learning: A New Perspective for Interpretable Machine Learning

Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making

Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment

How relevant is the prior? Bayesian causal inference for dynamic perception in volatile environments

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Causal Discovery for Fairness

Causal Attention to Exploit Transient Emergence of Causal Effect

Explaining The Efficacy of Counterfactually Augmented Data