Abstract:Resilience in autonomous agent systems is about having the capacity to anticipate, respond to, adapt to, and recover from adverse and dynamic conditions in complex environments. It is associated with the intelligence possessed by the agents to preserve the functionality or to minimize the impact on functionality through a transformation, reconfiguration, or expansion performed across the system. Enhancing the resilience of systems could pave way toward higher autonomy allowing them to tackle intricate dynamic problems. The state-of-the-art systems have mostly focussed on improving the redundancy of the system, adopting decentralized control architectures, and utilizing distributed sensing capabilities. While machine learning approaches for efficient distribution and allocation of skills and tasks have enhanced the potential of these systems, they are still limited when presented with dynamic environments. To move beyond the current limitations, this paper advocates incorporating counterfactual learning models for agents to enable them with the ability to predict possible future conditions and adjust their behavior. Counterfactual learning is a topic that has recently been gaining attention as a model-agnostic and post-hoc technique to improve explainability in machine learning models. Using counterfactual causality can also help gain insights into unforeseen circumstances and make inferences about the probability of desired outcomes. We propose that this can be used in agent systems as a means to guide and prepare them to cope with unanticipated environmental conditions. This supplementary support for adaptation can enable the design of more intelligent and complex autonomous agent systems to address the multifaceted characteristics of real-world problem domains.

The Lessons We (Don't) Learn: Counterfactual Thinking and Organizational Accountability after a Close Call

The Sin of Prediction: When Mentally Simulated Alternatives Compete With Reality

Improving workplace safety by thinking about what might have been: A first look at the role of counterfactual thinking

Counterfactual thoughts in complex causal domain: content, benefits, and implications for their function

An improved probabilistic account of counterfactual reasoning.

Improving Workplace Safety by Thinking About What Might Have Been

Effects on and consequences of responses to errors: Results from two experimental studies

Learning from Mistakes: A Thematic Analysis of Flight 401's Situational Awareness Lapses

Learning what they think vs. learning what they do: The micro-foundations of vicarious learning

Counterfactual thinking in cooperation dynamics

Culpable Control and Counterfactual Reasoning in the Psychology of Blame

Learning From Lessons Learned: Preliminary Findings From a Study of Learning From Failure

Counterfactual curiosity: motivated thinking about what might have been

Foolishness without consequence? From physical to virtual modeling in the history of military aircraft development at Saab

Confronting failure: antecedents and consequences of shared beliefs about failure in organizational work groups

How audits fail according to accident investigations: A counterfactual logic analysis

Counterfactual learning in enhancing resilience in autonomous agent systems

Cognitive Style and Flight Experience Influence on Confirmation Bias in Lost Procedures

What to Blame? Self-Serving Attribution Bias with Multi-Dimensional Uncertainty

Exploring the Influence of Failure Aversion on Learning From Project Failure: A Sensemaking Perspective

Counterfactual simulation in causal cognition