Abstract:Causal inference, a cornerstone in disciplines such as economics, genomics, and medicine, is increasingly being recognized as fundamental to advancing the field of robotics. In particular, the ability to reason about cause and effect from observational data is crucial for robust generalization in robotic systems. However, the construction of a causal graphical model, a mechanism for representing causal relations, presents an immense challenge. Currently, a nuanced grasp of causal inference, coupled with an understanding of causal relationships, must be manually programmed into a causal graphical model. To address this difficulty, we present initial results towards a human-centered augmented reality framework for creating causal graphical models. Concretely, our system bootstraps the causal discovery process by involving humans in selecting variables, establishing relationships, performing interventions, generating counterfactual explanations, and evaluating the resulting causal graph at every step. We highlight the potential of our framework via a physical robot manipulator on a pick-and-place task.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the difficult problem of **causal graph construction**, especially how to infer causal relationships from observational data in the field of robotics and apply them to practical tasks. Specifically, the authors propose a human - centered augmented reality (AR) framework to guide and assist in the creation of causal graphs. #### Background and Challenges 1. **Variable identification in complex environments**: - In dynamic environments, it is very difficult to identify variables and their causal relationships, especially when there are confounding variables. 2. **Distinguishing between causality and correlation**: - It is not always clear to distinguish between causal relationships and correlations from observational data. Determining causal relationships usually requires more data support. 3. **High - dimensional variables**: - The number of variables in the environment can be very large, which may lead to complex causal graphs or multiple causal graphs. #### Solutions To solve the above problems, the authors propose the following methods: - **Augmented reality (AR) interface**: Allows operators to directly construct causal graphs in real - time scenes and interact with robots through mechanisms such as virtual sliders and buttons. - **Human - centered approach**: Emphasizes highlighting key information when establishing causal graphs to ensure that robots give priority to human insights. - **Intervention and counterfactual reasoning**: Through human operations, the system can conduct intervention experiments, generate counterfactual explanations, and evaluate the effectiveness of causal graphs. #### Application Examples The paper shows the application potential of this framework through a physical robot performing a "grasping and placing" task. The specific steps include: 1. **Constructing a causal graph**: The operator annotates nodes and edges through the AR interface, sets the content of each node, and adjusts the strength of each edge. 2. **Intervention and adjustment**: According to task performance, for example, if the robot fails to successfully grasp an object, the operator can intervene and optimize performance by modifying object properties (such as weight, texture, etc.). 3. **Real - time feedback**: The system can parse and interpret causal graphs in real - time to ensure that the robot's actions are consistent with its current state. Through this method, researchers hope to simplify the process of constructing causal graphs and improve the adaptability and decision - making ability of robots in complex environments. #### Conclusion The paper shows preliminary results, demonstrating the potential of combining augmented reality technology and human - centered methods in the automated construction of causal graphs. Future research will be dedicated to solving the limitations brought by the simplification of causal graphs and achieving iterative updates of causal graphs. --- Hope the above summary can help you understand the main research problems and solutions of this paper. If you have more questions or need further information, please feel free to let us know!

A Human-Centered Approach for Bootstrapping Causal Graph Creation

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

An Introduction to Causal Inference Methods for Observational Human-Robot Interaction Research

ROS-Causal: A ROS-based Causal Analysis Framework for Human-Robot Interaction Applications

CausalOps -- Towards an Industrial Lifecycle for Causal Probabilistic Graphical Models

Causal Induction from Visual Observations for Goal Directed Tasks

CausalKG: Causal Knowledge Graph Explainability using interventional and counterfactual reasoning

A Visual Analytics Approach for Exploratory Causal Analysis: Exploration, Validation, and Applications

Experimental Evaluation of ROS-Causal in Real-World Human-Robot Spatial Interaction Scenarios

Credible causal inference beyond toy models

Causal Discovery of Dynamic Models for Predicting Human Spatial Interactions

Learning Low-Level Causal Relations using a Simulated Robotic Arm

A Conceptual Model for End-to-End Causal Discovery in Knowledge Tracing

CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning

causalgraph: A Python Package for Modeling, Persisting and Visualizing Causal Graphs Embedded in Knowledge Graphs

Causality is all you need

Sample Efficient Bayesian Learning of Causal Graphs from Interventions

Argumentative Causal Discovery

Interactive and Intelligent Root Cause Analysis in Manufacturing with Causal Bayesian Networks and Knowledge Graphs

Causal discovery using dynamically requested knowledge