Worst-risk minimization in generalized structural equation models

Philip Kennerberg,Ernst C. Wit

2024-07-23

Abstract:We consider rather general structural equation models (SEMs) between a target and its covariates in several shifted environments. Given $k\in\N$ shifts we consider the set of shifts that are at most $\gamma$-times as strong as a given weighted linear combination of these $k$ shifts and the worst (quadratic) risk over this entire space. This worst risk has a nice decomposition which we refer to as the "worst risk decomposition". Then we find an explicit arg-min solution that minimizes the worst risk and consider its corresponding plug-in estimator which is the main object of this paper. This plug-in estimator is (almost surely) consistent and we first prove a concentration in measure result for it. The solution to the worst risk minimizer is rather reminiscent of the corresponding ordinary least squares solution in that it is product of a vector and an inverse of a Grammian matrix. Due to this, the central moments of the plug-in estimator is not well-defined in general, but we instead consider these moments conditioned on the Grammian inverse being bounded by some given constant. We also study conditional variance of the estimator with respect to a natural filtration for the incoming data. Similarly we consider the conditional covariance matrix with respect to this filtration and prove a bound for the determinant of this matrix. This SEM model generalizes the linear models that have been studied previously for instance in the setting of casual inference or anchor regression but the concentration in measure result and the moment bounds are new even in the linear setting.

Statistics Theory,Probability

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper primarily investigates how to minimize the worst risk in Generalized Structural Equation Models (SEMs) across multiple shifted environments. Specifically, the paper considers the variations in the relationship between the target variable and its covariates in different environments and attempts to find a solution that can minimize the worst risk in these environments.

### Summary of Main Content

1. **Model Introduction**:
- The authors consider a more general structural equation model where the relationship between the target variable and covariates can be random.
- The model includes an observational environment and multiple shifted environments.

2. **Worst Risk Decomposition**:
- The authors define a "worst risk decomposition," which decomposes the worst risk into a linear combination of observational risk and the risks of multiple shifted environments.
- Through this decomposition, the authors find an explicit arg-min solution to minimize the worst risk.

3. **Plug-in Estimator**:
- Based on the above solution, the authors propose a plug-in estimator and prove that this estimator is consistent in the almost sure sense.
- The authors also prove the concentration in measure result for this estimator.

4. **Conditional Moments and Variance**:
- Since the central moments of the plug-in estimator do not exist in general, the authors consider the conditional versions of these moments under the condition that the inverse of the Gram matrix is bounded by a constant.
- The authors study the conditional absolute central moments, conditional variance, and conditional covariance matrix of the estimator and provide the corresponding bounds.

5. **Applications and Extensions**:
- The authors discuss the application of these results to more general estimators and point out that many results can be easily extended to more complex situations.

### Key Contributions

- **Worst Risk Decomposition**: A novel worst risk decomposition method is proposed, making the problem of minimizing the worst risk solvable.
- **Consistency and Concentration of the Plug-in Estimator**: The consistency of the plug-in estimator in the almost sure sense is proven, and its concentration result is provided.
- **Study of Conditional Moments**: By studying conditional moments, the issue of the non-existence of central moments for the plug-in estimator in general cases is overcome.

### Conclusion

This paper proposes an effective method for minimizing the worst risk in Generalized Structural Equation Models across multiple shifted environments. Through theoretical analysis and mathematical derivation, the effectiveness and reliability of this method are verified. These results are of significant importance to fields such as causal inference and anchor regression.