Abstract:Summary The probability of benefit is a valuable and meaningful measure of treatment effect, which has advantages over the average treatment effect. Particularly for an ordinal outcome, it has a better interpretation and can make apparent different aspects of the treatment impact. Unfortunately, this measure, and variations of it, are not identifiable even in randomized trials with perfect compliance. There is, for this reason, a long literature on nonparametric bounds for unidentifiable measures of benefit. These have primarily focused on perfect randomized trial settings and one or two specific estimands. We expand these bounds to observational settings with unmeasured confounders and imperfect randomized trials for all three estimands considered in the literature: the probability of benefit, the probability of no harm, and the relative treatment effect.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to provide precise signed non - parametric bounds for treatment effects (such as the probability of benefit, the probability of no harm, and the relative treatment effect) on ordinal outcome measurements in observational studies and imperfect randomized trials. Specifically, the paper aims to extend the existing non - parametric bounding methods so that they are applicable to three main estimators in observational studies with unmeasured confounding factors and imperfect randomized trials: the probability of benefit ($\theta$), the probability of no harm ($\psi$), and the relative treatment effect ($\varphi$). These estimators are of great significance in clinical trials and epidemiological studies because they can more intuitively reflect different aspects of treatment effects without relying on strong assumptions.
### Main problems:
1. **Probability of benefit ($\theta$)**: That is, the probability that the outcome in the treatment group is strictly better than that in the control group, defined as $\theta=\text{pr}\{Y_i(1)>Y_i(0)\}$.
2. **Probability of no harm ($\psi$)**: That is, the probability that the outcome in the treatment group is at least not worse than that in the control group, defined as $\psi = \text{pr}\{Y_i(1)\geq Y_i(0)\}$.
3. **Relative treatment effect ($\varphi$)**: That is, the proportion of benefited patients minus the proportion of damaged patients, defined as $\varphi=\text{pr}\{Y_i(1)>Y_i(0)\}-\text{pr}\{Y_i(1)<Y_i(0)\}$.
### Research background:
- **Identification problem**: Even in fully randomized trials, these estimators are difficult to directly identify because the joint distribution of potential outcomes needs to be known, which is usually unobservable.
- **Existing research**: Many studies have provided non - parametric bounds in fully randomized trials, but these methods are usually limited to specific estimators or assumption conditions.
- **New contribution**: This paper fills this gap by providing signed non - parametric bounds in observational studies and imperfect randomized trials, and these bounds can be applied to ordinal outcomes with any level.
### Methods and results:
- **Signed non - parametric bounds**: The author derived precise signed non - parametric bounds under different causal diagram (DAG) settings, including the case where there are unmeasured confounding factors.
- **Specific bounds**:
- For the probability of no harm $\psi$, in the presence of unmeasured confounding factors, the bound is $p_{00}+p_{1(K - 1)}\leq\psi\leq1$.
- For the probability of benefit $\theta$, in the presence of unmeasured confounding factors, the bound is $0\leq\theta\leq1 - p_{10}-p_{0(K - 1)}$.
- For the relative treatment effect $\varphi$, in the presence of unmeasured confounding factors, the bound is $p_{00}+p_{1(K - 1)}-1\leq\varphi\leq1 - p_{10}-p_{0(K - 1)}$.
### Practical applications:
- **Data example**: The author used data from a randomized trial on peanut allergy to show how to calculate these bounds and their confidence intervals under different study designs.
- **Conclusion**: The study shows that the intake of peanut powder has a significant benefit in reducing allergic reactions, and there is little evidence that it will cause harm.
### Discussion:
- **Extensibility**: Although the bounds provided in this paper are derived without additional measured covariates, these bounds are still valid when conditioned on additional discrete covariates.
- **Future research**: The effective inclusion of continuous covariates is an open research area and needs further exploration.
In conclusion, this paper provides a new tool for evaluating treatment effects by providing precise signed non - parametric bounds in complex study designs, which helps to more accurately understand different aspects of treatment effects.