A quantitative approach to evidence triangulation: development of a framework to address rigour and relevance

Chin Yang Shapland,Luke A McGuinness,Joshua Bell,Maria Carolina Borges,Ana Luiza Goncalves Soares,George Davey Smith,Tom R Gaunt,Deborah A. Lawlor,Kate Tilling,Julian PT Higgins

DOI: https://doi.org/10.1101/2024.09.20.24314046

2024-09-23

Abstract:Triangulation is an approach to strengthening causal inference by integrating evidence from multiple sources. Most studies using triangulation have qualitatively examined whether different studies agree upon the presence of a causal effect, rather than estimated the effect by quantitatively integrating results. Here, we develop a framework for quantitative triangulation. We first address how to relate study specific research questions to an overall target causal question (relevance), and then assess the directions and magnitudes of bias in each study (rigour), before combining the results using meta-analysis adjusted for the biases. We illustrate our framework by triangulating evidence from randomized controlled trials (RCTs), Mendelian randomization (MR) and conventional multivariable regression (MVR) to estimate the effect of beta-carotene on coronary heart disease (CHD) and cardiovascular disease (CVD). Five RCTs and one MR study showed little evidence of a causal relationship between beta-carotene and CHD (relative risk (RR)=1.00 with 95% CI=0.98 to 1.01 and RR=1.02 with 95% CI=0.98 to 1.07, respectively). 13 MVR studies indicated that high intake of beta-carotene reduces CHD risk (RR=0.83 with 95% CI0.76 to 0.91). After applying our framework, the three study designs agreed that there is little evidence of an effect of beta-carotene intake on the risk of CHD (RR=1.01 with 95% CI=0.99 to 1.02). Findings were similar for CVD. Our framework shows how to address rigour and relevance quantitatively when triangulating evidence from different study designs. We highlight the importance of explicitly defining the target and study-specific research questions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to develop a quantitative method to integrate evidence from different research designs in order to strengthen causal inferences. Specifically, the author proposes a framework for dealing with the relevance and rigour of different research results and combines these results through meta - analysis to more accurately estimate the magnitude of the causal effect.

### Detailed Explanation:

1. **Problem Background**:
- **Triangulation**: This is a method of enhancing causal inferences by integrating evidence from multiple sources. Most studies using triangulation mainly qualitatively examine whether different studies consistently indicate the existence of a causal effect, without quantitatively integrating these results to estimate the effect size.
- **Research Motivation**: When different studies are affected by different biases, if they all show some effect, it can increase our confidence in the authenticity of that effect. However, most existing studies remain at the qualitative level and lack a quantitative assessment of the effect size.

2. **Research Objectives**:
- Develop a framework for quantitative triangulation to estimate the magnitude of the causal effect.
- Deal with two key issues: relevance and rigour.
- **Relevance**: How to link the specific questions of each study to the overall target causal question.
- **Rigour**: How to assess the direction and magnitude of bias in each study.
- Use an adjusted meta - analysis method to combine the results of different studies.

3. **Case Study**:
- The author selects the impact of β - carotene on the risk of coronary heart disease (CHD) and cardiovascular disease (CVD) as a case study.
- Demonstrates the application of the framework by integrating the results of randomized controlled trials (RCTs), Mendelian randomization (MR) and conventional multivariable regression (MVR) studies.

4. **Main Findings**:
- The quantitative triangulation framework shows that different research designs, after adjusting for bias, consistently conclude that β - carotene intake has almost no effect on CHD risk (relative risk RR = 1.01, 95% CI = 0.99 to 1.02).
- Similar results also apply to CVD.

### Formula Representation:
- Relative Risk (RR): \[ \text{RR}=\frac{\text{P(outcome | exposed)}}{\text{P(outcome | unexposed)}} \]
- Confidence Interval (CI): \[ \text{CI}=\left[ \text{lower bound}, \text{upper bound} \right] \]

Through this method, the author not only shows how to quantitatively integrate evidence from different sources but also emphasizes the importance of clearly defining the target causal question.