Statistical Considerations for the Disappeared Significance in Regression Models
Ya Zhang,Shiping Zhu
DOI: https://doi.org/10.1016/j.jtcvs.2020.10.130
2022-01-01
Abstract:The authors reported no conflicts of interest.The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. The authors reported no conflicts of interest. The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. We read with great interest the valuable study by Tjoeng and colleagues1Tjoeng Y.L. Jenkins K. Deen J.F. Chan T. Association between race/ethnicity, illness severity, and mortality in children undergoing cardiac surgery.J Thorac Cardiovasc Surg. 2020; 160: 1570-1579Abstract Full Text Full Text PDF PubMed Scopus (20) Google Scholar reporting that compared with white children, African-American children had a greater mortality risk after cardiac surgery. This study is well designed. However, one limitation should be mentioned. Multivariable logistic regression is widely used to adjust for potential confounders. In the current study, the authors concluded that the association between ethnicity and mortality was mediated by disease severity, as this result became nonsignificant after adjusting for severity scores. This speculation is possible; however, it is not rigorous enough. Therefore, we briefly summarize other possible statistical conditions in 4 schematic diagrams (Figure 1) that may lead to the nonsignificant result. First, the multicollinearity2Barker L. Brown C. Logistic regression when binary predictor variables are highly correlated.Stat Med. 2001; 20: 1431-1442Crossref PubMed Scopus (31) Google Scholar should be evaluated when the association between the independent variable (ethnicity) and dependent variable (mortality) is significantly affected by a confounding variable. For instance, in the case of Figure 1, A, ethnicity has a strong relationship with disease severity, and the intersection of ethnicity and mortality is basically covered by disease severity. Under this circumstance, the association between ethnicity and mortality could be overwhelmed after adding disease severity as a confounder. Tolerance and variance inflation factors are commonly used for multicollinearity detection and should be reported in the current study. Second, the modulating or interacting effect between ethnicity and disease severity may also lead to a similar condition. For instance, in Figure 1, B, ethnicity has an impact on mortality; however, the effect size varies at different disease severity levels, suggesting that this association is modulated by disease severity. When this modulating effect between ethnicity and disease severity is bidirectional, we call it an interaction (Figure 1, C). The presence of an interacting/modulating effect should also be tested in the current study, and subgroup analysis may help to interpret these findings when a significant interacting/modulating effect was detected. Third, the mediation effect3Gaynor S.M. Schwartz J. Lin X. Mediation analysis for common binary outcomes.Stat Med. 2019; 38: 512-529Crossref PubMed Scopus (20) Google Scholar,4Mackenzie D. Pearl J. The Book of Why: The New Science of Cause and Effect. Penguin Books Limited, United Kingdom2018: 428-485Google Scholar also should be considered under this condition, using mediation analysis (Figure 1, D). The coefficient between ethnicity and mortality (γ), ethnicity and disease severity (α), disease severity and mortality (β) should be tested to determine the existence of a mediating effect. The percentage and significance of the direct and indirect effects also should be reported. Finally, Tjoeng and colleagues raised an important issue for patients undergoing cardiac surgery, and we suggest these statistical issues should be considered to reach a stable result. Association between race/ethnicity, illness severity, and mortality in children undergoing cardiac surgeryThe Journal of Thoracic and Cardiovascular SurgeryVol. 160Issue 6PreviewPrior studies demonstrate an association between nonwhite race/ethnicity, insurance status, and mortality after pediatric congenital heart surgery. The influence of severity of illness on that association is unknown. We examined the relationship between race/ethnicity, severity of illness, and mortality in congenital cardiac surgery, and whether severity of illness is a mechanism by which nonwhite patients experience increased surgical mortality. Full-Text PDF Open ArchiveReply from authors: Mediation Analysis in Health Disparities ResearchThe Journal of Thoracic and Cardiovascular SurgeryVol. 163Issue 1PreviewWe appreciate the Letter to the Editor1 in response to our article.2 We concluded that African-American children admitted postoperatively had 40% higher odds of mortality compared with White children, but that this survival advantage was eliminated after taking into account severity of illness on admission, suggesting that severity of illness is a mechanism for this observed health disparity. Zhang and colleagues1 identify additional statistical methods that have the potential to make the conclusions more stable, which we will address below. Full-Text PDF Reply: 3M—Multicollinearity, mediators, and moderatorsThe Journal of Thoracic and Cardiovascular SurgeryVol. 163Issue 1PreviewZhang and Zhu1 addressed limitations of multivariable modeling in their Letter to the Editor titled, “Statistical Considerations for the Disappeared Significance in Regression Models.” The letter refers to the study recently published in the Journal by Tjoeng and colleagues,2 on which we authored an editorial commentary.3 Zhang and Zhu discuss potential statistical reasons why the association between race (the aforementioned authors above incorrectly use the term ethnicity for black race) and greater risk of mortality is mitigated by disease severity—a phenomenon they refer to as “disappeared statistical significance.” These reasons include multicollinearity (when one predictor variable is highly linearly related to another predictor variable), moderating effects (when a variable influences the strength of a relationship between 2 other variables), and mediation effects (when a variable explains how or why there is a relationship between 2 other variables). Full-Text PDF