Abstract:Respiratory diseases are multifactorial heterogeneous diseases that have proved recalcitrant to understanding using focused molecular techniques. This trend has led to the rise of 'omics approaches (e.g., transcriptomics, proteomics) and subsequent acquisition of large-scale datasets consisting of multiple variables. In 'omics technology-based investigations, discrepancies between the number of variables analyzed (e.g., mRNA, proteins, metabolites) and the number of study subjects constitutes a major statistical challenge. The application of traditional univariate statistical methods (e.g., t-test) to these "short-and-wide" datasets may result in high numbers of false positives, while the predominant approach of p-value correction to account for these high false positive rates (e.g., FDR, Bonferroni) are associated with significant losses in statistical power. In other words, the benefit in decreased false positives must be counterbalanced with a concomitant loss in true positives. As an alternative, multivariate statistical analysis (MVA) is increasingly being employed to cope with 'omics-based data structures. When properly applied, MVA approaches can be powerful tools for integration and interpretation of complex 'omics-based datasets towards the goal of identifying biomarkers and/or subphenotypes. However, MVA methods are also prone to over-interpretation and misuse. A common software used in biomedical research to perform MVA-based analyses is the SIMCA package, which includes multiple MVA methods. In this opinion piece, we propose guidelines for minimum reporting standards for a SIMCA-based workflow, in terms of data preprocessing (e.g., normalization, scaling) and model statistics (number of components, R2, Q2, and CV-ANOVA p-value). Examples of these applications in recent COPD and asthma studies are provided. It is expected that readers will gain an increased understanding of the power and utility of MVA methods for applications in biomedical research.

Majority Vote and Other Problems when using Computational Tools

Monte Carlo algorithms for model assessment via conflicting summaries

Frequent Errors in Modeling by Machine Learning: A Prototype Case of Predicting the Timely Evolution of COVID-19 Pandemic

Full Characterization of Adaptively Strong Majority Voting in Crowdsourcing

Trials and tribulations of 'omics data analysis: assessing quality of SIMCA-based multivariate models using examples from pulmonary medicine

A practical guide to methods controlling false discoveries in computational biology

An observational analysis of the trope “A p-value of < 0.05 was considered statistically significant” and other cut-and-paste statistical methods

Forks Over Knives: Predictive Inconsistency in Criminal Justice Algorithmic Risk Assessment Tools

One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions

Impact of methodological choices on the analysis of code metrics and maintenance

Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method

Testing the Consistency of Performance Scores Reported for Binary Classification Problems

Biased versus unbiased numerical methods for stochastic simulations

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Three Gaps in Computational Text Analysis Methods for Social Sciences: A Research Agenda

Moving Beyond the Mean: Analyzing Variance in Software Engineering Experiments

"Medium-n studies" in computing education conferences

Evaluation of machine learning algorithms for health and wellness applications: A tutorial

Navigating the pitfalls of applying machine learning in genomics

Minor Issues Escalated to Critical Levels in Large Samples: A Permutation-Based Fix