Don't judge a book by its cover, don't judge a study by its abstract. Common statistical errors seen in medical papers

S. W. Choi,Chi-wai Cheung
DOI: https://doi.org/10.1111/anae.13506
IF: 12.893
2016-07-01
Anaesthesia
Abstract:Most published scientific studies or reviews will begin with an abstract, which is essentially a summary of what the full paper contains. For non-open access papers, the abstract is the only part of the paper which is freely available and therefore would be the most frequently read section of the paper [1]. Most journals have policies regarding how an abstract should be presented, whether it be structured or not, and word limits (often only 250 to 300 words are allowed) are strictly enforced. Since it is the only freely available part of the paper, it acts as the avenue investigators use to vie for readers’ attention. Although it is often not a scientist’s intention to deliberately mislead readers, spin and biased reporting in the abstract are difficult to avoid. Since over a thousand papers are published every single week, investigators are resorting to using words such as ‘novel’, ‘amazing’ and ‘innovative’ to boost readership [2]. Claims not supported by the data may be propagated in study abstracts, and it is only when data embedded deep in the text is carefully examined that the reader draw any conclusions about the findings. In 2005, John Ioannidis, Professor of Statistics at Stanford University, published a seminal paper in PLOS One explaining ‘why most published research findings are false’ [3]. Two of the myriad problems identified in science reporting were misuse of statistics, and overinterpretation of data. Statistical errors and inappropriate use of statistics in medical research are potentially very serious problems as they may lead researchers to draw erroneous conclusions, which are then trumpeted in a study’s abstract and may lead to press releases if the finding is interesting enough [4]. Though statistics is widely acknowledged as a powerful tool in the scientific research process, there is consensus that statistical standards in the medical literature are generally low and despite Ioannidis’ warning over a decade ago, errors are still commonplace. It is still the contention of many researchers that data on every single biomarker, demographic and outcome should be collected in any study, as far as is physically feasible, and then all data collected is analysed to see ‘where the chips fall’. That way, something ‘significant’ and/or ‘interesting’ is bound to come up. Few researchers realise that the most important phase of any research is the planning and design stage, and that correct application of statistical analyses is essential. This article will outline two of the most prolific statistical errors seen in medical manuscripts.
What problem does this paper attempt to address?