Bias in Multiple-Treatments Meta-Analysis: A Case on Comparative Efficacy and Tolerability of 15 Antipsychotic Drugs for Schizophrenia
Li Nan,Zhan Siyan,Si Tianmei
DOI: https://doi.org/10.3760/cma.j.issn.0366-6999.20141193
IF: 6.133
2014-01-01
Chinese Medical Journal
Abstract:Multiple-treatments meta-analysis is thought to be a feasible method to compare the efficacy and safety among different treatments, especially when there was no head-to-head research among some treatments. But sometimes some conclusions are inconsistent with the clinical experience. Recently, we read a multiple-treatment meta-analysis finished by Stefen Leucht et al, which was published in Lancet.1 The authors summarized the results of the RCT studies on 15 antipsychotics commonly used in practice; they also horizontally compared the efficacy and safety profile by the meta-analysis. We believe that the results provide more solid evidence for the rational usage of antipsychotics to the psychiatrists, also for the government to distribute health resources in a more reasonable way. There were several published comments,2–6 which focused on the discussion the probability of potential bias of multiple-treatment meta-analysis in principle level, or the better interpretation of the results from this paper, nevertheless, we want to propose our additional concerns from clinical and clinical epidemiological perspective, and share our thinking how to avoid the bias. At first, as psychiatrists, we found that some results of this study can't match our clinical experience. For example, the Amisulpride showed a second best effecacy.1 But this drug is rarely used in China. The main reason is the well-known side-effect of prolactin increase,7 which had not been fully analyzed since there was no enough available data. Besides, we found the conclusion on Amisulpride was mainly based on old researches. Only 2 of the 15 researches were published in recent 5 years, while the number of researches published before 1999 is 7.1 Thus, based on this interesting point, after rounds of discussions with clinical epidemiologists, expert on evidence-based medicine and other psychiatrists, we found there were some issues in the design of the meta-analysis requiring further reflection, which may lead to bias and affect the results. These reflections are outlined below. We think that the methodology of meta-analysis is on a continuous way to advance, and we would like to take this opportunity to discuss them with the authors, editors, also with the other readers. The authors included the RCTs of all the commonly used antipsychotics in order to evaluate the efficacy and safety profiles. Although studies included are all RCTs, there are fundamental differences among them. As the author did mention in the appendix 2 that fixed-dose studies, they also did some stratified analysis and sensitivity analysis with consideration of this potential bias. We believe this research provides us a very good paradigm, however, there are still some more points requiring taking into account as below. Fixed-dose design and flexible-dose design There are two kinds of dosing designs, the fixed-dose design and the flexible-dose design. In studies with fixed-dose design, subjects get fixed doses randomly without clinicians’ intervention. While, in studies with flexible-dose design, the dosage of each subject is depended on clinicians’ decision. Studies comparing efficacy and safety between different fixed doses of a particular drug should be included with more cautious, since the intervention here is different from those studies which the flexible-dose design is adopted. A further stratified analysis and/or sensitivity analyses that exclude all fixed-dose studies should be necessary. Distribution of the dosage is different In the RCTs evaluating efficacy and safety between post-marketing medications, the subjects were given the flexible dose at the clinician's discretion according to the Evidence Based Practice principle.8,9 Hence the dosage would be nearly normal distribution. When the authors compared efficacy and safety profile between dosages, the subjects would be evenly assigned to different dose groups randomly,10,11 which would be showed as a uniform distribution. It is apparently not very reasonable to ignore the difference in dosage distribution and directly compare the two kinds of studies’ results. Flexible-dose strategy can balance the efficacy and safety better In studies adopting fixed-dose design, the subjects got one dose by randomization,12 but in studies adopting flexible dose design, the subjects got the dose by clinicians’ comprehensive consideration of evidence, clinical experience and individual needs.13,14 This different dosing strategy would affect the evaluation of drug efficacy and safety, and stratified analysis or sensitivity analysis may help to eliminate this bias. With these differences in mind, including the fixed-dose studies into this multiple-treatments meta-analysis would have complicated impact on the final result. As the authors showed in the appendix, for those fixed-dose studies only the ones matching the International Consensus Study of Antipsychotic Dosing15 were included. Moreover, the sensitivity analyses were made on those studies which were more efficacious or less well defined, or had some special design. However, we think these are not enough. In further multiple-treatments meta-analysis, more strict inclusion criteria should be made, and stratified analysis and/or sensitivity analysis that exclude all fixed-dose studies is necessary. Pre-authorization and post-marketing studies Pre-authorization and post-marketing studies should be analyzed separately even with the same or similar inclusion and exclusion criteria, since the representative of the subjects are not the same. In pre-authorization studies, subjects are treated with drugs not being approved. As a result, comparing with the post-marketing studies, both investigators and patients would be more cautious when making the decision to participant in such a study. This difference may cause selection bias and channeling bias,16–18 which is a form of allocation bias, where drugs with similar therapeutic indications are prescribed to groups of patients with prognostic differences. Patients may be more conservative to the new drugs which have similar mechanism to some post marketing ones, not to mention the new drug with different mechanism. So those who have relatively mild symptoms or simple conditions, or those who are from poor economic status are more likely to participate in. However, these considerations are not likely to be involved in the post marketing studies. In order to avoid the influence of the different drug development stages on the result of meta-analysis, we suggest performing subgroup analysis and sensitivity analysis, and data from pre-authorization studies and post marketing ones should be assessed separately and compared with combined analysis result. Also, discussions about study subjects’ representation should be included in this paper. Chronology bias due to different launch time19,20 Chronology bias occurs when comparison with historic group. The difference between latter and former time points on diagnose, treatments or preferred outcome measures may cause inequality within groups. New drugs attracted more attention Investigators would focus on newly launched drugs and pay less attention to “older drugs”. Most investigators believe that no specific safety report is needed because most doctors are very familiar with the safety profile of the older drugs, instead, pay more attention to newer ones, which leads to a publication bias that the newer drugs have worse safety profile than the older ones. Use of older drugs are more precise The longer a drug is on the market, the more clinical trials and experience it would have, and the more accurate and reliable the data and the evidence-based guidance are. As a result, over years after marketing, evidences and guidelines for treatments would change several times.21 Take clozapine for an example.22 At first, clozapine was developed to treat schizophrenia in general. During the post-marketing research, it was found to have serious adverse reactions. After that, the usage of clozapine is confined to refractory schizophrenia patients with aggressive behavior and suicidal tendencies. As a result, in recent years, the report of adverse reactions of clozapine remained at a low level. In this example we could see that, we would gain, gradually, the knowledge and experience of the efficacy and safety of a drug. The usage of the drug would be optimized with time, due to the experience learned from clinical practice and vast numbers of studies after being launched. Also, clinicians would be more familiar with the drug profile, so they are more capable to use the drug to treat patient in an appropriate way. This time advantage would lead to the potential benefit of older medications. We suggest that the time effect should be considered in the analysis. Meta-analysis would all be affected by this way if only being involved different medications or interventions, so that there should be analysis and discussion about time on market. Meanwhile, a new method of data combination should be established to assess the relationship between the efficacy result and the time on market. The efficacy and safety difference between newly-launched drugs and older ones would be more reliable if the timing factor is taken into account when the data are analysed. Generally, multiple-treatments meta-analysis brings us a feasible method to compare the efficacy and safety of the majority treatments for one disease or therapeutic areas. It also provides better evidence for making clinical decisions and, especially, policy decisions. However, multiple-treatments meta-analysis is such a new method and facing a huge number of factors in one analysis, so the result should be applied with more cautions. As the authors showed in the article and appendix, considerable sensitivity analyses were conducted, however, we think the sensitivity analysis is not enough even in this study, which has good design and comprehensive consideration. Besides, although the results of this kind of meta-analysis are based on majority high quality studies, the interpretation of the results should not be made apart from the experience of clinicians. Especially when the results are based only on indirect comparison, the clinician's experience is more important. Furthermore, we think clinician's experience may be helpful for the optimizing of methodology in multiple-treatments meta-analysis. (Received May 22, 2014) Edited by Cui Yi