Short-form Zarit Caregiver Burden Interviews were valid in advanced conditions
Irene J. Higginson,Wei Gao,Diana Jackson,Joanna Murray,Richard Harding
DOI: https://doi.org/10.1016/j.jclinepi.2009.06.014
IF: 7.407
2010-01-01
Journal of Clinical Epidemiology
Abstract:Study Design and Setting Secondary analysis of carers' surveys in advanced cancer ( n = 105), dementia ( n = 131), and acquired brain injury ( n = 215). All completed demographic information and the ZBI-22 were used. Validity was assessed by Spearman correlations and internal consistency using Cronbach's alpha. Overall discrimination ability was evaluated using the area under the receiver operating characteristic curve (AUC). Results All short-form versions, except the ZBI-1 in advanced cancer (rho = 0.63), displayed good correlations (rho = 0.74–0.97) with the ZBI-22. Cronbach's alphas suggested high internal consistency (range: 0.69–0.89) even for the ZBI-4. Discriminative ability was good for all short forms (AUC range: 0.90–0.99); the best AUC was for ZBI-12 (0.99; 95% confidence interval [CI]: 0.98–0.99) and the second best for ZBI-7 (0.98; 95% CI: 0.96–0.98) and ZBI-6 (0.98; 95% CI: 0.97–0.99). Conclusions All six short-form ZBI have very good validity, internal consistency, and discriminative ability. ZBI-12 is endorsed as the best short-form version; ZBI-7 and ZBI-6 show almost equal properties and are suitable when a fewer-question version is needed. ZBI-4 and ZBI-1 are suitable for screening, but ZBI-1 may be less valid in cancer. Keywords Carer Outcome Palliative Burden Aging Validity 1 Background What is new? Key findings The 12-item Zarit Burden Interview (ZBI-12) was suitable in all situations; the ZBI-7 and the ZBI-6 were almost equally good and may be suitable for palliative care settings; the ZBI-4 and ZBI-1 were useful when a very short screening instrument was needed. What was known? The ZBI-22 is a widely used outcome measure of caregiver burden and has been validated in diverse caregiving samples. Several short-form versions of ZBI have been developed, but little is known about how well they perform in diverse populations. What does this study add? For the first time, six short-form versions of ZBI have been comprehensively and systematically evaluated in diverse populations. Evidence-based recommendations have been provided for choosing the best short-form version in various settings. The ZBI-6, a further improvement to the ZBI-7, has been developed for palliative care settings. What is the implication and what should change now? Stop using the ZBI-8; use ZBI-6 in palliative care; consider using the ZBI-1 when rapid screening is needed. Informal carers are the primary resource for patient care and are known to have high needs for support and psychological morbidity [1–4] . Although there are many suggested interventions seeking to improve their overall well-being, there is little evaluative research into the efficacy of such interventions [5,6] . Measurement of appropriate carer outcomes is essential for such studies. Although there are many measures to assess caregiver burden, strain, well-being, or other outcomes in specific disease, such as stroke or mental illness [7,8] , there are fewer measures targeted for the carers of patients with advanced disease. Mularski et al. in a major systematic review of measures for use toward the end of life for the National Institute of Health (USA) highlighted “significant gaps” in measuring caregiver outcomes, identifying only two measures in their literature search of 24,423 citations [9] . Caregiver burden is closely aligned to the goals of many interventions and is associated with negative health outcomes in carers of people with common conditions, such as dementia, stroke, and cancer [8,10,11] . Moreover, perceived burden had been shown to predict anxiety and depression in carers of patients with these conditions [12–14] . Caregiver burden had been defined as a context-specific negative affective outcome, occurring as a result of perceived inability to contend with role demands [15] . There is general agreement that caregiver burden is a multidimensional concept affected by objective elements related to the nature and time of the practical tasks undertaken by carers and subjective elements arising from the perceived emotional, social, and relationship stresses that can accompany this role [8,16] . Therefore, it would seem appropriate to measure caregiver burden as an outcome in advanced disease. Although there are other measures, the 22-item Zarit Burden Interview (ZBI) is the most widely used tool for measuring the level of subjective burden among carers [17,18] . Several shorter versions of the ZBI have been developed, including Bédard et al.'s 12-, 8-, 7-, and 4-item screening versions [19–21] . However, the factorial structures of ZBI were established among the carers of patients with dementia whose concerns may be different from carers of patients with cancer or sudden onset illness. Analysis of four abridged versions of the ZBI in 503 carers of people with dementia suggested that the 12-item version was optimal [15] . However, in some clinical settings, such as intensive care units, palliative care, and care of older people, the 12-item ZBI can still be a heavy assessment burden. There is a need to test shorter forms of the ZBI with the carers of people with advanced or progressive illness. Therefore, we designed this study to investigate the validity and internal consistency of six shorter forms of ZBI (ZBI-12, ZBI-8, ZBI-7, ZBI-6, ZBI-4, and ZBI-1) among informal carers of patients with three different conditions compared with the 22-item version as the gold standard. 2 Data and methods 2.1 Design and data sources This is a secondary analysis using data pooled from four studies. 1. Baseline data from a multicenter evaluation of palliative day care for cancer patients involving six centers across the south of England [22,23] ; 2. Baseline data from a two-center evaluation of the “90 Minute Group,” a supportive intervention for the carers of cancer palliative care patients [6] ; 3. A national postal questionnaire survey of caregiver experiences of acquired brain injury (ABI) [24] ; and 4. Baseline data from a prospective longitudinal cohort study of caregiver burden in dementia involving participants from South East London [25] . All studies collected data from informal carers using the self-reported 22-item ZBI (ZBI-22), with interviewers present in the cancer and dementia studies to provide support to respondents during data collection if needed. In addition to ZBI, the data set contains basic demographic data, including age, sex, and relationship, and clinical data regarding the patients. 2.2 Short-form versions of the Zarit Burden Interview Several short-form versions have been developed. The three most common short-form versions of ZBI are the 12-item version (ZBI-12) by Bédard et al. [19] , the eight-item version (ZBI-8) by Arai et al. [21] , and the four-item version (ZBI-4) by Bédard et al. [19] . The ZBI-12 and the ZBI-4 were reported in the same study [19] . The ZBI-22 data were factor analyzed using a principal component analysis and revealed a two-factor structure. The items for the ZBI-12 were selected through a combination of high factor loading and high item–total correlations across all six situations, and the ZBI-4 screening items were selected based on the item–total correlations while keeping the three-to-one item ratio between factors 1 and 2 [19] . The ZBI-8 items were chosen in terms of their factor loadings (≥0.65) on a two-factor structure [21] . A new seven-item version (ZBI-7) proposed specifically for palliative care was included for evaluation; the items extracted were decided by an expert committee [20] . However, ZBI-7 included item 22 of the full scale, a global question to assess overall subjective burden. This is unusual, because short-form versions do not usually contain the global question, especially for the measures of subjective burden [8,16,26] . Therefore, we derived a six-item version of ZBI (ZBI-6), excluding this global question. We also tested item 22 on its own (ZBI-1) to understand whether the single global question (“Overall how burdened do you feel”) would be useful as a screening tool. Items included in each of the short-item versions are listed in Table 1 . The responses to every item are in 5-point Likert scale from 0 (never) to 4 (nearly always). The overall burden is assessed by the total score of all items, with a higher score representing a greater caregiver burden. 2.3 Data analysis Demographic characteristics were described, and differences between diagnostic groups were compared using one-way ANOVA (for age) and chi-square test (for sex and relationship to patient). Total scores of ZBI-22, ZBI-12, ZBI-8, ZBI-7, ZBI-6, ZBI-4, and ZBI-1 were summarized using descriptive statistics. The score differences between three diagnoses were examined for overall differences using Kruskal–Wallis test followed by Wilcoxon two-sample test if the overall difference was significant. The Bonferroni procedure was used to adjust raw P values (multiplying 3) to control the type 1 error in multiple testing [27] . We planned to examine the subscales of ZBI, but found inconsistencies in the literature regarding the content of the subscales. For example, the original article by Whitlatch et al. [28] presented two subscales: role and personal strain with six and 12 items, respectively. Hébert et al. [29] used these in their factor analysis producing a 12-item version. However, Knight et al. [30] , Bédard et al. [19] , and O'Rourke et al. [31] , when conducting exploratory and confirmatory factor analysis, revealed that different items were included in the role and personal subscales, even though they gave them the same names. Because of the inconsistencies, we did not proceed with the subscale analysis. Validity was assessed by testing for correlations of the ZBI-12, ZBI-8, ZBI-7, ZBI-6, ZBI-4, and ZBI-1 with the ZBI-22 (as the gold standard) using Spearman rank order correlation. Terwee et al. [32] suggest that correlations of 0.7 or more are required. Internal consistency was examined with Cronbach's alpha, and a value in the range of 0.7–0.9 is good [32] . An alpha value of greater than 0.9 suggests redundant items. For small scales (e.g., four items), values of 0.6 are good, because alpha tends to underestimate internal consistency when the number of items is small [32,33] . Using a total burden score of 21 on the ZBI-22 as the cutoff point for high burden [18] , the discriminatory performance of various short-form versions of ZBI was assessed and compared with the receiver operating characteristic (ROC) curve [34] , which was constructed by plotting sensitivity against 1 − specificity. Each point in the ROC plot represents a sensitivity/1 − specificity pair corresponding to a particular cutoff value. A test with perfect discrimination has an ROC plot that passes through the upper-left corner (100% sensitivity and 100% specificity). Therefore, the closer a ROC plot is to the upper-left corner, the higher the overall accuracy of the test. The point closest to (0, 1) on the curve was used to determine the most optimal combination of sensitivity and specificity [34] . The areas under the curves (AUC) were calculated using the trapezoidal method [34,35] . The AUC represents the overall discriminative ability of a test, that is, the ability to correctly classify those with and without burden. The range of the AUC is 0.5–1.0. A discriminative test is considered perfect if AUC = 1.0, good if AUC = 0.8–1.0, moderate if AUC = 0.6–0.8, and poor if AUC = 0.5–0.6; an area of 0.5 reflects a random rating model [35] . 95% Confidence intervals of AUCs were computed. A P value below 0.05 was considered as statistically significant. All analyses were carried out using SAS 9.1 package (SAS Institute Inc., Cary, NC, USA). 3 Results One hundred and five, 131, and 215 informal carers for patients with advanced cancer, dementia, and ABI respectively, were recruited, making a total sample of 451. Most carers were women—81% for ABI, 72% for cancer, and 72% for dementia ( χ df = 2 2 = 5.50, P = 0.06). The carers of ABI patients were the youngest—mean age (standard deviation) of 54 (11) compared with 66 (12) for cancer and 62 (13) for dementia ( F (2,448) = 42.8, P < 0.0001). Spouse/partner carers were the most common—59% for ABI, 82% for cancer, 37% for dementia—followed by parent (37% for ABI, 4% for cancer, and 0% for dementia) or son/daughter (44% for dementia, 11% for cancer, and 0% for ABI) ( χ 6 2 = 227.6, P < 0.0001). Table 2 shows the descriptive statistics of the total score for various short-form versions of ZBI in each diagnostic group. All scores showed highly significant overall difference across three groups (all P values < 0.001). In all groups, there was a wide range of burden scores. The disease-specific burden pattern reflected in the full-scale ZBI was well captured by all short-form versions. Subjective caregiver burden were lowest for the carers of cancer patients and highest for those of patients with ABI. However, the significance of the difference shown in ZBI-22 (ABI vs. cancer: z = 7.66, P adj < 0.0001; ABI vs. dementia: z = 5.62, P adj < 0.0001; cancer vs. dementia: z = 2.03, P adj = 0.12) was only satisfactorily revealed by ZBI-12, ZBI-6, and ZBI-4. ZBI-1 was among the worst in all the short-form versions for misjudging two pairwise comparisons of ABI vs. dementia ( z = 2.15, P adj = 0.10) and cancer vs. dementia ( z = 4.30, P < 0.0001), whereas ZBI-7 ( z = 2.76, P adj = 0.018) and ZBI-8 ( z = 4.24, P < 0.0001) did not reflect the difference in comparing cancer with dementia, as was the case in ZBI-22. High correlation coefficients were found between the full version and short forms ( Table 3 ), with correlations well above our criteria (>0.7) for the full scale (range: 0.88–0.97) using all short-form versions except for the ZBI-1. Even with the one-item version (ZBI-1), satisfactory correlation with ZBI-22 was obtained in dementia (rho = 0.74) and ABI (rho = 0.78) groups. The full ZBI showed a high Cronbach's alpha in all three diagnostic groups, ranging from 0.88 to 0.93. The alpha values met our internal consistency criteria of good for all the short-form versions, ranging from 0.69 (for a four-item scale in cancer) to 0.90. Correlations and internal consistencies were similar to those in Table 3 separately for men and women, for those older and younger than 70 years, and for higher and lower burdened carers. The most optimal combination of sensitivity and specificity, as visualized from ROC curves ( Fig. 1 ), was 92% and 94% for ZBI-12 (cutoff score: 12), 82% and 92% for ZBI-8 (cutoff score: 6), 95% and 86% for ZBI-7 (cutoff score: 7), 91% and 91% for ZBI-6 (cutoff score: 6), 88% and 85% for ZBI-4 (cutoff score: 4), and 91% and 53% for ZBI-1 (cutoff score: 1). All shorter versions were overall successful in differentiating low- and high-burden individuals with all AUCs well above 0.90. The short-form version with the best discriminative ability was ZBI-12 and that with the lowest was ZBI-1. ZBI-6 performed slightly better than ZBI-8 and to the same level as the ZBI-7. 4 Discussion We tested and validated six short forms of the ZBI in three caregiving populations. In all groups, there was a wide range of scores for the ZBI-22; therefore, the short forms were tested in samples reporting varying caregiver burden. However, the highest burden scores were in the dementia and the ABI groups, and we were not able to test burden scores above 34 in the advanced cancer group. It may be that caregiver burden using ZBI was lower in advanced cancer compared with dementia and ABI because of the lower levels of cognitive disturbance or greater specialist palliative support for the cancer carers (the cancer carers were sampled from palliative care services) [4,22,24,25] . However, it may equally be because of ZBI failing to measure some aspects of caregiver burden in advanced cancer. Caregiver burden is a complex construct. It has been described as having physical, social, financial, and emotional components, as well as leading to relationship and personal strain [16] . Some measures seek to capture “burden” and others “strain” [7–9] . More recently measures to capture carer positivity and satisfaction have been developed [36] . Which components of burden are present and whether these are different across different conditions are important questions and need to be the subject of future research. Three key findings emerge from our analysis. First, we found high levels of validity (with correlations ranging from 0.74 to 0.97) for all the short forms compared with ZBI-22 in three diagnostic groups with the only exception of the one-item version in cancer. The ZBI-12 has the highest validity (rho = 0.95–0.97), and this is consistent across advanced cancer, dementia, and ABI samples. In our populations, the performance of the ZBI-8 and the ZBI-4 are almost identical and slightly worse than that of ZBI-6 in both correlation and ROC analysis. Although the correlations between short and long forms appeared to be slightly and consistently stronger for the ABI group, and lowest in the advanced cancer group, the differences were marginal, not significant, and could be an artifact of the narrow range of scores for the cancer group. Second, we found high internal consistency of all versions of the ZBI, suggesting that some items are redundant and short forms can be used. Only the ZBI-4 had lower, but still good internal consistency (Cronbach's alpha: 0.69–0.79) in the cancer group. Third, as we planned our analysis, we found confusions in the categorization of the role and personal strain subscales. Given the variability regarding the subscales and the high internal consistency of the full Zarit scale and all the short forms, we doubt that using the “personal” and “role” strain subscales has either face or psychometric validity [30] . The choice of ZBI version should be based on the specific aims of the research. For most situations, 12-item ZBI should have comparable performance with the full version with the differentiating capacity close to 1. For situations requiring rapid identification of caregiver burden, for example, screening for assessment or referral, four-item and even one-item versions will be the ideal choice, given their simplicity and optimal combination of high sensitivity (>80%) and high specificity (>50%) as evident by the ROC curves. Given that burden is multidimensional, the success of the single- and four-item versions surprised us. Although the ZBI-8 has more items, it did not exhibit any superiority over the ZBI-7 or the ZBI-6 in psychometric characteristics and differential ability. An earlier small study reported a perfect performance of the ZBI-7 with 100% sensitivity and 100% specificity in the palliative care setting [20] . The validation of the ZBI-7 in our larger study was close to this—90% specificity at the sensitivity level of 90% ( Fig. 1 ). However, we also found that, compared with the ZBI-7, excluding the “global burden question” did not compromise the ability of the ZBI-6 to distinguish carers with burden from those without, and results were almost equal to ZBI-12 ( Fig. 1 ). Therefore, when investigators need to keep the number of questions short, the ZBI-6 appears to be a good choice. Short forms to assess caregiver burden may be important to ensure feasible collection of data—burdened carers often focus on the needs of the patients whom they care for and not their own experiences and stresses related to caring [37] and may not wish to spend time completing anything but the briefest questionnaire. When using short-form versions of the ZBI as a screening test, sensitivity and specificity are standard measures for the diagnostic performance compared with the gold standard—ZBI-22 [38] . However, these two measures are inversely related. Increasing one measure (by changing the cutoff value) results in a decrease in the other. Which one is more important is a question that can only be answered in the context in which it is used. Often a balance is needed. For example, sensitivity is important when identifying highly burdened carers (as measured by the summary score of the ZBI), because they can be offered more support that is unlikely to do harm [39] . However, when resources are scarce or if carers felt they were “labeled” as not coping by false-positive screening results, specificity is more important than sensitivity. It should be noted however that, in this study, we were primarily testing the performance of short forms of the ZBI as a screening tool. In clinical practice, a wider exploration of the components contributing to burden may be needed. Ideally, qualitative or cognitive interviewing would be needed to establish if all relevant aspects for caregiver burden are included. We recognize several limitations for this study. First, the performance comparisons were evaluated with cross-sectional data; therefore, they provide no information on short forms' responsiveness to change (an essential psychometric property in intervention and longitudinal studies) [40] . Three of the original studies collected data at several time points; we are planning to use these data to assess the adequacy of short-form ZBI to detect change. Second, our analyses were restricted to the limited number of common demographic variables in the pooled data set; therefore, we could not make detailed performance comparisons across subsamples. Third, our validation was based on a comparison between short forms of ZBI with the full 22-item version, and thus makes an assumption that ZBI-22 accurately captures caregiver burden. Ideally, we would have assessed the short-form versions with other measures of burden or against clinical findings, but this would have required more intensive data collection among burdened carers, which may not have been feasible. Our data only allow conclusions to be drawn about the short-form versions of ZBI compared with the full version. 5 Conclusions We found strong validity and internal consistency for each of the short-form versions in all three samples. The ZBI-12 is suitable in all situations, whereas the ZBI-7 or the ZBI-6 is suitable when a fewer-question version is needed, for example, in palliative care setting. The ZBI-7 is equivalent to the ZBI-6 although with one more question. The ZBI-4 and ZBI-1 may be useful when a very short screening instrument is needed, but the ZBI-1 may be less valid in cancer. Acknowledgments We thank the patients, carers, staff, and volunteers who participated in the original studies, including (1) six day and home hospice and palliative care services which recruited and interviewed patients and carers and Danielle Goodwin and other interviewers in the study; (2) two home palliative care services in London, Celia Leam and Liz Taylor who worked with us to recruit patients, and Alison Pearce (research assistant); (3) Research assistants Shehla Kazim, Amanda Tadrous, and Joel Sheridon and representatives of Headway, the Encephalitis Society and the Meningitis Trust, who helped to disseminate information about the ABI study to carer participants; (4) Community Mental Health Teams for Older Adults in the South London and Maudsley Mental Health Trust and the research workers Beth Foley and Louise Atkins. In addition, we thank the funders of the original studies: the NHS Executive (London and South East) for funding projects 1 and 2, the Department of Health (R&D grant 030/0066) for project 3, and the Department of Health, Policy Research Programme for project 4. Dr. Gao Wei is 50% supported by the National Cancer Research Institute, UK, a part of the “COMPlex interventions: Assessment, trialS and implementation of Services in Supportive and Palliative Care (COMPASS)” collaborative. We also thank Professors Peter Fayers, Gordon Murray, and Julia Brown for their helpful comments to improve this manuscript. References [1] R. Charlton Palliative care in non-cancer patients and the neglected caregiver J Clin Epidemiol 45 1992 1447 1449 [2] S.Y. Hung A.S. Pickard W.P. Witt B.L. Lambert Pain and depression in caregivers affected their perception of pain in stroke patients J Clin Epidemiol 60 2007 963 970 [3] J. Maher H. Green Carers 2000. Office for National Statistics 2002 The Stationary Office London [4] The Resource Implications Study Group of the MRC Study of Cognitive Function and Aging (RIS MRC CFAS) Psychological morbidity among informal caregivers of older people: a 2-year follow-up study. The Resource Implications Study Group of the MRC study of cognitive function and ageing (RIS MRC CFAS) Psychol Med 30 2000 943 955 [5] R. Harding I.J. Higginson What is the best way to help caregivers in cancer and palliative care? A systematic literature review of interventions and their effectiveness Palliat Med 17 2002 63 71 [6] R. Harding I.J. Higginson C. Leam N. Donaldson A. Pearce R. George Evaluation of a short-term group intervention for informal carers of patients attending a home palliative care service J Pain Symptom Manage 27 2004 396 408 [7] K. Harvey J. Catty A. Langman H. Winfield S. Clement E. Burns A review of instruments developed to measure outcomes for carers of people with mental health problems Acta Psychiatr Scand 117 2008 164 176 [8] J.M. Visser-Meily M.W. Post Riphagen II E. Lindeman Measures used to assess burden among caregivers of stroke patients: a review Clin Rehabil 18 2004 601 623 [9] R.A. Mularski S.M. Dy L.R. Shugarman A.M. Wilkinson J. Lynn P.G. Shekelle A systematic review of measures of end-of-life care and its outcomes 3 Health Serv Res 42 2007 1848 1870 [10] N.E. Goldstein J. Concato T.R. Fried S.V. Kasl R. Johnson-Hurzeler E.H. Bradley Factors associated with caregiver burden among caregivers of terminally ill patients with cancer J Palliat Care 20 2004 38 43 [11] J. Schneider A. Hallam J. Murray B. Foley L. Atkin S. Banerjee Formal and informal care for people with dementia: factors associated with service receipt Aging Ment Health 6 2002 255 265 [12] F. Alvarez-Ude C. Valdes C. Estebanez P. Rebollo Health-related quality of life of family caregivers of dialysis patients J Nephrol 17 2004 841 850 [13] S.H. Zarit K.E. Reever J. Bach-Peterson Relatives of the impaired elderly: correlates of feelings of burden Gerontologist 20 1980 649 655 [14] A.M. Gort M. Mingot X. Gomez T. Soler G. Torres O. Sacristan Use of the Zarit scale for assessing caregiver burden and collapse in caregiving at home in dementias Int J Geriatr Psychiatry 22 2007 957 962 [15] N. O'Rourke H.A. Tuokko The relative utility of four abridged versions of the Zarit Burden Interview J Ment Health Aging 9 2003 55 64 [16] M.C. Chenier Review and analysis of caregiver burden and nursing home placement Geriatr Nurs 18 1997 121 126 [17] Y.G. Bachner N. O'Rourke Reliability generalization of responses by care providers to the Zarit Burden Interview Aging Ment Health 11 2007 678 685 [18] S.H. Zarit R.D. Orr J.M. Zarit The hidden victims of Alzheimer's disease: families under stress 1985 New York University Press New York [19] M. Bédard D.W. Molloy L. Squire S. Dubois J.A. Lever M. O'Donnell The Zarit Burden Interview: a new short version and screening version Gerontologist 41 2001 652 657 [20] A.M. Gort J. March X. Gomez M.M. de S. Mazarico J. Balleste [Short Zarit scale in palliative care] Med Clin (Barc) 124 2005 651 653 [21] Y. Arai N. Tamiya E. Yano [The short version of the Japanese version of the Zarit Caregiver Burden Interview (J-ZBI_8): its reliability and validity] Nippon Ronen Igakkai Zasshi 40 2003 497 503 [22] D.M. Goodwin I.J. Higginson K. Myers H.-R. Douglas C. Normand Effectiveness of palliative day care in improving pain, symptom control and quality of life J Pain Symptom Manage 25 2003 202 212 [23] I.J. Higginson N. Donaldson Relationship between three palliative care outcome scales Health Qual Life Outcomes 2 2004 68 75 [24] D. Jackson L. Turner-Stokes J. Murray M. Leese K.M. McPherson Acquired brain injury and dementia: a comparison of carer experiences Brain Inj 23 2009 1 12 [25] S. Banerjee J. Murray B. Foley L. Atkins J. Schneider A. Mann Predictors of institutionalisation in people with dementia J Neurol Neurosurg Psychiatry 74 2003 1315 1316 [26] E.S. Knowles C.A. Condon Why people say “Yes”: a dual-process theory of acquiescence J Pers Soc Psychol 77 1999 379 386 [27] J.P. Shaffer Multiple hypothesis-testing Annu Rev Psychol 46 1995 561 584 [28] C.J. Whitlatch S.H. Zarit E.A. von Efficacy of interventions with caregivers: a reanalysis Gerontologist 31 1991 9 14 [29] R. Hebert G. Bravo M. Preville Reliability, validity and reference values of the Zarit Burden Interview for assessing informal caregivers of community-dwelling older persons with dementia Can J Aging 19 2000 494 507 [30] B.G. Knight L.S. Fox C.P. Chou Factor structure of the burden interview J Clin Geropsychol 6 2000 249 258 [31] N. O'Rourke H.A. Tuokko Psychometric properties of an abridged version of The Zarit Burden Interview within a representative Canadian caregiver sample Gerontologist 4 2003 121 127 [32] C.B. Terwee S.D. Bot M.R. de Boer D.A. van der Windt D.L. Knol J. Dekker Quality criteria were proposed for measurement properties of health status questionnaires J Clin Epidemiol 60 2007 34 42 [33] J.C. Nunnally I.H. Bernstein Psychometric theory 1994 McGraw-Hill New York [34] M. Coffin S. Sukhatme Receiver operating characteristic studies and measurement errors Biometrics 53 1997 823 837 [35] M.C. Weinstein H.V. Fineberg A.S. Elstein H.S. Frazier D. Neuhauser R.R. Neutra Clinical decision analysis 1980 WB Saunders Philadelphia [36] J.W. Lim B. Zebrack Caring for family members with chronic physical illness: a critical review of caregiver literature Health Qual Life Outcomes 2 2004 50 [37] R. Harding I. Higginson Working with ambivalence: informal caregivers of patients at the end of life Support Care Cancer 9 2001 642 645 [38] P.M. Bossuyt J.B. Reitsma D.E. Bruns C.A. Gatsonis P.P. Glasziou L.M. Irwig Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative Ann Intern Med 138 2003 40 44 [39] R.H. Fletcher S.W. Fletchher Prevention R.H. Fletcher S.W. Fletchher Clinical epidemiology: the essentials 2005 Lippincott Williams & Wilkins Baltimore, MD 158 161 [40] B. Kirshner G. Guyatt A methodological framework for assessing health indices J Chronic Dis 38 1985 27 36