Abstract:BACKGROUND: Patient safety concerns have focused attention on organisational and safety cultures, in turn directing attention to the measurement of organisational and safety climates.OBJECTIVES: First, to compare levels of agreement between survey- and observation-based measures of organisational and safety climates/cultures and to compare both measures with criterion-based audits of the quality of care, using evidence-based markers drawn from national care standards relating to six common clinical conditions. (This required development of an observation-based instrument.) Second, to examine whether observation-based evaluations could replace or augment survey measurements to mitigate concerns about declining response rates and increasing social desirability bias. Third, to examine mediating factors in safety and organisational climate scores.DESIGN: The study had three strands: (A) a postal questionnaire survey to elicit staff perceptions of organisational and safety climates, using six prevalidated scales; (B) semistructured non-participant observation of clinical teams; and (C) a retrospective criterion-based audit carried out by non-clinical auditors to minimise hindsight bias. Standardised summary scores were created for each strand, and pairs of measurements were compared (strand A with strand B, strand A with strand C, and strand B with strand C) using Bland-Altman plots to evaluate agreement. Correlations were also examined. Multilevel modelling of Strand A scores explored mediating factors.SETTING AND PARTICIPANTS: Eight emergency departments and eight maternity units in England, UK.INTERVENTIONS: None.MAIN OUTCOME MEASURES: Examination of feasibility, correlation and agreement.RESULTS: Strand A: the overall response rate was 27.6%, whereas site-specific rates ranged from 9% to 47%. We identified more mediating factors than previous studies; variable response rates had little effect on the results. Organisational and safety climate scores were strongly correlated (r = 0.845) and exhibited good agreement [standard deviation (SD) differences 0.449; 14 (88%) within ± 0.5; one large difference]. Two commonly used scales had high levels of positive responses, suggesting positive climates or social desirability bias. Strand B: scoring on a four-point scale was feasible. Observational evaluation of teamwork culture was good but too limited for evaluating organisational culture. Strand C: a total of 359-399 cases were audited per condition. The results varied widely between different markers for the same condition, so selection matters. Each site performed well on some markers but not others, with few consistent patterns. Some national guidelines were contested. Comparisons: the comparison of safety climate (survey) and teamwork culture (observation) revealed a moderately low correlation (r = 0.316) and good agreement [SD differences 1.082; 7 (44%) within ±0.5; one large difference]. The comparison of safety climate (survey) and performance (audit) revealed lower correlation (r = 0.150, i.e. relationship not linear) but reasonably good agreement [SD differences 0.992; 9 (56%) within ± 0.5; two large differences]. Comparisons between performance (audit) and both organisational climate (survey) and teamwork culture (observation) showed negligible correlations (< 0.1) but moderately good agreement [SD differences 1.058 and 1.241; 6 (38%) and 7 (44%) within ± 0.5; each with two large differences (at different sites)]. Field notes illuminated large differences.CONCLUSIONS: Climate scores from staff surveys are not unduly affected by survey response rates, but increasing use risks social desirability bias. Safety climate provides a partial indicator of performance, but qualitative data are needed to understand discrepant results. Safety climate (surveys) and, to a lesser degree, teamwork culture (observations) are better indicators of performance than organisational climate (surveys) or attempts to evaluate organisational culture from time-limited observations. Scoring unobtrusive, time-limited observations to evaluate teamwork culture is feasible, but the instrument developed for this study needs further testing. A refined observation-based measure would be useful to augment or replace surveys.FUNDING: The National Institute for Health Research Health Technology Assessment programme.

Psychometric properties of the full and short version Nursing Home Survey on Patient Safety Culture (NHSOPSC) instrument: a cross-sectional study assessing patient safety culture in Norwegian homecare services

Psychometric properties of the Norwegian version of the hospital survey on patient safety culture in a prehospital environment

Employees’ perceptions of patient safety culture in Norwegian nursing homes and home care services

The safety attitudes questionnaire – ambulatory version: psychometric properties of the Norwegian translated version for the primary care setting

Psychometric properties and criterion related validity of the Norwegian version of hospital survey on patient safety culture 2.0

Psychometric properties of the Indonesian version of the nursing home survey on patient safety culture

Psychometric Evaluation of the Korean Version of Hospital Survey on Patient Safety Culture

Patient Safety Climate in Danish Primary Care: Adaption and Validation of the Danish Safety Attitudes Questionnaire (SAQ-DK-PRIM)

Sound psychometric properties of a short new screening tool for patient safety climate: applying a Rasch model analysis

A psychometric evaluation of the Chinese version of the nursing home survey on patient safety culture

Transcultural adaptation and psychometric study of the French version of the nursing home survey on patient safety culture questionnaire

The role of transformational leadership, job demands and job resources for patient safety culture in Norwegian nursing homes: a cross-sectional study

Adaptation and validation of the evidence-based practice profile (EBP2) questionnaire in a Norwegian primary healthcare setting

A methodological study to compare survey-based and observation-based evaluations of organisational and safety cultures and then compare both approaches with markers of the quality of care.

The Nordic Patient Experiences Questionnaire (NORPEQ): cross-national comparison of data quality, internal consistency and validity in four Nordic countries

Norwegian translation, cultural adaption and testing of the Person-centred Practice Inventory – Staff (PCPI-S)

Development and Validation of a Brief Culture-of-Safety Survey

Transcultural adaptation and psychometric evaluation of the mainland China version of Nursing Home Survey on Patient Safety Culture Questionnaire: a cross-sectional survey based on 50 nursing homes in China

Home Care Nurses’ Experiences of the Use and Introduction of the Subacute Functional Decline in the Elderly Instrument

Psychometric evaluation of the Persian version of the Hospital Survey on Patient Safety Culture Questionnaire (HSOPSC) in neonatal intensive care units: a methodological study

Patient Safety Culture and Safety Attitudes in the Estonian Context: Simultaneous Bilingual Cultural Adaptation and Validation of Instruments