Abstract:Click to increase image sizeClick to decrease image size Notes 1. Although RCTs have many powerful features, there is a continuing debate among evaluators concerning the strengths and weaknesses of RCTs. Many evaluations have revealed some of the theoretical and practical limitations of RCT designs. Some of the limitations that have been mentioned include: the designs are rigid and inflexible and do not adapt well to changes in project design or implementation (which occur frequently) or to the changes in context in which the project is implemented; mono-method bias (relying on a single data collection method and outcome indicator; construct validity (simplifying complex outcome indicators to a single indicator that is easier to manipulate); difficulties in measuring qualitative dimensions; lack of attention to contextual variables (economic, political, institutional, socio-cultural; ignoring the process of project implementation; and long delays in providing feedback to policy-makers (compared with other evaluation models that are designed to provide feedback to managers and policy-makers during implementation). There are ways to address many of these issues in mixed design. REAP also participates in, designs, and advocates evaluations that use mixed design. 2. It should be noted that having relationships with local policy-makers can be tricky. Policy-makers are expecting their programmes/policies to work. If the evaluation shows success, it is easy. If the evaluation shows the programme/project is not successful, there may be friction. The key is to make sure that policy-makers understand the ground rules in advance. It is also helpful to develop a careful logic chain analysis plan in order to try to help policy-makers understand why a certain programme did not work as originally designed. This is true in all projects. But, it is especially true in the case of collaborations between evaluation teams and policy-makers. 3. We know cases when our advice has been heard and ‘accepted’ by the State Council but rejected by the Ministry of Education. When this happened, we saw no policy change. For example, we wrote a brief on financial aid for college for poor students in poor rural areas. In China all financial aid for college is given after the student enters college. Therefore, when the student is making his/her college decision, poor students do not know whether they will receive aid or not. Because of this we showed that poor students tended to pick majors that had tuition waivers (in exchange for postgraduation obligations – for example, teaching majors do not pay tuition and fees but they must teach in rural schools for five to seven years). We showed that if poor students were given financial aid offers before the college decision was made, they would change their decision-making process a lot. In other words, the current financial aid system distorts the decisions of those that need the aid the most. We suggested giving aid decisions out earlier. The Ministry of Education (for many possible reasons) objected. Until now there has not been any change.

An Experiment on Building Evaluation Capacity to Address Health Inequities in China

Outcome-based Health Equity Across Different Social Health Insurance Schemes for the Elderly in China

Measurement and analysis of equity in health: a case study conducted in Zhejiang Province, China

Conceptual Indicators Framework for Strengthening the Chinese Health System

Evaluation and Learning in Complex, Rapidly Changing Health Systems: China's Management of Health Sector Reform.

Conducting Influential Impact Evaluations in China: the Experience of the Rural Education Action Project

PD27 A Case Study of Equity in Health from Zhejiang Province, China

Brief introduction of "Research and Practice on Performance Evaluation of Disease Control and Prevention System in China"

Evaluation and mechanism for outcomes exploration of providing public health care in contract service in Rural China: a multiple-case study with complex adaptive systems design

Building the Framework of Performance Evaluation for Disease Control and Prevention System in China

Designing evaluation studies to optimally inform policy: what factors do policy-makers in China consider when making resource allocation decisions on healthcare worker training programmes?

Building capacity for health promotion--a case study from China

Equity in health and health care: the Chinese experience

The Effects Evaluation of Health Promotion Under Different Insurance Models in Rural China

Hierarchical medical system and local medical performance: A quasi-natural experiment evaluation in Shanghai, China

Design and analysis of evaluation on discipline constructing project for health system in Shanghai

The roles of health culture and physical environment in workplace health promotion: a two-year prospective intervention study in China

Synthetic Evaluation of the Effect of Health Promotion: Impact of A Unicef Project in 40 Poor Western Counties of China

Development and application of ex-ante performance evaluation framework for health projects in China

Prioritizing the perceived equity of the residents to construct an equitable health care system: evidence from a national cross-sectional study in China

Equity Zones: Building the Bridge to Transformation