Realist Evaluation

Realist evaluation or realist review (also realist synthesis) is a type of theory-driven evaluation used in evaluating social programmes.^[1] It was originally based on the epistemological foundations of critical realism. Ray Pawson, one of the originators of realist evaluation was "initially impressed" by how critical realism explains generative causation in experimental science; however, he later criticised its "philosophical grandstanding" and "explain-all Marxism".^[2]

Based on specific theories, realist evaluation provides an alternative lens to empiricist evaluation techniques for the study and understanding of programmes and policies. This technique assumes that knowledge is a social and historical product, thus the social and political context as well as theoretical mechanisms, need consideration in analysis of programme or policy effectiveness.^[3][4][5]

Realist evaluation techniques recognise that there are many interwoven variables operative at different levels in society, thus this evaluation method suits complex social interventions, rather than traditional cause-effect, non-contextual methods of analysis. This realist technique acknowledges that intervention programmes and policy changes do not necessarily work for everyone, since people are different and are embedded in different contexts.^[6]

Realist evaluation was popularised by the work of Ray Pawson and Nick Tilley in 1997.^[7] They described the procedure followed in the implementation of realist evaluation techniques in programme evaluation and emphasise that once hypotheses have been generated and data collected, the outcomes of the programme are explored, focusing on the groups that the programme benefitted and those who did not benefit. Effectiveness of a programme is thus not dependent on the outcomes alone (cause–effect), rather there is a consideration of the theoretical mechanisms that are applied, and the socio-historical context in which the programmes were implemented. Thus, the final explanation of a programme considers context-mechanism-outcome.^[8]

All research methods are applicable in realist evaluations, according to Pawson and Tilley (1997):^[9]

"... it is quite possible to carry out realistic evaluation using: strategies, quantitative and qualitative; timescales, contemporaneous or historical; viewpoints, cross-sectional or longitudinal; samples, large or small; goals, action-oriented or audit-centred; and so on and so forth."

Later work emphasised that the realist evaluation approach was an attempt to introduce basic ideas from social science to evaluation. For example (Pawson & Tilley, 2001, p. 324):^[10]

We have argued that good evaluation is good social science. For us, this embraces the gallant aims of precision in articulation of theory, rigor in empirical testing, confederation in lines of inquiry, and cumulation in the body of findings. The “realist movement,” of which we are a part, is often considered the brash upstart of the evaluation schools. In fact, it depends on these rather venerable ideas. The future, for us, thus lies in keeping faith with some of the grand old principles of social science and in not forgetting the hard-won lessons of the old studies.

Although context-mechanism-outcome configurations are often seen as the hallmark of realist evaluations, this too is argued to be pervasive across social and other sciences (Pawson, 2024, p. 42):^[11]

All scientific investigation utilises explanations relating mechanisms and contexts to empirical patterns.

A 2024 book argues that it is possible to run realist randomized controlled trials^[12] and Gill Westhorp and Simon Feeny (2024) explain the relevance of surveys and regression models (including interaction terms and covariate adjustment) to testing Context-Mechanism-Outcome configurations.^[13] This form of theory-driven evaluation has been increasingly used across a variety of different settings and research agendas including health systems and social policy.^[14][15][16] Guidelines and methodological resources on realist evaluation have been translated and made available in Spanish through the RAÍCES initiative.^[17]