In a before-and-after study, you can find out if, and to what extent, an outcome changes in your sample on average after an intervention. For most interventions, it is reasonable to assume it will work better for some people and in some contexts, but not others. Realist evaluation is an approach to explain how and why an intervention works, for whom and under what circumstances. Therefore, you can use realist evaluation:

  • when an intervention seems to work, but it is not fully understood why
  • when an intervention produces mixed results, and you wish to disentangle for whom or in which contexts it works

Conducting a realist evaluation includes the following steps:

  1. Develop an initial programme theory. The initial programme theory describes how you expect the intervention (or programme) to lead to its effects and in which conditions it should do so based on research, knowledge, experience and the assumptions of the intervention designers (Marchal et al., 2016).
  2. Choose the evaluation methods. You can choose different qualitative and quantitative methods depending on the type of data you require to test your programme theory. These can include interviews, focus groups, survey data and relevant documents.
  3. Use realist data analysis. In realist data analysis, qualitative data are coded and the appropriate statistical analyses for quantitative data are applied. The process of analysis results in a set of context-mechanism-outcome-statements like: ‘In this context, this mechanism generates this outcome’ (The RAMESES II Project, 2017).

For more information about realist evaluation please explore the resources below.

(Author: Leonie Ader)

What is it?

Videos:

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

Realist evaluation explained by McMaster Codesign (2021)

This video gives a basic introduction to realist evaluation and explains when and how to use the method. First, the underlying philosophical concept of realism is defined. Then it is explained how it can be applied to realist evaluation including examples.

(Academic reference: McMaster Codesign (2021, June 17). Realist evaluation explained [Video]. YouTube. https://www.youtube.com/watch?v=gybh7gkvjrc&ab_channel=McMasterCodesign)

Manuals and Guides:

A brief introduction to realist evaluation by Public Health England (2021)

This manual explains the most important terms and concepts in realist evaluation over six pages. It also provides a short overview of the steps involved when carrying out a realist evaluation.

(Academic reference: Public Health England. (2021). A brief introduction to realist evaluation [White paper]. Crown. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1004663/Brief_introduction_to_realist_evaluation.pdf)

Websites:

The RAMESES II project by The RAMESES Projects (2022)

The website provides useful training materials that teach the background, basic concepts, methods and reporting standards of realist evaluation. Scroll down to ‘Resources and training materials for realist evaluation’ to find short PDF-documents for each topic.

(Academic reference: The RAMESES Projects (2022). The RAMESES II project. https://www.ramesesproject.org/Standards_and_Training_materials.php#RAMESES_II)

Articles:

Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review by Katherine Salter and Anita Kothari (2014)

This article contains a useful Figure 1. It’s a helpful resource if you want to see how realist evaluations work in a diagram. Please note that ‘C-M-O configurations’ refers to how things work (‘M’ for mechanisms), where they work (‘C’ for context), and what happens as a result (‘O’ for outcomes). It is a ‘configuration’ because you are looking at how ‘C’, ‘M’, and ‘O’ interact with each other.

(Academic reference: Salter, K.L. & Kothari, A. (2014). Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implementation Science, 9(115). https://doi.org/10.1186/s13012-014-0115-y)

What’s in a mechanism? Development of a key concept in realist evaluation by Sonia Michelle Dalkin, Joanne Greenhalgh, Diana Jones, Bill Cunningham and Monique Lhussier

This article describes the concept of mechanism in realist evaluation.

(Academic reference: Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015). What’s in a mechanism? Development of a key concept in realist evaluation. Implementation science10, 1-7. DOI 10.1186/s13012-015-0237-x)

How is it done?

Videos:

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

Incorporating contexts and mechanisms in addressing inequities: How to conduct a realist evaluation by PAHO TV (2021)

This is a recorded two-hour webinar from the Health Promotion and Social Determinants Unit of the Pan-American Health Organization.

(Academic reference: PAHO TV. (2021, November 18). Incorporating contexts and mechanisms in addressing inequities: How to conduct a realist evaluation [Video]. YouTube. https://www.youtube.com/watch?v=0YfwEQY9laQ)

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

The talk is given by researchers from the Realist Research, Evaluation and Learning Initiative (RREALI) Team at Charles Darwin University. They explore three ways for data analysis in realist evaluations.

(Academic reference: The REALI Team. (2023, November 24). The Realist Hive: Doing realist analysis three different ways in practice [Video]. YouTube. https://www.youtube.com/watch?v=tRKTQDzEtGw)

Websites:

Realist Evaluation by Sara van Belle, Gill Westhorp, and Bruno Marchal (2016)

This website article (also available in Spanish) provides a detailed, more advanced description of the steps involved in realist evaluation including a real world example.

(Academic reference: Marchal, B., Van Belle, S., & Westhorp, G. (2016). Realist evaluation. Better Evaluation. https://www.betterevaluation.org/en/approach/realist_evaluation)