In a before-and-after study, you can find out if, and to what extent, an outcome changes in your sample on average after an intervention. For most interventions, it is reasonable to assume it will work better for some people and in some contexts, but not others. Realist evaluation is an approach to explain how and why an intervention works, for whom and under what circumstances. Therefore, you can use realist evaluation:

  • when an intervention seems to work, but it is not fully understood why
  • when an intervention produces mixed results, and you wish to disentangle for whom or in which contexts it works

Conducting a realist evaluation includes the following steps:

  1. Develop an initial programme theory. The initial programme theory describes how you expect the intervention (or programme) to lead to its effects and in which conditions it should do so based on research, knowledge, experience and the assumptions of the intervention designers (Marchal et al., 2016).
  2. Choose the evaluation methods. You can choose different qualitative and quantitative methods depending on the type of data you require to test your programme theory. These can include interviews, focus groups, survey data and relevant documents.
  3. Use realist data analysis. In realist data analysis, qualitative data are coded and the appropriate statistical analyses for quantitative data are applied. The process of analysis results in a set of context-mechanism-outcome-statements like: ‘In this context, this mechanism generates this outcome’ (The RAMESES II Project, 2017).

For more information about realist evaluation please explore the resources below.

(Author: Leonie Ader)

What is it?

Videos:

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

Realist evaluation explained by McMaster Codesign (2021)

This video gives a basic introduction to realist evaluation and explains when and how to use the method. First, the underlying philosophical concept of realism is defined. Then it is explained how it can be applied to realist evaluation including examples.

(Academic reference: McMaster Codesign (2021, June 17). Realist evaluation explained [Video]. YouTube. https://www.youtube.com/watch?v=gybh7gkvjrc&ab_channel=McMasterCodesign)

Manuals and Guides:

A brief introduction to realist evaluation by Public Health England (2021)

This manual explains the most important terms and concepts in realist evaluation over six pages. It also provides a short overview of the steps involved when carrying out a realist evaluation.

(Academic reference: Public Health England. (2021). A brief introduction to realist evaluation [White paper]. Crown. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1004663/Brief_introduction_to_realist_evaluation.pdf)

Websites:

The RAMESES II project by The RAMESES Projects (2022)

The website provides useful training materials that teach the background, basic concepts, methods and reporting standards of realist evaluation. Scroll down to ‘Resources and training materials for realist evaluation’ to find short PDF-documents for each topic.

(Academic reference: The RAMESES Projects (2022). The RAMESES II project. https://www.ramesesproject.org/Standards_and_Training_materials.php#RAMESES_II)

Articles:

Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review by Katherine Salter and Anita Kothari (2014)

This article contains a useful Figure 1. It’s a helpful resource if you want to see how realist evaluations work in a diagram. Please note that ‘C-M-O configurations’ refers to how things work (‘M’ for mechanisms), where they work (‘C’ for context), and what happens as a result (‘O’ for outcomes). It is a ‘configuration’ because you are looking at how ‘C’, ‘M’, and ‘O’ interact with each other.

(Academic reference: Salter, K.L. & Kothari, A. (2014). Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implementation Science, 9(115). https://doi.org/10.1186/s13012-014-0115-y)

How is it done?

Videos:

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

Incorporating contexts and mechanisms in addressing inequities: How to conduct a realist evaluation by PAHO TV (2021)

This is a recorded two-hour webinar from the Health Promotion and Social Determinants Unit of the Pan-American Health Organization.

(Academic reference: PAHO TV. (2021, November 18). Incorporating contexts and mechanisms in addressing inequities: How to conduct a realist evaluation [Video]. YouTube. https://www.youtube.com/watch?v=0YfwEQY9laQ)

Websites:

Realist Evaluation by Sara van Belle, Gill Westhorp, and Bruno Marchal (2016)

This website article (also available in Spanish) provides a detailed, more advanced description of the steps involved in realist evaluation including a real world example.

(Academic reference: Marchal, B., Van Belle, S., & Westhorp, G. (2016). Realist evaluation. Better Evaluation. https://www.betterevaluation.org/en/approach/realist_evaluation)

Reports:

Realist evaluation: an overview by Gill Westhorp, Ester Prins, Cecile Kusters, Mirte Hultink, Irene Guijt, and Jan Brouwers (2011)

This report summarises a seminar on realist evaluations. It explains what realist evaluations are, how they work in different situations and how they align with other methods. It also discusses how our beliefs about reality affect these evaluations. Throughout the report, there are nine examples of realist evaluations provided.

(Academic reference: Westhorp, G., Prins, E., Kusters, C.S.L., Hultink, M., Guijt, I.M., & Brouwers, J.H.A.M. (2011). Realist evaluation: an overview, Report from an Expert Seminar with Dr. Gill Westhorp. (Wageningen UR Centre for Development Innovation. https://library.wur.nl/WebQuery/wurpubs/fulltext/173918)

Method in action

Videos:

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Privacy Policy.
I Accept

Realist review film by Welsh Institute for Health and Social Care (2020)

This is a short video that explains how an organisation is using realist evaluations to help them develop a student wellbeing model. Please note in this video realist evaluations are referred to as ‘realist reviews’. If you would like to explore this further and read a more in-depth article about this project, click here.

(Academic reference: Welsh Institute for Health and Social Care. (2020, October 8). Realist review film [Video]. YouTube. https://www.youtube.com/watch?v=EhBmxi7Aom4)

Articles:

A realist evaluation of patients’ decisions to deprescribe in the EMPOWER trial by Philippe Martin and Cara Tannenbaum (2017)

This academic journal article describes the results of a realist evaluation that was conducted to evaluate an intervention to motivate chronic benzodiazepine consumers to deprescribe.

(Academic reference: Martin, P. & Tannenbaum, C. (2017). A realist evaluation of patients’ decisions to deprescribe in the EMPOWER trial. BMJ Open http://dx.doi.org/10.1136/bmjopen-2017-015959)

“You can kind of just feel the power behind what someone’s saying”: a participatory-realist evaluation of peer support for young people coping with complex mental health and substance use challenges by Tanya Halsall, Mardi Daley, Lisa Hawke, Joanna Henderson & Kimberly Matheson (2022)

This article uses realist evaluations to study a programme. The programme aims to support young people with mental health and substance use challenges. It’s a helpful read if you want to see how realist evaluations can be used to improve other research methods.

(Academic reference: Halsall, T., Daley, M., Hawke, L., Henderson, J. & Matheson, K. (2022). “You can kind of just feel the power behind what someone’s saying”: a participatory-realist evaluation of peer support for young people coping with complex mental health and substance use challenges. BMC Health Services Research, 22(1358) https://doi.org/10.1186/s12913-022-08743-3)