|Are you interested in contributing to HLWIKI International? contact
To browse other articles on a range of HSL topics, see the A-Z index.
- 2 February 2018
- critical realist methodology, realist methodology, realist evaluation, realist synthesis/reviews
- See also: rapid review, rapid realist review, rapid approach, rapid synthesis, meta-method, meta-evaluation, rapid evidence assessment, expedited review, accelerated review and realist review
See also Evidence-based health care | Expert searching | Hand-searching | Network meta-analysis | Positivism | Systematic review searching | Systematic narrative review methods
- "..[the] realist review is a theory-driven interpretive approach to evidence synthesis. [As such] it applies realist logic of inquiry to produce an explanatory analysis of an intervention that is, what works, for whom, in what circumstances, in what respects. The search strategy will have a number of phases including a combination of: (1) electronic database searching, for example, EMBASE, MEDLINE, the Cochrane Library, ASSIA, (2) 'cited by' articles search, (3) citation searching, (4) contacting authors and (5) grey literature searching... — Brennan et al, 2014
- "...the realist review was originally developed by Pawson, 2005 for complex social interventions to explore systematically how contextual factors influence the link between intervention and outcome
(summed up in the question "what works, how, for whom, in what circumstances and to what extent?")..." — Greenhalgh et al, 2011
- "...a realist review differs from empirically focused qualitative or quantitative methods in a number of ways, including its theory-driven and abductive (informed-intuitive) approach to understanding context, mechanism, and outcome..." — Jagosh et al, 2012
Realist reviews are a new form of knowledge synthesis that opens the ‘black box’ of an intervention by showing how it triggers mechanisms in specific contexts to produce outcomes. Their aim is to produce middle-range theories (MRTs) that specify how interventions work, for which populations, and under what circumstances (Pawson et al. 2005; Pawson 2006). MRTs (Merton 1968) are considered a suitable level of abstraction to maintain the operationality needed for applied research while producing cross-cutting lessons (Weick 1989; Hercot et al. 2011), especially on access to care (Dixon-Woods et al. 2006). Further, RRs are a way for researchers to examine the complexity of programs in medicine, and seek to understand "...what works, for who, in what circumstances and why" rather than providing one-off verdicts on the success or failure of a program (or intervention) (Pawson and Tilley 2007). There is growing interest in theory-driven, qualitative and mixed-method approaches to systematic reviews as an alternative to conventional Cochrane-style reviews. Realist reviews and evaluation offer the potential to expand the knowledge base in policy-relevant areas - for example by explaining the success, failure or mixed fortunes of complex interventions. Realist evaluation recognizes the value in drawing on a wide range of evidence, including qualitative research, grey literature and the insights of program staff. In each case it seeks to attend to the relative rigour of the data gathered, and looks for insights that will illuminate the theory of change, and as with other evaluation methods, triangulation across different kinds of evidence is vital for drawing relatively robust conclusions.
The term realist has a number of connotations in medicine. Researchers make distinctions among the types of scientific reviews that might be viewed as realist. Hammersley distinguishes ‘naïve realism’ and ‘subtle realism’ whereas Pawson and Tilley say ‘scientific realism’ is what is relevant in medicine. Holden uses realism in the way Hammersley uses it (‘naïve realism’) which some say is simply a form of positivism. ‘Subtle realism’, critical realism, scientific realism are all constructed to sit between positivism and constructivism. That said, a) words have many meanings, b) interpretations vary and evolve, and c) ‘there’s no such thing as final truth or knowledge’ (a realist tenet is that all knowledge is tentative). Thus there is no ‘correct’ description except as one might ensure that something can be understood by what one author is discussing at any one time. Finally there are at least two meanings for ‘stratified reality’; one of ‘systems within systems’ in that every system comprises sub-systems and is part of a larger system (regardless of whether one is discussing material reality, social reality or ideas); the other is Bhaskar’s philosophical construct of the empirical, the actual and the real. Both are thought to be necessary to understand the concept of realism.
Ray Pawson discusses 'realist reviews'
See also Pawson R. The Realist Foundations of Evidence-Based Medicine: A Review Essay. Evaluation. 2018 Jan;24(1):42-50.
Pawson's Five (5) Practice Steps
Pawson’s five practical steps for conducting a realist review are:
- clarifying the scope of the review
- determining the search strategy, including adopting broad inclusion/exclusion criteria and purposive snowballing techniques
- ensuring proper article selection and study quality assessment using multiple methods, #extracting and organising data through the process of note taking, annotation and conceptualization and
- synthesizing the evidence and drawing conclusions through a process of reasoning.
Literature Searching for Realist Reviews
With respect to the search strategy in the realist review, Mogre et al (2014) suggest the following: "...A combination of MeSH/thesauri and free text terms will be employed in the search strategy. Truncation and appropriate Boolean operators will be employed. Scoping searches will be conducted to refine the search terms further...". For more background, see Literature searching for realist reviews. The Realist Hive blog. June 2014 and Literature searching for realist reviews (Simon Briscoe)
Realist reviews & their impact
This introduction to realist reviews was developed to help evaluators decide whether a realist impact evaluation is appropriate and feasible for a particular policy or program. It does rehash introductory material about realist approaches but also addresses more ‘applied’ questions and issues, such as
- For what purposes is realist impact evaluation most appropriate?
- In what circumstances is a realist approach more appropriate than other methods?
- What circumstances make it easier to undertake a realist impact evaluation?
- Understanding program theory and designing realist evaluations
The paper is available at either of the following:
Issues in realist reviews
- As a researcher, if you have identified an outcome of interest, and worked back to understand the underlying mechanisms that have manifested within key contextual conditions, and identified and developed relevant (mid-range) theories to support your explanatory analysis, you have indeed used a realist review methodology; check your methods back against the Rameses standards and decide whether or not you have in fact modified them.
- Realist reviews help to recognize the complex array of factors involved in production of interventions, but researchers are always working with partial knowledge so they are justified to pick one specific outcome and theorize the intervention around that outcome. Describe it clearly and provide your rationale.
- A realist review does not have to be 'completely comprehensive'; emphasize the theoretical output of your work and how it contributes to cumulation of knowledge in the field over time; include a strong section on 'direction for future research' which can be informed by the results of your review. Transparency in your reporting is key.
- Realist reviews do not necessarily flow from previous studies; usually SRs, for example, do not reveal the information needed to do a realist review
- realist reviewers do not need to report all of a previous SR; report only what's relevant to the realist question under consideration
- one question might be: For what aspect(s) of your rough theory of change does the previous review provide evidence?
- What evidence does the previous review provide? Is it about outcomes? outcomes for whom? outcomes for particular contexts?
- How does the evidence in the previous review 'fit' (or not) with the evidence from other sources? What analytical processes will you use to make sense of the relationship between that evidence and the evidence from other sources? (EG Pawson 2006, p 74-76: juxtaposing, reconciling, adjudicating, consolidating, situating)
- How does the evidence from the previous review suggest that your rough theory should be refined?
- If you can move towards answering these questions, you would better know 'where' in your synthesis the information from the previous review fitted, what information I'd be drawing from it, how to approach the analysis, and what it implied for your findings
- Realist methodologies have been gaining ground in primary healthcare research and in other disciplines; it helps to think of RMs as 'scientific outlooks' rather than a set of methods which can really be applied to research questions in most disciplines.;
- As you may know RMs are inspired by the philosophy of critical realism, and transformed into an applied form through the sociological expertise of Ray Pawson and Nick Tilley (1997) and by the realist research community. RMs are being applied innovatively in many fields.
- Creating sustained change with RMs is a challenge because there is a complex host of factors that play into the success or failure of sustainability. A realist approach can support sense-making in understanding how sustainability is achieved, for whom, and under what circumstances/context.
- For example, in linked Context-Mechanism-Outcome configurations, where outcomes from one phase create changes in another, there is a time sensitive analysis where 'ripple effects' create unexpected outcomes, all of which shape the nature and extent of sustainability in certain settings.
The research into conducting a realist review in medicine and nursing is in a very fertile period. Meta-narrative reviews are one of the emerging approaches to qualitative and mixed-methods systematic reviews. A meta-narrative review seeks to illuminate a heterogeneous topic by highlighting contrasting and complementary ways researchers have studied the topic. The standards below will guide researchers according; according to the authors, no previous publication standards exist for the reporting of meta-narrative reviews. See also Systematic narrative review methods
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: meta-narrative reviews. J Adv Nurs. 2013;
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. J Adv Nurs. 2013;
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: meta-narrative reviews. BMC Medicine 2013:11:20.
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Medicine 2013;11;21.
- This listserv is for researchers interested in learning about realist and meta-narrative reviews. The group hopes to build a community that enables peer-to-peer knowledge construction ultimately leading to a community of members with greater understanding and competence in undertaking these types of reviews. If you would like to join this list, please email Geoff Wong.
Realist Synthesis Training
- Ongoing trials and tribulations of applying realist methods in primary research and research synthesis – aiming to promote understanding of the method and give insight into methodological developments.
- Some examples of realist reviews published and indexed in PubMed...
- Bhasker RA. A realist theory of science. London: Verso, 1997.
- Boote J, Baird W, Sutton A. Public involvement in the systematic review process in health and social care: a narrative review of case examples. Health Policy. 2011 Oct;102(2-3):105–16.
- Brennan N, Bryce M, Pearson M, Wong G, Cooper C, Archer J. Understanding how appraisal of doctors produces its effects: a realist review protocol. BMJ Open. 2014 Jun 23;4(6):e005466.
- Collier A. Critical realism: an introduction to Roy Bhaskar's philosophy. London: Verso, 1994.
- Danermark B, Ekstrom M, Jakobsen L. Explaining society: an introduction to critical realism in the social sciences. Routledge; 2001 Nov 22.
- Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol--realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011 Aug 16;11:115.
- Greenhalgh T, Potts HW, Wong G. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q. 2009;87(4):729-88.
- Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005 Nov 5;331(7524):1064-5.
- Greenhalgh T, Robert G, Macfarlane F, Bate P. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005 Jul;61(2):417-30.
- Hammersley M. Ethnography and realism. Qualitative Researcher’s Companion. 2002.
- Hardwick R, Pearson M, Byng R, Anderson R. The effectiveness and cost-effectiveness of shared care: protocol for a realist review. Systematic Reviews. 2013;2:12.
- Hercot D, Meessen B, Ridde V, Gilson L. Removing user fees for health services in low-income countries: a multi-country review framework for assessing the process of policy change. Health Policy Plan. 2011 Nov;26 Suppl 2:ii5-15.
- Hutchison BG. Critical appraisal of review articles. Can Fam Physician. 1993;39:1097-102.
- Jagosh J, Macaulay AC, Pluye P, Salsberg J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Quarterly. 2012;90(2).
- Khangura et al. Evidence summaries: the evolution of a rapid review approach. Systematic Reviews. 2012;1:10.
- Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106:485–8.
- Jagosh et al. Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. Res Syn Meth. 2014;5(2):131–141.
- Jones K. Mission drift in qualitative research, or moving toward a systematic review of qualitative studies, moving back to a more systematic narrative. Qualitative Report. 2004;9(1):95-112.
- Kreis J, Puhan MA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2012 Mar 6.
- Long AF, Godfrey M, Randall T, Brettle A, Grant MJ. HCPRDU Evaluation tool for mixed methods studies , University of Leeds, Nuffield Institute for Health, Leeds, 2002.
- Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016;1:19.
- Marchal , et al. Realist RCTs of complex interventions – an oxymoron. Soc Sci Med. 2013 July.
- Merton RK. On sociological theories of the middle range. In: Merton RK. (ed) Social Theory and Social Structure. New York, NY: Free Press, 1968.
- Mishler EG. Models of narrative analysis: a typology. J Narrative Life History. 1995;5(2):87-123.
- Mogre V, Scherpbier A, Dornan T, Stevens F, Aryee PA, Cherry MG. A realist review of educational interventions to improve the delivery of nutrition care by doctors and future doctors. Syst Rev. 2014 Dec 20;3:148.
- Oxman AD. Checklists for review articles. BMJ. 1994;309:648-51.
- Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44(11):1271-8.
- Oxman AD, Guyatt GH, Singer J, Goldsmith CH, Hutchison BG, Milner RA. Agreement among reviewers of review articles. J Clin Epidemiol. 1991;44(1):91-8.
- Oxman AD, Guyatt GH. Guidelines for reading literature reviews. CMAJ. 1988;138:697–703.
- Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18(2):176–191.
- [Pawson R. Digging for nuggets: how ‘bad’ research can yield ‘good’ evidence’. Int J Soc Res Methods. 2006;9(2:127-142.]
- Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions. J Health Services Research and Policy. 2005;10(S1):21-34.
- Pawson R: Nothing as practical as a good theory. Evaluation. 2003;9:471-490.
- Pawson R. Mentoring relationships: an explanatory review. Working Paper 21. ERSC UK Centre for Evidence based Policy and Practice, 2004.
- Pawson R. Evidence-based policy: a realist perspective. London: Sage, 2006.
- Pawson R. Evidence-based policy: the promise of realist synthesis. Evaluation. 2002;8(3):58..
- Pawson R, Greenhalgh P, Harvey G, Walshe K. Realist synthesis: an introduction. ERSC Research Methods Programme Papers. 2004:2.
- Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system forappraising mixed methods research, and concomitantly appraising qualitative,quantitative and mixed methods primary studies in Mixed Studies Reviews. Int J Nurs Stud. 2009 Apr;46(4):529-46.
- Porpora DV. Reconstructing sociology: the critical realist approach. Cambridge University Press, 2015.
- Porter S. The uncritical realism of realist evaluation. Evaluation. 2015;21(1):65–82.
- Porter S. Realist evaluation: an immanent critique. Nursing Philosophy. 2015.
- Porter S, O'Halloran P. The use and limitation of realistic evaluation as a tool for evidence-based practice: a critical realist perspective. Nurs Inq. 2012 Mar;19(1):18-28.
- Pawson R. The science of evaluation: a realist manifesto. London: Sage, 2013.
- Saul JE, Willis CD, Bitz J, Best A. A time-responsive tool for informing policy making: rapid realist review. Implement Sci. 2013 Sep 5;8:103.
- Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010;10:12.
- Woolf SH. Review articles and disclosure of methods. Am J Prev Med. 1991;7:53-4.
- Weick KE. Theory construction as disciplined imagination. Academy of Management Review. 1989;14:516–531.
Realist reviews in health informatics
- Carrington JM. Meta-narrative to examine the purpose of the electronic health record. Communicating Nursing Research. 2011;44:535-535.
- Danermark B, Ekstrom M, Jakobsen L. Explaining society: an introduction to critical realism in the social sciences. Routledge; 2001 Nov 22.
- de Lusignan S. Informatics as tool for quality improvement: rapid implementation of guidance for the management of chronic kidney disease in England as an exemplar. Healthcare Informatics Research. 2013;19(1), 9-15.
- Edgley A, Stickley T, Timmons S, Meal A. Critical realist review: exploring the real, beyond the empirical. J Further Higher Ed. 2014 Sep 28:1-5.
- Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Informatics. 2010;79(11):736-771.
- Hebert MA. Realist Review approach to e-health: the case of type 2 diabetes in youth. Studies in Health Technology & Informatics. 2013;183:74-78.
- Liaw ST, Rahimi A, Ray P, Taggart J, Dennis S, de Lusignan S. et al. Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature. Int J Med Informatics. 82(1):10-24. Corrigendum in International Journal of Medical Informatics, 82(2), 139.
- Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. How outcomes are achieved through patient portals: a realist review. J Am Med Inform Assoc. 2014 Jul-Aug;21(4):751-7.
- Vichitvanichphong S, Talaei-Khoei A, Kerr D, Ghapanchi AH. Adoption of Assistive Technologies for Aged Care: A Realist Review of Recent Studies. 2014 47th Hawaii International Conference on System Sciences (Hicss), 2706-2715.