Are you interested in contributing to HLWIKI International? contact
To browse other articles on a range of HSL topics, see the A-Z index.
- 2 May 2016
See also Critical thinking | Journal clubs | Evidence-based health care | Research Portal for Academic Librarians | Scholarship 2.0
"...critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness and its value and relevance in a particular context." — Burls (2009) What is critical appraisal?
Critical appraisal refers to the process of carefully and systematically examining published research in order to judge its value and relevance in particular contexts. More specifically, critical appraisal of a study or article is more than simply reading it; it is a technique that aims to improve your ability to read research and assess its quality. Further it is the use of an explicit method or methods to evaluate research particularly studies and clinical trials in journals. The process of critical appraisal involves the application of rules around matters of internal validity, reporting standards, conclusions and generalizability of the research in question. Critical appraisal is a central tenet in the systematic review and used in evidence-based health care to assist in making decisions. Moreover, critical appraisal is an important lifelong learning skill when reading the research and requires some knowledge of qualitative information and its validity/ generalizability. In reading the research, whether more rigorous systematic reviews, randomized controlled trials (RCTs), economic evaluations or other, remember to consider the following questions: 1) Is external or internal validity demonstrated? 2) How has the study obtained its results? and 3) What is the study's relevance to research to practice (or within the context of knowledge synthesis in medicine).
Critical appraisal questions
Consider the following questions as you assess the research :
- Has the research been done on a patient sample before? What size? How was data collected?
- Was research conducted to minimize bias? (What is bias?)
- What does the study tell us? Are the results applicable to other contexts?
- What do the results mean for specific patients where decisions must be made?
The skills of critical appraisal are absolutely essential in an era of evidence-based health care:
- Research questions typically require appropriate study designs; the best study design for testing medical interventions is the randomized controlled trial (RCT)
- Information science (and health librarianship) generates some RCTs (and systematic reviews) but in small quantities
- Some studies are subject to bias; we must take steps to minimize bias by control groups, randomization techniques and blinding
- Does the study add anything new to the body of literature? Was a literature review conducted? Are methods explicit?
- Systematic reviews, which collect, appraise and combine evidence, are useful at pooling studies to determine overall impact see Meta-analysis
Towards EBLIP in health librarianship
Critical appraisal is an essential skill for evidence-based library and information practice (EBLIP) for it allows librarians to find and use research evidence reliably and efficiently. However, librarians do not always have ease of access to best evidence in their field. In order to make decisions about information practices, health librarians also need to develop their critical appraisal skills beyond those that are discussed by proponents of EBLIP. Research involves gathering, collating and analyzing data to identify patterns and trends. However, not all research has been conducted using reliable methods and some studies are biased. This can lead to false conclusions; how can we tell whether research has been done properly and that the information it reports is reliable and trustworthy? How can we trust research when it comes to conclusions that contradict previous research? This is where critical appraisal is indispensable as we read the library and information science (LIS) literature.
Key websites & video
Appraisal guides and tools
- Audunson R. Is that really so?: some guidelines when evaluating research. Oslo University College, Oslo, Norway.
- Ashcroft L, McIvor S. LIS research and publishing: the forces of change. Library Review. 2000;49(9):461-469.
- Association of College and Research Libraries Task Force on Institutional Priorities and Faculty Rewards. Academic librarianship and the redefining scholarship project; 1998.
- Booth A, Brice A. Increasingly the health information professional's role in supporting evidence-based practice requires familiarity with critical appraisal skills, resources and techniques. Health Info Libr J. 2001 Sep;18(3):175-7.
- Booth A. Who will appraise the appraisers: the paper, the instrument and the user. Health Info Libr J. 2007;24(1)72-76.
- Boyer EL. Scholarship reconsidered: Priorities of the professoriate (1st ed.). San Francisco: Jossey-Bass Publishers; 1997.
- Bruce C. Faculty-librarian partnerships in Australian higher education: Critical dimensions. Ref Serv Rev. 2001;29(2):106-116.
- Christiansen L, Stombler M. Thaxton L. A report on librarian-faculty relations from a sociological perspective. J Acad Librarianship. 2004;30(2):116-121.
- Cook DA, Enders F, Linderbaum JA, Zwart D, Lloyd FJ. Speed and accuracy of a point of care web-based knowledge resource for clinicians: a controlled crossover trial. Interact J Med Res 2014;3(1):e7.
- Danermark B, Ekstrom M, Jakobsen L. Explaining society: an introduction to critical realism in the social sciences. Routledge; 2001 Nov 22.
- Harker E. Evaluation of teaching and training sessions for maximum impact. Health Info Libr J. 2009;26(3):252-4.
- Iverson L. Report on eLibrary@UBC 4: Research, collaboration and the digital library - visions for 2010. D-Lib Magazine. 2002;8(12).
- Jeanfreau SG, Jack L Jr. Appraising qualitative research in health education:guidelines for public health educators. Health Promot Pract. 2010 Sep;11(5):612-7.
- Jakubec SL, Astle BJ. Students connecting critical appraisal to evidence-based practice: a teaching-learning activity for research literacy. J Nurs Educ. 2013 Jan;52(1):56-8.
- Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;322(7302):42–46.
- Lougee WP. Diffuse libraries: emergent roles for the research library in the digital age. Perspectives on the evolving library. Council on Library and Information Resources; 2002.
- Maden-Jenkins M. Healthcare librarians and the delivery of critical appraisal training: barriers to involvement. Health Info Libr J. 2011 Mar;28(1):33-40.
- Maden-Jenkins M. Healthcare librarians and the delivery of critical appraisal training: attitudes, level of involvement and support. Health Info Libr J. 2010 Dec;27(4):304-15.
- Marjanovic S, Hanney S, Wooding S. A historical reflection on research evaluation studies their recurrent themes and challenges. 2009 RAND Corporation.
- Medical Library Association. Role of expert searching in health sciences libraries. J Med Libr Assoc. 2005;93(1):42–4.
- McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):44–80.
- Neal JG. The research and development imperative in the academic library: path to the future. Portal. 2006;6(1):1-3.
- Shariff SZ, Bejaimal SAD, Sontrop JM, Iansavichus AV, Haynes RB, Weir MA, Garg AX. Retrieving clinical evidence: a comparison of PubMed and Google Scholar for quick clinical searches. J Med Internet Res. 2013;15(8):e164.
- Stelle V, Elder SD. Becoming a fundraiser: the principles and practice of library development. Chicago: American Library Association, 2000.
- Tieman JJ, Bradley S. Systematic review of the types of methods and approaches used to assess the effectiveness of healthcare information websites. Australian Journal of Primary Health. 2013;19(4):319-24.
- University of Toronto. Writing in the Health Sciences: a comprehensive guide, 2009.
- Xin C, Feenberg A. Pedagogy in cyberspace: the dynamics of online discourse. J Distance Education. 2006;21(2):1-25.