Critical appraisal

To browse other articles on a range of HSL topics, see the wiki index.

Introduction

See also Journal clubs, Major clinical studies & trial types and Research for librarians - portal

• Critical appraisal is the process of examining information and research to judge its quality and value - and, ultimately its generalizability (see also Critical thinking)
• Research questions require appropriate study designs; the best study design for evaluating medical treatments is the randomized controlled trial (RCT) (the gold standard)
• Librarianship generates some RCTs (and systematic reviews) but in small quantities
• Some studies are subject to bias; we must take steps to minimize bias by control groups, randomization techniques and blinding
• Does the study add anything new to the body of literature? Was a literature review conducted? Are methods explicit?
• Systematic reviews, which collect, appraise and combine evidence, are useful at pooling studies to determine overall impact

Critical appraisal is an essential skill for evidence-based library and information practice (EBLIP) for it allows librarians to find and use research evidence reliably and efficiently. However, librarians do not always have ease of access to best evidence in their field. In order to make decisions about information practices, health librarians also need to develop their critical appraisal skills beyond those that are discussed by proponents of EBLIP. Research involves gathering, collating and analyzing data to identify patterns and trends. However, not all research has been conducted using reliable methods and some studies are biased. This can lead to false conclusions; how can we tell whether research has been done properly and that the information it reports is reliable and trustworthy? How can we trust research when it comes to conclusions that contradict previous research? This is where critical appraisal is indispensible as we read the library and information science (LIS) literature.

• National Collaborating Centre for Methods and Tools. Registry of Methods and Tools: knowledge translation methods and tools for public health

Websites

• Cardiff University Critical Appraisal of Health Care Literature Guidelines http://www.cardiff.ac.uk/insrv/resources/guides/inf083.pdf
• Centre for Evidence Based Medicine Systematic Review Appraisal Sheets http://www.cebm.net/index.aspx?o=1157
• Critical Appraisal of Research Evidence (Ontario Public Health Libraries Association) http://www.health.gov.on.ca/english/providers/program/pubhealth/oph_standards/ophs/progstds/pdfs/caore.pdf
• International Centre for Allied Health Evidence Critical Appraisal Tools http://www.unisa.edu.au/cahe/Resources/CAT/default.asp
• Stanford University - How to evaluate a systematic review or meta-analysis http://emed.stanford.edu/education/Tutorials/Tutorial4.htm
• University of Kent's Critical Appraisal of Journal Literature http://www.kent.ac.uk/library/subjects/healthinfo/critapprais.html

Appraisal tools

See Appraisal tools at the NHS

When reading any research be it a systematic review, RCT, economic evaluation or other study design – it is important to remember that there are at least three things to consider in your critical appraisal. The validity of the study, its results and the larger relevance of the research to practice. It is necessary to consider the following questions as you assess research:

1. Has the research been conducted so as to minimize bias?
2. If so, what does the study show exactly? Are the results generalizable to other contexts?
3. What do the results mean for a particular patient (or library context) in which a decision is being made?

• See EBLIP Critical Appraisal Checklist. Lindsay Glynn, MLIS, Memorial University of Newfoundland
• Qualitative studies - critical appraisal case study

References

• Audunson R. Is that really so? Some guidelines when evaluating research. Oslo University College, Oslo, Norway.
• Ashcroft, L., & McIvor, S. (2000). LIS research and publishing: The forces of change. Library Review, 49(9), 461-469.
• Association of College and Research Libraries Task Force on Institutional Priorities and Faculty Rewards. (1998). Academic librarianship and the redefining scholarship project.
• Booth, Andrew. (2007). Who will appraise the appraisers--The paper, the instrument and the user. Health Information & Libraries Journal. 24(1)72-76.
• Boyer, E. L. (1997). Scholarship reconsidered: Priorities of the professoriate (1st ed.). San Francisco: Jossey-Bass Publishers.
• Bruce, C. (2001). Faculty-librarian partnerships in Australian higher education: Critical dimensions. Reference Services Review, 29(2), 106-116.
• Christiansen, L., Stombler, M., & Thaxton, L. (2004). A report on librarian-faculty relations from a sociological perspective. Journal of Academic Librarianship, 30(2), 116-121.
• Gerding SK, MacKellar PH. Grants for libraries: a how-to-do-it manual. New York : Neal-Schuman, 2006.
• Harker E. (2009). Evaluation of teaching and training sessions for maximum impact. Health Information & Libraries Journal. 26(3):252-4.
• Iverson, L. (2002). Report on eLibrary@UBC 4: Research, collaboration and the digital library - visions for 2010. D-Lib Magazine, 8(12).
• Juni, P, Altman D.G., & Egger, M. (2001). Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ, 322(7302), 42–46.
• Koehler W. Why Research Methods for Librarians? Valdosta State University, Valdosta, USA.
• Lougee, W. P. (2002). Diffuse libraries: Emergent roles for the research library in the digital age. Perspectives on the evolving library. U.S. District of Columbia: Council on Library and Information Resources.
• Marjanovic S, Hanney S, Wooding S. A historical reflection on research evaluation studies their recurrent themes and challenges. 2009 RAND Corporation.
• Medical Library Association. Role of expert searching in health sciences libraries. J Med Libr Assoc. 2005 Jan; 93(1):42–4.
• McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):44.–80.
• Neal, J. G. (2006). The research and development imperative in the academic library: Path to the future. Portal, 6(1), 1-3.
• St Clair, G., Miller, R., & Fiander, P. M. (2000). Academic librarianship and the redefining scholarship project. Serials Librarian, 38(1/2), 63-67.
• Stelle V, Elder SD. Becoming a fundraiser : the principles and practice of library development. Chicago: American Library Association, 2nd ed., 2000.
• University of Toronto. Writing in the Health Sciences: a comprehensive guide, 2009.
• Xin, C. & Feenberg, A. (2006). Pedagogy in Cyberspace: The Dynamics of Online Discourse. Journal of Distance Education 21 (2):1-25.