Software for systematic reviewing

From HLWIKI Canada
Jump to: navigation, search
Hierarchical view of "evidence"
Creative Commons source:
Are you interested in contributing to HLWIKI International? contact:

To browse other articles on a range of HSL topics, see the A-Z index.


Last Update

  • Updated.jpg 3 February 2014


See also Bibliographic citation software | Expert searching | Filters (ie. hedges) | Hand-searching | Network meta-analysis | Systematic reviews

What is the purpose of a systematic review (SR)?

"...a systematic review seeks to identify as many potentially relevant studies as possible that meet the research question for a given topic" — NHS Centre for Reviews and Dissemination

Before starting your systematic review

Before undertaking a systematic review, it is recommended that you read a good introductory handbook on the topic to get guidance on starting your review. When first planning your systematic review, consider registering the protocol. Registration improves transparency of research, and prevents unnecessary duplication of research. Also consider using one of the following software tools:


  • Perform quality assessment, data extraction and final decisions to include and exclude studies online in real-time. Formerly known as ReGroup.
  • Discuss in the forums
  • Price: Free for one review, unlimited use is $20/month.


  • used for a review of 170K refs with no problems; most costly. Multi –reviewer packages it ranges from $329 - $1249 per month.
  • application designed for screening and data extraction phases of systematic reviews.
  • $179 US Dollar a month


  • EndNoteX5 able to run duplicate checks (slow); split library into 3 for screening and marking results and used EPPI for analysis
  • fine for 100K records as long as PDFs were not attached; library of 40,000. It was slow but managed it.

EPPI reviewer<

  • support given is v good (from a person who bought the support package)
  • recommended as it does the whole process; de-duplicating was slow
  • subscription fee payable; 90K review; flexibility to handle variety of data collecting and used by multiple reviewers accessed by signing online
  • frustrations with using EPPI; handbook not user friendly or detailed so you learn by trial and error (they have Youtube videos)
  • data extraction can take some time and only after you've used EPPI reviewer are you aware of some of the quirks of how it will present data
  • open to human error - if a particularly box or subcategory isn't ticked then the data extracted on that variable won't show up in tables. It gives no warning and its hard to spot when you've made this mistake particularly if you're having to click dozens of boxes. This is particularly problematic if you're trying to make statements like this many studies looked at this behaviour or this many studies used this type of intervention etc. as its hard to be confident the numbers you're getting are accurate.
  • claims it has functionality to compare data extraction between two reviewers extracting in duplicate I found this really unhelpful and that it was much easier to compare reviewers data extraction manually
  • poor with dealing with studies where there are multiple publications for one dataset - it provides little help


  • workshop at the Oct Cochrane Colloquium (EROS dialogues with RevMan: data extraction, quality assessment and more )
  • Web-based software designed specifcially to perform the first stages of a systematic review.
  • Discuss in the forums
  • Price: Free


SDSR – systematic review data repository (free)

  • create your own abstraction form with an instant validation check but it doesn’t analyse data.
  • Web-based tool for extraction and management of data for SRs; open and searchable archive of systematic reviews and their data
  • Price: Free.

SUMARI (System for the Unified Management, Assessment and Review of Information)

  • Appraise and synthesise evidence of feasibility, appropriateness, meaningfulness and effectiveness
  • Conduct economic evaluations of activities and interventions
  • Price: Free, but requires registering
  • SUMARI has four sub modules;
  • QARI (Qualitative Assessment and Review Instrument), which is designed to facilitate critical appraisal, data extraction and meta-aggregation of the findings of qualitative studies;
  • MAStARI (Meta Analysis of Statistics Assessment and Review Instrument), which is designed to conduct the meta-analysis of the results of comparable cohort, time series and descriptive studies using a number of statistical approaches;
  • ACTUARI (Analysis of Cost, Technology and Utilisation Assessment and Review Instrument), which is is designed to facilitate critical appraisal, data extraction and synthesis of economic data, and:
  • NOTARI Narrative, Opinion and Text Assessment and Review Instrument), which is designed to facilitate critical appraisal, data extraction and synthesis of expert opinion texts and of reports.


  • group work capabilities (though from my experience I doubt this would be able to efficiently import and de-duplicate 100,000 records)

Reference Manager and Microsoft Access in combination

  • not recommended as ‘messy using 2 systems’


See comparisons in Gough D, Oliver S, Thomas J. An introduction to systematic reviews. Sage Publications, London, 2012.


The PRISMA guidelines replace QUORUM and offer a checklist for preparing the manuscript reporting a systematic review. The PRISMA Flow Diagram Generator is also useful.
Consensus-based extension to PRISMA that offers a checklist for "condensing [a] systematic review into the essentials for an abstract."
"Provide authors and users of The Cochrane Library with clear and transparent expectations of review conduct and reporting"
Guide developed by the McGill Life Sciences Library to assist researchers in preparing manuscripts for publication and selecting venues for submission.
Personal tools