Systematic Reviews

Library Skills

Systematic Reviews

What is a Systematic Review?

We conduct systematic reviews to find the answer to a pre-determined research question.

A reviewer (or review team) would:

  • decide on the review question and any inclusion or exclusion criteria
  • set out the review process, usually in a protocol
  • conduct a rigorous search for all relevant studies (published or unpublished)
  • screen the studies and decide whether to include them or not
  • asses the ‘included’ studies for quality
  • synthesise, analyse, interpret and report in a written review

This process is also known as a road map.

Are you a student? Check your assignment brief for a suggested particular process.

Are you an academic completing a systematic review for a particular organisation? Their handbook may give guidance on the required procedure.

Myths and Misconceptions

Systematic Reviews vs Literature Searches

Systematic reviews are NOT big literature reviews; they are much more explicitly rigorous in their approach (Petticrew 2001).

Systematic Review Literature Review
Question focused on a single question not necessarily focused on a single question - may describe an overview
Protocol included not included
Background provides a summary of the available literature on the topic provides a summary of the available literature on the topic
Objectives identified and stated may not be identified
Inclusion and exclusion criteria stated not stated
Search strategy described – comprehensive and systematic not often stated/described
Process of selecting articles stated often not stated
Process of evaluating articles' quality included often not assessed/stated to be assessed
Extraction of relevant information described often not described
Results and data synthesis includes summaries of studies based on high-quality evidence - possible bias reduced. includes summary based on studies where quality may not have been assessed - possibility of bias.

(adapted from Bettany-Saltikov 2010)


Petticrew (2001) dispels some myths about systematic reviews, including that they:

include only randomised controlled trials (RCT) and/or only quantitative evidence

This is not a requirement of a systematic review; many subjects lend themselves to a review of qualitative evidence.

always involve statistical synthesis

They can include qualitative research and different forms of analysis

can only be used in medicine and health

Systematic reviews have been conducted in a variety of subjects including advertising, astronomy, biology, chemistry, criminology, ecology, education, law, manufacturing, psychology, public policy, and zoology.

are a substitute for doing good quality individual studies

Systematic reviews do not always find definitive answers. They may conclude that more primary research is required.

Major Organisations

The work of the organisations listed below comprises or includes the production of systematic reviews and/or guidance and/or training on conducting systematic reviews.

Don't forget that systematic reviews are also published in journals by academics. The library's lists of major databases provide access to MEDLINE, CINAHL with Full Text, SocINDEX with Full Text, Web of Science and other databases which contain details of systematic reviews published in journal articles.

Question and Protocol

Scoping Search

You should conduct a ‘scoping search’ before finalising your research question, to check that:

  • your topic has not already been the subject of a completed systematic review, or the subject of a forthcoming one
  • there is sufficient literature on your topic
  • that your proposed topic may contribute to knowledge and understanding in your field (if you’re academic staff)

You may search on relevant databases, websites of major organisations conducting systematic reviews, or PROSPERO.

Finalising your Question


Imagine you are interested in the use of dolls with people with dementia residing in care homes. You want to examine if there is a reduction in distress among residents.

The PICO model will help refine your research topic and question.

Element Question
P Population people with dementia residing in care homes Does the therapeutic use of dolls reduce distress among adults with dementia residing in care homes?
I Intervention the use of dolls
C Comparison none
O Outcome reduction in distress

For qualitative research without defined outcomes, you can use the PICo model (e.g. where the research is concerned with experiences or processes):

Element Question
P Population family members of people with dementia What are the attitudes of family members to the use of doll therapy with people with dementia in care homes?
I Intervention Attitudes to doll therapy
Co Context Care homes

Other models:

  • PICOS: Population, Intervention, Comparison, Outcome, Study design
  • PICOT: Population, Intervention, Comparison, Outcome, Time frame (a period over which outcomes are assessed)
  • SPICE: Setting, Perspective, Intervention, Comparison, Evaluation
  • SPIDER: Sample, Phenomenon of Interest, Design, Evaluation, Research type

Protocol Structure

While formats may differ, protocols include:

  • The review title
  • The reviewer(s)
  • The objectives and research question
  • Background
  • Includion and exclusion criteria
  • Methods
  • Search Strategy
  • Assessment of quality
  • Data extraction and synthesis
  • Any conflict of interest

First, extract the main keywords/phrases from the topic. Then, think of alternative words (i.e. synonyms, plurals, different spelling, related words, etc.)


Does the therapeutic use of dolls reduce distress among adults with dementia residing in care homes?

Main Keywords Alternative Keywords
dolls doll
distress distressed, distressing anxiety, anxious, tears, tearful, tearfulness, cry, crying, fear, fearfulness, fright, frightened, apprehension, apprehensive agitation, agitated
dementia Alzheimers, Alzheimer
care homes nursing homes, residential care, residential homes, institutions

Hint: For additional keywords, also have a look at the 'Subjects:' section in the article's description.


The symbol * , placed at the beginning or end of the root of a word, finds all forms of the word at the same time. It expands your search. So, you are sure you’re not missing useful terms.


  • *stress* - stress, stressed, distress, distressing, stressful, etc.
  • agit* - agitate, agitation, agitated, agitates, etc.

Phrase Searching

You can place words between quotation marks “” to find them together. It will reduce the number of results, but increases accuracy.


  • nursing home – will find the words nursing and home together or separately in the result entry
  • “nursing home” – ensures the words are found next to each other

Use linking words (Booleans) to construct your search query. Connect the main elements with AND, and alternative words with OR.

Our search query is:

(doll OR dolls) AND (dementia OR Alzheimer*) AND (*stress* anxi* OR tear* OR cry OR crying OR fright* OR apprehensi*) AND (“nursing home” OR “care home” OR “residential care” OR “institution”)

Some databases will let you have keywords on several rows, so you don't have to write a very long query.

You can also browse the lists of controlled subject headings and select relevant terms. For example, the heading for Dementia will show you all relevant diseases. A heading can be expanded further to reveal even more relevant terms.

Remember to keep notes of your search process. These may include databases searched, keywords, numbers of results. You may have to produce a PRISMA flow diagram.

Grey Literature

A systematic review must include a search for both published and unpublished materials. This mitigates the risk of “publication bias”.

Grey literature is not formally or commercially published.


Sometimes keyword searching may miss out on some items like letters, editorial notes, comments, etc. Hence, it’s useful to also look through relevant journals manually. You may uncover unreported research!

Additional Sources

Your search can also include:

  • Contacting recognised experts/academic colleagues
  • Searching reference lists
  • Citation searching


Ideally, screening should be conducted by several people. But, as a student, you will have to rely on your own judgement. Remember to refer to the criteria set out in the protocol to include/exclude articles.

First, look at the title and abstract. Ask:

  • Does the article cover the right population?
  • Does it cover the type of intervention set out in the review question/protocol?
  • Is the outcome the one being examined in the review?

Next, repeat the screening process, but now apply it to the article’s full text. Then, you can make a final decision of whether the article qualifies for the systematic review.

Assessing Quality

Articles of higher quality will contain more trustworthy results and findings.

When assessing, ask the following questions:

  • Was the population forming the subject of the study appropriately chosen?
  • Were the research/data analysis methods appropriate?
  • Was the research designed, conducted and the results analysed in a way that minimised bias?
  • Are the results generalisable?
  • Are there any ethical doubts about the research?

Extracting Data

When preparing for writing the review, you may want to extract and record data from studies, including:

  • Citation details (author, title, source, year, etc.)
  • Study design (as apprpriate to the review bt may include blinding)
  • Number and types of participants (population) and sample size
  • Interventions
  • Results

Synthesise, Analyse and Report

Types of Syntheses

Narrative synthesis

A narrative synthesis provides a summary and explanation of the results in word form.


A meta-analysis is a statistical method for combining the numerical results of studies included in the review. It is commonly used in systematic reviews comprising Randomised Controlled Trials (RCTs). Its quality depends on the quality of the search and data extraction.


It is a qualitative equivalent to meta-analysis, where findings from qualitative research studies are compiled into groups based on their similarity in meaning.

Writing up your Systematic Review

As a suggestion, your report will generally contain:

  • Title and authors
  • Executive summary or abstract
  • Background
  • Review question
  • Methods (of search, selection of studies, assessment of quality, data extraction, synthesis)
  • Results, discussion, conclusions, and (if any) recommendations for further research
  • References

Other styles suggestions:

Remember to be honest about any issues of bias and any flaws in the studies included in the review.