Library Skills
Systematic Reviews
What is a Systematic Review?
We conduct systematic reviews to find the answer to a pre-determined research question.
A reviewer (or review team) would:
- decide on the review question and any inclusion or exclusion criteria
- set out the review process, usually in a protocol
- conduct a rigorous search for all relevant studies (published or unpublished)
- screen the studies and decide whether to include them or not
- asses the ‘included’ studies for quality
- synthesise, analyse, interpret and report in a written review
This process is also known as a road map.
Are you a student? Check your assignment brief for a suggested particular process.
Are you an academic completing a systematic review for a particular organisation? Their handbook may give guidance on the required procedure.
Myths and Misconceptions
Systematic Reviews vs Literature Searches
Systematic reviews are NOT big literature reviews; they are much more explicitly rigorous in their approach (Petticrew 2001).
| Systematic Review | Literature Review | |
|---|---|---|
| Question | focused on a single question | not necessarily focused on a single question - may describe an overview |
| Protocol | included | not included |
| Background | provides a summary of the available literature on the topic | provides a summary of the available literature on the topic |
| Objectives | identified and stated | may not be identified |
| Inclusion and exclusion criteria | stated | not stated |
| Search strategy | described – comprehensive and systematic | not often stated/described |
| Process of selecting articles | stated | often not stated |
| Process of evaluating articles' quality | included | often not assessed/stated to be assessed |
| Extraction of relevant information | described | often not described |
| Results and data synthesis | includes summaries of studies based on high-quality evidence - possible bias reduced. | includes summary based on studies where quality may not have been assessed - possibility of bias. |
(adapted from Bettany-Saltikov 2010)
Myths
Petticrew (2001) dispels some myths about systematic reviews, including that they:
include only randomised controlled trials (RCT) and/or only quantitative evidence
This is not a requirement of a systematic review; many subjects lend themselves to a review of qualitative evidence.
always involve statistical synthesis
They can include qualitative research and different forms of analysis
can only be used in medicine and health
Systematic reviews have been conducted in a variety of subjects including advertising, astronomy, biology, chemistry, criminology, ecology, education, law, manufacturing, psychology, public policy, and zoology.
are a substitute for doing good quality individual studies
Systematic reviews do not always find definitive answers. They may conclude that more primary research is required.
Major Organisations
The work of the organisations listed below comprises or includes the production of systematic reviews and/or guidance and/or training on conducting systematic reviews.
Don't forget that systematic reviews are also published in journals by academics. The library's lists of major databases provide access to MEDLINE, CINAHL with Full Text, SocINDEX with Full Text, Web of Science and other databases which contain details of systematic reviews published in journal articles.
Question and Protocol
Scoping Search
You should conduct a ‘scoping search’ before finalising your research question, to check that:
- your topic has not already been the subject of a completed systematic review, or the subject of a forthcoming one
- there is sufficient literature on your topic
- that your proposed topic may contribute to knowledge and understanding in your field (if you’re academic staff)
You may search on relevant databases, websites of major organisations conducting systematic reviews, or PROSPERO.
Finalising your Question
Example
Imagine you are interested in the use of dolls with people with dementia residing in care homes. You want to examine if there is a reduction in distress among residents.
The PICO model will help refine your research topic and question.
| Element | Question | ||
|---|---|---|---|
| P | Population | people with dementia residing in care homes | Does the therapeutic use of dolls reduce distress among adults with dementia residing in care homes? |
| I | Intervention | the use of dolls | |
| C | Comparison | none | |
| O | Outcome | reduction in distress | |
For qualitative research without defined outcomes, you can use the PICo model (e.g. where the research is concerned with experiences or processes):
| Element | Question | ||
|---|---|---|---|
| P | Population | family members of people with dementia | What are the attitudes of family members to the use of doll therapy with people with dementia in care homes? |
| I | Intervention | Attitudes to doll therapy | |
| Co | Context | Care homes | |
Other models:
- PICOS: Population, Intervention, Comparison, Outcome, Study design
- PICOT: Population, Intervention, Comparison, Outcome, Time frame (a period over which outcomes are assessed)
- SPICE: Setting, Perspective, Intervention, Comparison, Evaluation
- SPIDER: Sample, Phenomenon of Interest, Design, Evaluation, Research type
Protocol Structure
While formats may differ, protocols include:
- The review title
- The reviewer(s)
- The objectives and research question
- Background
- Includion and exclusion criteria
- Methods
- Search Strategy
- Assessment of quality
- Data extraction and synthesis
- Any conflict of interest
Searching Systematically
Databases
Some databases may have already been suggested to you. These are our recommendations, but you should also have a look at your Subject Databases.
- Nursing: CINAHL with Full Text | Medline | Web of Science | Cochrane Library
- Nutrition: Food Science Source | CAB Abstracts | Medline | Web of Science | Cochrane Library
- Occupational Therapy: CINAHL with Full Text | Medline | Web of Science | Cochrane Library | OTseeker
- Pharmacy: CINAHL with Full Text | Medline | Web of Science | International Pharmaceutical Abstracts | Cochrane Library
- Physiotherapy: CINAHL with Full Text | Medline | Web of Science | PEDro | Cochrane Library
- Psychology: PsycARTICLES | SocINDEX | Web of Science | Medline | Cochrane Library
- Radiography: CINAHL with Full Text | Medline | Web of Science | Cochrane Library
- Social Sciences and Social Work: SocINDEX | Web of Science | CINAHL with Full Text | Cochrane Library
Search Strategy
First, extract the main keywords/phrases from the topic. Then, think of alternative words (i.e. synonyms, plurals, different spelling, related words, etc.)
Example
Does the therapeutic use of dolls reduce distress among adults with dementia residing in care homes?
| Main Keywords | Alternative Keywords |
|---|---|
| dolls | doll |
| distress | distressed, distressing anxiety, anxious, tears, tearful, tearfulness, cry, crying, fear, fearfulness, fright, frightened, apprehension, apprehensive agitation, agitated |
| dementia | Alzheimers, Alzheimer |
| care homes | nursing homes, residential care, residential homes, institutions |
Hint: For additional keywords, also have a look at the 'Subjects:' section in the article's description.
Wildcards
The symbol * , placed at the beginning or end of the root of a word, finds all forms of the word at the same time. It expands your search. So, you are sure you’re not missing useful terms.
Examples
- *stress* - stress, stressed, distress, distressing, stressful, etc.
- agit* - agitate, agitation, agitated, agitates, etc.
Phrase Searching
You can place words between quotation marks “” to find them together. It will reduce the number of results, but increases accuracy.
Examples:
- nursing home – will find the words nursing and home together or separately in the result entry
- “nursing home” – ensures the words are found next to each other
Use linking words (Booleans) to construct your search query. Connect the main elements with AND, and alternative words with OR.
Our search query is:
(doll OR dolls) AND (dementia OR Alzheimer*) AND (*stress* anxi* OR tear* OR cry OR crying OR fright* OR apprehensi*) AND (“nursing home” OR “care home” OR “residential care” OR “institution”)
Some databases will let you have keywords on several rows, so you don't have to write a very long query.
You can also browse the lists of controlled subject headings and select relevant terms. For example, the heading for Dementia will show you all relevant diseases. A heading can be expanded further to reveal even more relevant terms.
Remember to keep notes of your search process. These may include databases searched, keywords, numbers of results. You may have to produce a PRISMA flow diagram.
Grey Literature
A systematic review must include a search for both published and unpublished materials. This mitigates the risk of “publication bias”.
Grey literature is not formally or commercially published.
Handsearching
Sometimes keyword searching may miss out on some items like letters, editorial notes, comments, etc. Hence, it’s useful to also look through relevant journals manually. You may uncover unreported research!
Additional Sources
Your search can also include:
- Contacting recognised experts/academic colleagues
- Searching reference lists
- Citation searching
Screening
Ideally, screening should be conducted by several people. But, as a student, you will have to rely on your own judgement. Remember to refer to the criteria set out in the protocol to include/exclude articles.
First, look at the title and abstract. Ask:
- Does the article cover the right population?
- Does it cover the type of intervention set out in the review question/protocol?
- Is the outcome the one being examined in the review?
Next, repeat the screening process, but now apply it to the article’s full text. Then, you can make a final decision of whether the article qualifies for the systematic review.
Assessing Quality
Articles of higher quality will contain more trustworthy results and findings.
When assessing, ask the following questions:
- Was the population forming the subject of the study appropriately chosen?
- Were the research/data analysis methods appropriate?
- Was the research designed, conducted and the results analysed in a way that minimised bias?
- Are the results generalisable?
- Are there any ethical doubts about the research?
Joanna Briggs Institute - Critical Appraisal Tools
Critical Appraisal Skills Programme (CASP)
Centre for Reviews and Dissemination - Guidance for Undertaking Reviews
Extracting Data
When preparing for writing the review, you may want to extract and record data from studies, including:
- Citation details (author, title, source, year, etc.)
- Study design (as apprpriate to the review bt may include blinding)
- Number and types of participants (population) and sample size
- Interventions
- Results
Synthesise, Analyse and Report
Types of Syntheses
Narrative synthesis
A narrative synthesis provides a summary and explanation of the results in word form.
Meta-analysis
A meta-analysis is a statistical method for combining the numerical results of studies included in the review. It is commonly used in systematic reviews comprising Randomised Controlled Trials (RCTs). Its quality depends on the quality of the search and data extraction.
Meta-aggregation
It is a qualitative equivalent to meta-analysis, where findings from qualitative research studies are compiled into groups based on their similarity in meaning.
Writing up your Systematic Review
As a suggestion, your report will generally contain:
- Title and authors
- Executive summary or abstract
- Background
- Review question
- Methods (of search, selection of studies, assessment of quality, data extraction, synthesis)
- Results, discussion, conclusions, and (if any) recommendations for further research
- References
Other styles suggestions:
Methodological Expectations of Campbell Collaboration (C2) Intervention Reviews (MEC2IR) (see chapter 9)
Cochrane Handbook (see chapters 4, 9, 10, 11 and 12)
Remember to be honest about any issues of bias and any flaws in the studies included in the review.