Library Skills
Critical Appraisals
Critical appraisal can be defined as "the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context" (Burls 2009 p. 1).
Why Critically Appraise Literature?
Critical appraisal skills are often discussed in the context of evidence-based practice in the fields of nursing, midwifery, health sciences, pharmacy and medicine. This involves finding and critically appraising relevant research for its validity, relevance and reliability before it is combined with the expertise of the health practitioner (and the patients preferences) in treating patients.
This systematic approach can be applied in assessing the quality of research studies in other fields.
It is often assumed that if an article is published in a journal it must be of quality; this is not always the case.
It has been argued that most, "if not in fact 99%, of published articles belong in the bin, and should certainly not be used to inform practice" (Greenhalgh 2014 p. 28). They may use "inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation" (Altman 1994 p. 283).
When reading journal articles and studies it is important to ask yourself if the research is methodologically sound, relevant, is of value to the subject area and patients, has minimized the effects of potential biases and is reported in a clear way.
How Do I Start?
Consider some fundamental questions when you begin your appraisal (Greenhalgh 2014):
What was the research question and why was the study needed?
- The question / hypothesis should be clearly stated in the study.
- Does the study add something to knowledge in the area?
- Consider the PICO method (MacInnes and Lamont 2014):
- PROBLEM: Is there a focused question?
- INTERVENTION: Is there a clear intervention being examined?
- COMPARISON: Is there a suitable control or alternative?
- OUTCOMES: Have results and outcomes been properly gathered and assessed?
- PROBLEM: Is there a focused question?
- INTERVENTION: Is there a clear intervention being examined?
- COMPARISON: Is there a suitable control or alternative?
- OUTCOMES: Have results and outcomes been properly gathered and assessed?
What was the research design?
- Quantitative or qualitative research? Combination?
- Primary or secondary research?
- What's the methodology?
- This is crucial to your in-depth appraisal (will determine the appropriate checklist to apply)
- The design used should be made clear. Unsure? Check out University of Minnesota's Understanding Research Study Designs guide, for a reminder of some of the types of health research.
Was the design appropriate to the question?
- Different questions lend themselves to different sorts of research design.
Did the study meet expected standards of ethics and governance?
- Were ethical issues given proper consideration?
You may also want to use the structure of the article as a framework for your appraisal, which will logically lead you to ask the right questions (Ingham-Broomfield 2014):
- Has the research question been clearly defined?
- Does the abstract make clear the methods used, results and conclusions?
- Is there a clear description of the problem being studied?
- Has current and relevant literature been critically reviewed - rather than just described?
- Is the choice of research method justified?
- Are there issues of bias?
- is the author sponsored by the company selling any drug being tested?
- does any sample population really reflect the population for which a treatment being tested is intended?
- has any randomisation (between any test and control groups) and/or blinding been properly done?
- more on bias: Burls (2009), Understanding Health Research, Bandolier bias guide and Pannucci and Wilkins (2010)
- Was any sample population appropriately selected to adequately represent the population being targeted by any particular treatment or drug?
- Were any issues of consent, confidentiality etc. properly addressed?
- Are the results set out clearly?
- Have they been analysed properly?
- Was the choice of particular statistical methods justified?
- Are any conclusions set out clearly?
- Do the results support the conclusion?
Key Tools
If you are an undergraduate student you will probably find these resources sufficient to allow you to successfully complete your critical appraisal assignment.
Postgraduate research students and staff working at a higher level may also wish to consult our additional resources page for further materials.
There is considerable overlap in the checklists for the same types of research from different organisations. The CASP checklists are possibly the most widely used but you may find you prefer the style of one organisation's lists to another. If in doubt you may wish to consult your lecturer.
CRITICAL APPRAISAL SKILLS PROGRAMME (CASP)
SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK (SIGN)
HOW TO READ A PAPER: MEDICAL JOURNAL
HOW TO READ A PAPER: THE BASICS OF EVIDENCE-BASED MEDICINE
CENTRE FOR EVIDENCE-BASED MEDICINE (CEBM): CRITICAL APPRAISAL TOOLS
BESTBETS: CRITICAL APPRAISAL WORKSHEETS
JOANNA BRIGGS INSTITUTE CRITICAL APPRAISAL TOOL
Additional Resouces
The following may facilitate the appraisal of more complicated studies and assist postgraduate research students and members of staff working in greater depth and/or seeking to critically appraise studies for possible inclusion or exclusion from systematic reviews.
INTERNATIONAL CENTRE FOR ALLIED HEALTH EVIDENCE: CRITICAL APPRAISAL TOOLS
DUKE UNIVERSITY MEDICAL CENTRE UNIVERSITY LIBRARY & ARCHIVES: EVIDENCE-BASED PRACTICE
PREFERRED REPORTING ITEMS FOR SYSTEMATIC REVIEWS AND META-ANALYSIS (PRISMA)