Critically Appraising Research For Inclusion into Systematic Reviews - James Ngumo

Enhance the rigor of your systematic reviews with our detailed slides on "Critically Appraising Research for Inclusion in Systematic Reviews."
This presentation includes:

1) The importance of critical appraisal in systematic reviews
2) Key criteria and frameworks for appraising research quality
3) Tools and checklists for critical appraisal (e.g., CASP, AMSTAR, PRISMA)
4) Practical steps for assessing the validity, reliability, and applicability of studies
5) Examples and case studies illustrating the critical appraisal process
6) Common challenges and strategies to overcome them

Designed for researchers and practitioners aiming to elevate their systematic review methodology, these slides provide essential guidance for thorough and effective critical appraisal. Ensure the integrity and quality of your systematic reviews by mastering critical appraisal techniques.

1. Critically Appraising
Research For
Inclusion into
Systematic Reviews
By: James Kariuki
Kenya Medical Research Institute (KEMRI)
Nairobi, Kenya
Systematic Review & Meta -Analysis Workshop
KNUST, Kumasi, Ghana, 29 July 2024 to 1August
2024

2. Learning Objectives
At the end of the session, participants will
have gained knowledge in 3 broad areas of
systematic review processes
•Gain an understanding of the Common Research
Study Designs
•Be able to Critically Appraising Research
Publications
•Be able to Identify Study Biases in
Publications

3. Prerequisite Topics
•This session picks from the previous session in
which the participants have:
i.Identified topics of interest on which they will
base their systematic reviews
ii.Developed research questions/topics/hypothesis
(where applicable)

4. Hierarch
y of
Research
Study
Designs
Citation Source: Haley, K. E., & Huber, K. E. (2023). Design principles:
hierarchy of study types. Translational Radiation Oncology, 213 -216.
Very
Strong
Relatively
Weak

5. Appraising Research
Publications
Quality Assessment
•“The validityof a systematic review
ultimately depends on the scientific
method of the retrieved studies and the
reporting of data.”
Margaliot, Zvi, Kevin C. Chung.
“Systematic Reviews: A Primer for Plastic
Surgery Research.” PRS Journal. 120/7
(2007) p.1839

6. Why conduct critical appraisal
of each publication?
•Critical appraisal (CA) is used to systematicallyassess research papers
and to judge the reliabilityof the study being presented in each paper.
•CA also helps in assessing the worthand relevanceof the study [1].
•There are many key areas to CA including assessing suitability of the study to
answer the hypothesisedquestion and the possibility of introducing bias into
the study.

7. Data Abstraction
•PRISMA Flow chart
•See Excel File

8. Types of Study Bias in
Publications

9. Types of Study Biases
(Literature Search)
Database Bias-“No single database is likely to contain all
published studies on a given subject.”
Publication Bias -selective
publication of articles that
show positive treatment of
effects and statistical
significance.
Hence, it is important to search
for unpublished studies through a
manual search of conference
proceedings, correspondence with
experts, and a search of clinical
trials registries.
English-language bias-occurs when reviewers exclude papers
published in languages other than English
Citation bias-occurs when studies with significant or
positive results are referenced in other publications,
compared with studies with inconclusive or negative findings

10. Appraising Bias in Each
Publication
•Selectionbias
–Identificationof studies
•Searchstrategy
–Greyliterature
–Selectionof studies
•Eligibilitycriteria/Consesus
–Abstractionof data
•Reproducibility/Reliability
–Publicationbias
•Reportingbias
–Interpretation/conclusions
“…Not all systematic
reviews are
systematic reviews…”
Moher D. et al PLoSMed,
2007. 4(3).Methods for detecting
publication bias:
Graphical: funnel plot
asymmetry
Tests: Egger test,
Rosenthal’s Fail-safe N

11. Funnel
plot to
detect
publicat
ion bias
Egger et al. Systematic reviews in health care.
London: BMJ books, 2001.

12. Publication bias caveat
Funnel plot asymmetry does not always
indicate bias:-
It is possible that smaller studies
enrolled higher risk patients, for
example, and therefore found a
greater effect.
Small studies are often conducted
before larger studies. In the
intervening years, other
interventions may have improved,
thus reducing the relative efficacy
of the treatment.

13. The End…
Thank You so much for your
attention!
Asanteni Sana!