Overview of systematic review and meta analysis

RESEARCH- SYSTEMATIC REVIEW, META ANALYSIS, FOREST PLOT, CHECKLISTS FOR VARIOUS STUDY DESIGNS.

1. OVERVIEW OF SYSTEMATIC REVIEW AND META- ANALYSIS DR SNEHA POST GRADUATE DEPARTMENT OF COMMUNITY MEDICINE

2. SYSTEMATIC REVIEW “ Systematic review is a high level overview of a particular research question that systematically identifies, selects, evaluates and synthesizes all high quality research evidence relevant to that question in order to answer it”

3. “ A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise and synthesize on a specific issue” It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors.

4. Often written by panel of experts after reviewing all the information from both published and unpublished literature ( grey literature). Useful for policy makers for decision making on any intervention/ treatments, etc.

5. Forms the basis of decision making in evidence based medicine and evidence based behavioral practice. Hierarchy of evidence: top position Most reliable evidence with less bias. Considered gold standard.

6. PYRAMID OF EVIDENCE IN RESEARCH

7. CHALLENGES FACED DURING SYSTEMATIC REVIEW Too many studies to search Publication bias: only studies with significant findings are published. Inconsistencies in findings because of flaws in study conduct or random errors.

8. STEPS IN CONDUCTING SYSTEMATIC REVIEW Formulate review question Identification of relevant studies Extraction of data Assessment of bias in included studies Synthesis of data Interpreting the evidence Writing up the review Updating the review.

9. 1. FORMULATE REVIEW QUESTION Address the variety of issues : incidence/ prevalence/ etiology of diseases / diagnostic accuracy/ effectiveness of interventions. Should be precise. Should specify key components: PICO

10. P= Participants I = Interventions C = Control / comparison O = Outcome of interest.

11. Eg: “ in patients with TB does daily regimen or alternative regimen have effective success rate “ PARTCIPANT INTERVENTION CONTROL OUTCOME OF INTEREST

12. 2. IDENTIFY RELEVANT STUDIES Time consuming Eligibility criteria should be set for inclusion and exclusion. Criteria is set up with relevance to PICO components

13. CRITERIA WITH REGARD TO EACH COMPONENTS P= PARTCIPANTS I=INTERVENTIONS C=CONTROL O = OUTCOME Sociodemographic characteristics and study setting What intervention? how delivered? Who delivers? Intensity of intervention? What kind of comparison? Active control : different regimen of same drugs/ diff therapy Inactive control: placebo/ standard protocol/ no Rx Should be clearly stated. Type of outcomes: short term or long term Primary or secondary outcome One benefit and one ADR should also be assessed.

14. Define the study design also based on the research question. Prevalence of diseases/ diagnostic accuracy of tests: cross sectional design Aetiology of disease : cohort design Effects of intervention: RCT

15. SEARCHING FOR STUDIES SOURCES: MEDLINE, EMBASE, CENTRAL, LILAC, etc Can use MeSH terms to search articles. Search for unpublished literature and ongoing studies Unpublished literature/ grey literature: conference abstracts, dissertations, books, etc. Studies in non English journals and small sample size studies to be selected too.

16. PRISMA GUIDELINES FOR STUDY SELECTION

17. 3. DATA COLLECTION AND EXTRACTION WHAT DATA TO BE COLLECTED? Data regarding; Eligibility of study Study methodology Details of participants No of intervention groups and details specific to interventions given

18. Information regarding outcomes: definitions, how measured, all pre specified and unspecified outcomes are to be collected and analysed. Information on ethical approval, funding, conflicts of interest, name and contact of authors.

19. DATA EXTRACTION Process of data recording into data collection forms. Two reviewers should work independently: to reduce the risk of errors. Blinding of data extractors: to reduce the risk of bias. However routine blinding is not usually recommended. Check for study duplication: same studies reported in more than one journal.

20. 4. ASSESSING THE PRESENCE AND RISK OF BIAS Assess for bias in study design, conduct, analysis or reporting of study. Several methods available. For clinical trials: DOMAIN BASED EVALUATION recommended by Cochrane library.

21. TYPE OF BIAS DESCRIPTION DOMAIN SELECTION BIAS Differences in the baseline characteristics of the participants in the groups compared Allocation concealment PERFORMANCE BIAS Differences in the care given to groups Blinding ATTRITION BIAS Differences in the withdrawals Blinding DETECTION BIAS Differences in how outcome is measured Blinding REPORTING BIAS Publication bias Selective outcome reporting

22. 5. SYNTHESIS OF DATA: Meta analysis 6. INTERPRETING THE EVIDENCE 7. WRITING THE REVIEW: PRISMA guidelines 8. UPDATING THE REVIEW

23. ADVANTAGES OF SYSTEMATIC REVIEW Uses explicit methods which limits bias Draws reliable and accurate conclusions Best form of evidence Very useful decision making tools for clinicians, researchers and for policy makers. Generation of new hypothesis about subgroups of study Increases the precision of the results.

24. LIMITATIONS OF SYSTEMATIC REVIEW Location and selection of studies Heterogeneity Loss of information on important outcomes Inappropriate subgroup analyses

25. META-ANALYSIS (MA) ACCORDING TO GENE GLASS WHO FIRST DEFINED META ANALYSIS IN 1976, “ meta analysis refers to a statistical analysis of a large collection of analysis results from individual studies , for the purpose of integrating the findings .”

26. TYPES OF META ANALYSIS CUMULATIVE MA: new studies are added and MA repeated every time an new study is published. RETROSPECTIVE MA: commonly done. PROSPECTIVE MA: criteria and protocol for selection is stated even before studies of interest are published. ( low bias)

27. EFFECT SIZE IN META ANALYSIS EFFECT SIZE: measure of analysis Dependent variable Any standard index Eg: prevalence, incidence, odds ratio, relative risk, effects of intervention.

28. TYPE OF ANALYSIS DONE IN MA

29. SUBGROUP ANALYSIS If a meta analysis is performed across heterogenous trials, it may be inappropriate to draw conclusions from the pooled treatment effect . If the same trials are subgrouped and there is no heterogeneity within trials then valid conclusions can be drawn using results from subgroup analysis.

30. If subgroup analysis demonstrate that the treatment is more or less effective for certain subgroups of patients, interpretation of these subgroup analyses can provide valuable insight into how the treatment should be used in clinical practice . Participants are divided into subgroups based on certain characteristics ( gender, ses ) or trial characteristics ( geographic location) and then analysed

31. SENSITIVITY ANALYSIS done to see if the estimate changes by changing some parameters. To see how far the result is affected by changes. Eg: estimates are checked before and after including low quality studies

32. PRESENTATION OF THE RESULT OF MA: FOREST PLOT Graphical representation of the results Always included in presenting the results. Displays the effect size estimates and confidence intervals for each study included in MA .

33. Studies to be ordered either according to ; effect size estimate/ magnitude Study weightage ( precision) Chronological order Any other meaningful order

34. HOW TO INTERPRET THE RESULT OF META ANALYIS??

35. WERE APPLES COMBINED WITH ORANGES??

36. LOOK FOR HETEROGENIETY Refers to difference between studies not due to chance . Types of heterogeneity : clinical ( pt characteristics, interventions, outcomes) and statistical ( diff in study design and quality). Clinical heterogeneity always exists and can be identified without any calculation or tests. Statistical heterogeneity doesn’t exist always and needs tests to identify them.

37. HOW TO DETECT HETEROGENIETY? REVIEW TABLES AND CHECK FOR THE TYPE OF PTS: for clinical heterogeneity . ( mixing of pts with different diseases and treatment pattern) EYEBALL TEST: look at the forest plot for overlapping of confidence interval. (Overlap + = no heterogeneity). STATISTICAL TEST: tells the extent of heterogeneity and its significance.

38. OVERLAPING OF CONFIDENCE INTERVAL+. NO HETEROGENIETY NO OVERLAPPING OF CONFIDENCE INTERVAL. HETEROGENIETY +

39. STATISTICAL TEST OF HETEROGENIETY 1 . x2 test : commonly used If P >0.05 heterogeneity + Not useful 2. I2 test : to quantify heterogeneity I2 = % of variation across studies that is due to heterogeneity and not due to chance. 25%= low heterogeneity ( 25% of variation is not due to chance) 50%= moderate 75% = high

40. NO HETEROGENIETY HETEROGENIETY +

41. WHY IS IT IMPORTANT TO KNOW ABOUT HETEROGENIETY? “ LARGE HETEROGENIETY AMONG STUDIES MAY MAKE ANY POOLED ESTIMATE MEANINGLESS”

42. FIXED AND RANDOM EFFECT MODEL FIXED EFFECT: differences among studies are purely due to chance . RANDOM EFFECT: differences among studies due to chance and other reasons also. WHICH MODEL TO BE USED? when heterogeneity is absent: use fixed effect When heterogeneity is present: use random model Some researchers suggest to use both the models irrespective of heterogeneity.

43. NO HETEROGENIETY HETEROGENIETY + USED FIXED MODEL USED RANDOM EFFECT MODEL

44. WAYS TO DEAL WITH HETEROGENIETY Do not perform MA Do subgroup and sensitivity analysis: to find the reason for heterogeneity. Do MA based on random model: use only when the reason for heterogeneity cannot be explained. Change the effect measure: may sometimes introduce artificial heterogeneity

45. Publication bias analysis : funnel plot Happens when studies with positive and significant results are only selected. Tool to visually assess the possibility of publication or small study bias in meta analysis. Scatter plot of effect size over standard error of effect size. X axis: effect size Y axis: SE of effect size. Not recommended in a very small study MA ( n<10) Studies with smaller sample size are scattered at the bottom of the plot. Large and most powerful studies at the top.

46. Asymmetry in a funnel plot Publication bias Poor methodology By chance or random error True heterogeneity To differentiate between publication bias and other reasons of plot asymmetry CONTOUR ENHANCED FUNNEL PLOT

47. No studies at the bottom of the plot. No smaller studies included . Possibility of publication bias .

48. CONTOUR ENHANCED FUNNEL PLOT: Funnel plot with additional contour lines of statistical significance Lines at p=0.01,0.00,0.05,etc.. Interpretation: if studies missing in area of non significance: PUBLICATION BIAS If studies missing in areas of significance: OTHER REASONS No studies in areas of significance : PUBLICATION BIAS

49. Studies missing in the area of non significance. PUBLICATION BIAS .

50. studies missing in the area of significance . OTHER REASON FOR BIAS

51. QUALITY ASSESSMENT OF SYSTEMATIC REVIEW AND META ANALYSIS 3 commones t ways ; Overview quality assessment questionnaire PRISMA checklist The AMSTAR tool

52. CHECKLIST FOR STUDIES S.NO STUDY DESIGN GUIDELINES 1. CASE REPORT CARE 2. OBSERVATIONAL STUDY STROBE 3. ANALYTICAL STUDY STROBE 4. RCT CONSORT 5. NON RANDOMIZED CONTROLLED TRIAL TREND 6. SYSTEMATIC REVIEW AND META ANALYSIS PRISMA 7. DIAGNOSTIC TEST ACCURACY QUADAS, STARD 8. INTERVENTIONS FOR QUALITY AND SAFETY OF CARE SQURE 9. ECONOMIC EVALUATION OF HEALTH INTERVENTIONS CHEERS 10. QUALITATIVE STUDY COREQ