Meta analysis

META ANALYSIS, OVERVIEW WITH IMPORTANCE IN PSYCHIATRY

1. META ANALYSIS DR SETHU P S JUNIOR RESIDENT DEPT. OF PSYCHIATRY

2. INTRODUCTION Meta-analysis is a statistical technique, or set of statistical techniques, for summarizing the results of several studies into a single estimate. Meta-analysis takes data from several different studies and produces a single estimate of the effect, usually of a treatment or risk factor. It aims to utilize the increased power of pooled data to clarify the state of knowledge on that issue

3. HISTORY The first meta-analysis  Karl Pearson in 1904 The first metaanalysis assessing the effect of a therapeutic intervention was published in 1955. The term meta-analysis was coined in 1976 by psychologist Glass (1976). The Greek root ‘meta’ means ‘with’, ‘along’, ‘after’, or ‘later’ The relevance of meta-analysis to psychiatry stems from one of the earliest meta-analysis ever undertaken, that evaluated the efficacy of various forms of psychotherapy (Smith & Glass, 1977).

4. The term meta-analysis is slowly being replaced by the term ‘overviews’ or, more recently, ‘systematic reviews’. The Potsdam International Consultation on Meta-analysis in 1994 provided the following definitions of terminology Systematic Review (or Overview): the application of scientific strategies that limit bias to the systematic assembly, critical appraisal, and synthesis of all relevant studies on a specific topic. Meta-analysis (or Quantitative Overview): a systematic review that employs statistical methods to combine and summarise the results of several studies

5. USES Assimilation of large quantities of information into more precise studies. Help decision makers like clinician and health policy makers to integrate the critical pieces of available bio-medical information Generalisability of research findings can be established.

6. Efficient scientific technique Quicker and less costly than embarking on a new study. b) Prevents unnecessary duplication of already sound studies. c) Shortens the time between medical research discoveries and their clinical implementation

7. Assess the consistency of relationships Consistency of effects i.e. Whether they are in same direction and of same magnitude given the variance in study protocols. They determine consistency among studies of same intervention or different Intervention. Explain data inconsistencies or conflicts Increased statistical power Increased precision

8. TYPES Traditional narrative reviews  provide a qualitative reviews but not quantitative assessment of published results. From literature which are generally performed from freely available publications without the need of co-operation and without agreement of authors of original studies.

9. Meta-analysis with individual patient data, in which individual data from published and sometimes also unpublished studies are re-analyzed . Often there is a close co-operation between the researcher of met analysis and investigators of individual studies. Prospectively planned meta-analysis of several studies in which pooling is already a part of protocol.

10. Cumulative meta-analysis The repeated performance of meta-analysis whenever a new trial becomes available for inclusion. Such cumulative meta-analysis can retrospectively identify the point in time when a treatment effect first reached conventional levels of significance. Eg : streptokinase thrombolytic therapy in MI (after 8 studies)

11. STEPS

12. Test for Heterogeneity Can be assessed clinically or statistically Graphical representation or statistical data

13. A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. (p<0.1) Meta-regression is a way of exploring the reasons for the heterogeneity of results and adjustment for confounding effects.

14. STATISTICAL PROCEDURES Fixed effect model (FEM) considers the variability of the results as ‘random variation’ and individual studies are simply weighted by their precision (inverse of the variance). In other words, this looks only at within-study variation. Random effect model (REM) assumes a different underlying effect for each study and takes this as an additional source of variation, being randomly distributed. In other words, this takes into account between-study variations, as well as within-study variation.

15. Each study is summarised using a measure of effect (such as an odds ratio or a relative risk for dichotomous data, using the Mantel - Haenszel technique or the Peto modification or a weighted mean difference for continuous data). The pooled, weighted measures are combined to produce the summary statistic, usually with confidence intervals that are similar to confidence intervals in single studies. This ensures that participants in each study are only compared to people in the same study, thus preserving the power of randomisation .

16. BIAS Sampling bias Publication bias Retrieval bias Multiple publication bias Selection Bias Inclusion criteria bias Bias in assessing quality of trials Data extraction Bias Bias in analyzing results

17. Sampling bias Publication bias This refers to the tendency for studies that report statistically significant results to be published. Due to authors of negative trials not submitting manuscripts for publication Editorial and referees’ policy in not publishing studies with results that challenge Conflicts of interest caused by drug companies discouraging publication of sponsored trials with negative or adverse results Non-English-language references are under-represented in electronic databases such as MEDLINE and only published articles are included “FAIL-SAFE NUMBER” AND “FUNNEL PLOT”

18. Retrieval bias Electronic databases such as MEDLINE (the electronic form of Index Medicus ) and EMBASE (the electronic form of excerpta medica ) are powerful tools for locating studies. However, only 30 - 80% of all known published randomised controlled trials are identifiable using MEDLINE, depending on the area or specific question In the field of mental health, MEDLINE searches failed to identify 30-50% of RCTs while hand-searching journals identified 95% of the trials

19. Multiple publication bias trials being published in different journals as a series of articles, thus leading to the possibility of them being recorded as separate trials This is more likely if the articles are published with different authors Huston & Moher (1996) recorded seven different publications with different authorship of a single trial of Risperidone in chronic schizophrenia.

20. Selection Bias Inclusion criteria bias: The use of inclusion criteria can introduce bias if the reviewer consciously or otherwise sets criteria in a manner that excludes trials he/she is aware of. select a comprehensive and clearly formulated protocol a priori Incorporating more than one investigator Bias in assessing quality of trial To assess quality of an RCT, check for Selection bias, performance bias, attrition bias and detection bias.

21. Data extraction bias Inter-observer reliability can be maximised and extraction bias minimised by the use of specific and comprehensive pre-tested data-extraction sheets with clear instructions on how to interpret and handle data Contributing to extraction bias may be ‘reporting bias’ where inadequate reporting of data for various outcomes, especially unfavorable outcomes, leads to biased results.

22. Bias in analyzing results Inappropriate use of statistics inadequate data, or if it does not make sense to combine disparate results or outcome measures, or if there is significant heterogeneity between studies that cannot be explained on methodological groundsDealing with heterogeneity Dealing with heterogeneity Clinical and statistical heterogeneity

23. Sensitivity analysis evaluations aimed at assessing how sensitive the results of the analysis are to the way the review was conducted commonly used decisions in sensitivity analyses Changing inclusion criteria (excluding studies without operationally defined diagnostic criteria, or excluding very old trials) including or excluding studies of lower quality or with ambiguous inclusion criteria excluding unpublished studies reanalysing the data using different approaches to handling data.

24. HOW TO EVALUATE A SYSTEMATIC REVIEW 1. Are the results of the review valid? • Does the review address a focused clinical question? • Were inclusion criteria used to select articles appropriate? • Was the search for relevant studies thorough? Is there a likelihood of publication bias? • Was the validity of included studies adequately assessed ? • Were benefits of the intervention as well as harmful effects assessed? • Was the review based entirely on the results of small sample RCTs ?

25. 2. Was the data handled appropriately? Was data extraction from trials free from bias? Was there an attempt to obtain missing information from authors? What is the magnitude of the effect? How precise are the overall results of the review? Is it logical to combine results to produce a summary statistic? Is there an absolute measure? Is there significant heterogeneity? If so, can it be explained on methodological or clinical grounds? How sensitive are the results to changes in the way the analysis was done?

26. 3. Are the conclusions generalizable? • Are participants & Interventions of included studies relevant to your practice? • Are the conclusions Justified by the evidence? • Are subgroup analysis interpreted cautiously? • Is the clinical significance of the results sufficient to warrant a change of our practice?

27. THE COCHRANE COLLABORATION The Cochrane Collaboration is an international network of individuals and institutions formed in 1993, and committed to preparing, maintaining and disseminating systematic reviews of the effects of health care. Cochrane reviews are highly structured systematic reviews prepared by a group of collaborating authors, called a Cochrane review group, using explicitly refined methods to reduce bias.

28. These reviews are published in the Cochrane Library, an electronic journal on CD and computer discs (and now showing on the Internet) released quarterly, which contains the Cochrane Database of Systematic Reviews (CDSR). These are up-to-date reviews that are periodically updated in response to comments and criticisms and when new data becomes available.

29. Meta-analyses can provide a more precise estimate of an effect size if properly designed and performed. Their results can be used in clinical decision making not only to estimate the average effect of an intervention, but also to investigate sources of variation and differing effects among subgroups.

30. References The relevance of meta-analysis, systematic reviews And the cochrane collaboration to clinical psychiatry; Dr. Prathap Tharyan ; Indian J Psychiatry. 1998 Apr-Jun; 40(2): 135–148 Basic concepts in meta-analysis: A primer for clinicians. Khoshdel A, Attia J, Carney SL. Int J Clin Pract . 2006 Oct;60(10):1287-94. Review. PubMed PMID: 16981972.

31. THANK YOU