Evidence Based Surgery Kiki Lukman Medical Faculty of Universitas Padjadjaran , Dr Hasan Sadikin Hospital Bandung PRA BEDAH DASAR FKUP 1
What is it? Evidence-based medicine is the integration of best research evidence with individual clinical experience and patient values and expectations
Component of EBM Clinical Expertise Research Evidence Patient Preferences
Rules of Evidence All evidence is not created equal Evidence alone never makes clinical decisions Values always influence decisions
Why Evidence Based Medicine ?
Backgrounds Our daily need for valid information about diagnosis , prognosis , therapy and prevention The inadequacy of traditional sources for this information: Textbooks ( out of date ), experts ( frequently wrong ), didactic CME ( ineffective ), & Journals ( overwhelmed =volume, variable =in validity)
Backgrounds The disparity between our diagnostic skills & clinical judgment (increase) and our up-to-date knowledge & clinical performance (decline) Our inability to afford more than a few seconds /patient for finding & assimilating evidence, or to set aside > ½ hour/week for general reading & study
Epistomology of EBM
Recent developments The development of strategies for efficiently tracking down & appraising evidence The creation of systemic reviews & concise summaries of the effects of health care The creation of evidence based journals of secondary publications.
Recent Developments The creation of information systems (in seconds) The identification & application of effective strategies for lifelong learning & improving our clinical performance
Good Electronics Information Sites
EBM Method – 5A
Case A 55 year old male admitted to the ICU referred from private hospital as a case of bilateral pneumonia, respiratory failure complicated by development of septic shock in the last 6 hours prior to his transfer to ICU. Past medical history : DM/HTN/ESRD S/P renal transplant in 1998
Temp ; 38C, BP 90/50 mmHg, RR 24, pulse 90, SpO2 78% at room air lab ; Glucose 220mg/dl, Urea 25, Creatinin 180 mmol/l, Na 138, K 4.2, HCO3 18, Bilirubin 2mg%. Fluid loading, inotropic support, ventilatory support and appropriate AB are initiated . ICU consultant states that he prefers to start the patient on an intensive insulin therapy, You would like to see the evidence
Let’s apply the 5 A’s of Evidence Based Medicine A sk an answerable question Find an A rticle A ppraise the evidence A pply the evidence and A ssess
Asking An Answerable Question :
You formulate the question, “in patients with severe septic shock secondary to pneumonia with multiple organ dysfunction/ failure and hyperglycemia, is intensive insulin regimen reduce the risk of death compared to the conventional therapy? Formulating The Question
Ask Clinical Questions Patient/ Population Outcome Intervention/ Exposure Comparison In a patient with severe septic shock secondary to pneumonia, MODS/MOF post renal transplant Compared Conventional therapy Intensive insulin therapy Reduced the risk of death ?
Definitions Absolute risk reduction difference in two event rates CER - EER = ARR Relative risk reduction proportion of control rate CER-EER / CER = RRR 3.4/8= 42.5% 8. - 4.6 = 3.4 % 95% confidence interval for RR = 22 – 62
Definitions Number needed to treat number of extra patients you need to treat to prevent one bad outcome 1 / ARR = NNT 1/3.4/100= 29.4 The number of patients who need to be treated to save one life is 29 (95% confidence interval 22 - 62
Why do we need to know types of study design ? To answer certain research question Specific study design Specific statistics measure Misleading study design and statistical analysis misleading conclusion
Patient’s problem Formulating Answerable question Search The evidence Critically Appraise The evidence Applying The evidence Evaluating the management
Journal Article Appraisal
Primary Reference
PIN IKABDI 12/8/2017 29
Strengths and Weakness 30
Major Types of Study Design Observational Study Case Reports, Case Series Cross sectional study Case-Control Study Cohort Study Meta-analysis Experimental Study Animal study Clinical Trials Community Trials
T i m e l i n e Different study design in relation to time line Case Control Clinical Trial Cohort study ` Case Series Cross Sectional Case Report
CASE-SERIES STUDIES Simple description of interesting observations in a small group of patients Generally not planned before. No research hypothesis Does not include controls subjects Thus, some investigators would not include case- series as studies. They lead to generation of hypothesis. 33
According to the purpose of EBM : Diagnosis : Cross-sectional Prognosis : Cohort, Survival study, Therapy : Clinical trial Harm : Clinical trial, cohort, case control
DIAGNOSTIC GOLD STANDARD DIAGNOSTIC TEST GROUPS OF PATIENT WITH SPESIFIC SIGN & SYMPTOM DISEASE + ― + True Pos False Pos TEST ― False Neg True Neg
TESTS ARE USEFUL IF THEY MOVE THE PATIENT ACROSS YOUR ACTION THRESHOLD Probability of disease before test (Pre-test probability) Probability of disease after test (Post-test probability) Action Threshold Expl: 30% 90% PRETEST PROBABILITY A MEASURE OF TEST PERFORMANCE (Likelihood ratios or sensitivity and specificity
DISEASE (GOLD STANDARD) + ― + A B TEST ― C D A+C B+D A+B+C+D Prevalence: (A+C) / (A+B+C+D) X 100% Expl: 30% Pre-test probability = Prevalence 30% Pre-test Odds = Prevalence / (1-Prevalensce) 30 / 70 = 0,43 DISEASE : NO DISEASE
SENSITIVITY: PEOPLE WITH THE DISEASE PROBABILITY OF A POSITIVE RESULT IN PEOPLE WITH THE DISEASE A / A + C SPECIFICITY: PEOPLE WITHOUT THE DISEASE PROBABILITY OF A NEGATIVE RESULT IN PEOPLE WITHOUT THE DISEASE D / B+D Sp P In ; Sn N Out A highly Sp ecific test, if P ositive result , rules the diagnosis In A highly S e n sitivity test, if N egative result, rules the diagnosis Out
If you want to quantify the effect of a diagnostic test, you need to specify the pre-test odds: the likelihood that the patient would have a specific disease prior to testing. Odds post-test = Odds pretest X likelihood ratio LIKELIHOOD RATIO: probability (likelihood) of the test result in people with disease probability (likelihood) of the test result in people without disease LIKELIHOOD RATIO FOR A POSITIVE TEST: SENSITIVITY 1 - SPECIFICITY LIKELIHOOD RATIO FOR A NEGATIVE TEST: 1 – SENSITIVITY SPECIFICITY
First, turn the pre-test probability to pre-test Odds Pre-test Odds X Likelihood ratio = Post-test Odds Then turn the Post-test Odds to Post-test Probability Odds = probability / (1 – probability) Probability = Odds / (1 + Odds) Post-test probability for test result positive = Positive Predictive Value Probability of person with test positive who have the disease ( A / A+B )
THE EASY WAY – WITH A NOMOGRAM You can use the likelihood ratio together with the pre-test probability to work out the post-test probability using the nomogram
0 .10 .20 .30 .40 .50 .60 .70 .80 .90 TEST THRESHOLD TREATMENT THRESHOLD Do not tes Do not treat Test, and treat on the basis of the test’s results Do not test Get on with treatment WILL THE RESULTING POST-TEST PROBABILITIES AFFECT OUR MANAGEMENT?
Rev Saude Publica. 2005 Jun;39(3):340-9. Epub 2005 Jun 30. Breast cancer's secondary prevention and associated factors. Sclowitz ML , Menezes AM , Gigante DP , Tessaro S . Departamento Materno-infantil, Faculdade de Medicina, Universidade Federal de Pelotas, Pelotas, RS, Brasil. OBJECTIVE: To evaluate the prevalence of secondary prevention of breast cancer and associated factors. METHODS: A population-based cross-sectional study was conducted in the city of Pelotas, southeastern Brazil, in 2002. The study sample comprised 879 women aged 40 to 69 years. Information was collected on demographic, social, economic, behavioral, biological and care management variables. Statistical analysis based on Poisson regression model was carried out. RESULTS: The prevalence of breast self-examination (BSE) was 83.5% (95% CI: 80.9-85.9). Of them, 80.4% (95% CI: 77.3-83.2) carried out BSE at least once a month. The prevalence of clinical breast examination was 83.3% (95% CI: 80.6-85.7). Mammography was occasionally performed in 70% (95% CI: 66.8-73.0) of the sample. Of these women, 83.7% (95% CI: 80.5-86.6) underwent mammography at least once in the last two years. Sixty-two percent (95% CI: 58.7-65.2) of the women interviewed attended a gynecological visit at least once in the last year. The factors mainly associated to the high prevalence of secondary prevention of breast cancer were: higher social status; greater association of risk factors for breast cancer; family history of breast cancer; hormone replacement therapy and previous breast biopsy or surgery. CONCLUSIONS: Preventive measures for breast cancer have been widely taken in the study sample; however, data points out to some limitations related to efficacy. Social and economic status seems to be a major determinant to gynecological care access and, consequently, access to secondary prevention of breast cancer.
CROSS-SECTIONAL STUDIES Cross-sectional studies analyse data collected on a group of subjects at one time rather than over a period of time “What is happening?” right now Because they focus on a point in time, they are sometimes also called prevalence studies. Surveys and polls are generally cross-sectional studies, although surveys can be part of a cohort study.
Cross-sectional Study
Cross-sectional Study No breast cancer Breast cancer Population Sample Risk factor (+) A (-) B Risk factor (+) C (-) D Statistics measure : Rate ratio A/A+B C/C+D (Prevalence ratio)
Br J Surg. 1994 May;81(5):733-5. The 5-year natural history of complicated diverticular disease. Farmakis N , Tudor RG , Keighley MR . Department of Surgery, Queen Elizabeth Hospital, Birmingham, UK. The natural history of complicated diverticular disease based on details of 300 patients entered into a national audit between 1985 and 1988 is reported. Questionnaires were sent to the general practitioners of 176 patients with this condition 5 years after hospital admission; 120 responded. Of these 120 patients, ten died from recurrent complicated diverticular disease, 29 died from other disorders and 81 remain alive. Forty of 110 patients (excluding those who died from recurrence) are still symptomatic or were so at the time of unrelated death. Thirty-nine patients developed a severe complication after the index admission, 14 of whom had the same complication initially. Of the 77 patients who had initially been managed by sigmoid resection, only two developed recurrent complications compared with 37 of 43 managed conservatively. Of the ten patients who died from recurrent diverticular disease, nine had not undergone sigmoid colectomy at or after the original admission. These data argue for interval sigmoid colectomy in most patients who initially present to hospital with complicated diverticular disease to prevent later development of potentially lethal complications.
COHORT STUDIES A cohort is a group of people who have something in common and who remain part of a group an extended period of time. Cohort studies ask the question “What will happen?” Typically prospective studies
Surg Endosc. 2005 Mar;19(3):334-7. Laparoscopic hand-assisted versus open transhiatal esophagectomy: a case-control study. Bernabe KQ , Bolton JS , Richardson WS . Department of General Surgery, Ochsner Clinic Foundation, 1514 Jefferson Highway, New Orleans, LA 70121, USA. BACKGROUND: This case-control study evaluated and compared the outcomes of laparoscopically assisted (LTE) and open transhiatal esophagectomy (OTE). METHODS: In this study, 17 patients who underwent LTE during this period August 1999 through June 2003 were compared with 14 matched control patients who underwent OTE during this period December 1989 through September 2001. The groups had stage I esophageal cancer or lesser disease at the preoperative evaluation. Patients with prior upper abdominal or thoracic surgery were excluded. RESULTS: There was no significant difference between the groups with respect to age, body mass index, American Society of Anesthesiology (ASA) classification, or operating time. The estimated blood loss was 331 (+/- 220) ml for LTE and 542 (+/- 212) ml for OTE (p = 0.01). The hospital stay was 9.1 (+/- 3.2) days for LTE and 11.6 (+/- 2.9) days for OTE (p = 0.04). Comparing only the last six LTE with the OTE, the operating time was 311 (+/- 31) min for LTE and 388 (+/- 14) min for OTE (p = 0.02). CONCLUSIONS: The findings showed shorter operative time, less blood loss, and a shorter hospital stay with LTE than with OTE.
CASE-CONTROL STUDIES Case-control studies begin with the absence or presence of an outcome and then look backward in time to try to detect possible causes or risk factors The cases in case-control studies are individuals selected on the basis of some disease or outcome The controls are individuals without the disease or outcome
Case Control Studies
Calculating the Odds Ratio
The odds ratio (OR) is an estimation of relative risk from a case-control study OR = ad / bc The ratio of The odds in favour of exposure among the cases (a/c) To the odds in favour of exposure among non-cases (b/d)
Interpretation of the Odds Ratio/ Relative risk OR/RR = 1 Exposure is not associated with outcome or disease OR/RR > 1 Increased exposure accompanies increased outcome 0 < OR/RR < 1 Increased exposure accompanies decreased outcome
Randomised trial of laparoscopic versus open cholecystectomy for acute and gangrenous cholecystitis. Kiviluoto T , Siren J , Luukkonen P , Kivilaakso E . Second Department of Surgery, Helsinki University Central Hospital, Finland. BACKGROUND: Laparoscopic cholecystectomy (LC) has become the treatment of choice for elective cholecystectomy, but controversy persists over use of this approach in the treatment of acute cholecystitis. We undertook a randomised comparison of the safety and outcome of LC and open cholecystectomy (OC) in patients with acute cholecystitis. METHODS: 63 of 68 consecutive patients who met criteria for acute cholecystitis were randomly assigned OC (31 patients) or LC (32 patients). The primary endpoints were hospital mortality and morbidity, length of hospital stay, and length of sick leave from work. Analysis was by intention to treat. Suspected bile-duct stones were investigated by preoperative endoscopic retrograde cholangiography (LC group) or intraoperative cholangiography (OC group). FINDINGS: The two randomised groups were similar in demographic, physical, and clinical characteristics. 48% of the patients in the OC group and 59% in the LC group were older than 60 years. 13 patients in each group had gangrene or empyema, and one in each group had perforation of the gallbladder causing diffuse peritonitis. Five (16%) patients in the LC group required conversion to OC, in most because severe inflammation distorted the anatomy of Calot's triangle. There were no deaths or bile-duct lesions in either group, but the postoperative complication rate was significantly (p=0.0048) higher in the OC than in the LC group: seven (23%) patients had major and six (19%) minor complications after OC, whereas only one (3%) minor complication occurred after LC. The postoperative hospital stay was significantly shorter in the LC than the OC group (median 4 [IQR 2-5] vs 6 [5-8] days; p=0.0063). Mean length of sick leave was shorter in the LC group (13.9 vs 30.1 days; 95% CI for difference 10.9-21.7). INTERPRETATION: Even though LC for acute and gangrenous cholecystitis is technically demanding, in experienced hands it is safe and effective. It does not increase the mortality rate, and the morbidity rate seems to be even lower than that in OC. However, a moderately high conversion rate must be accepted.
EXPERIMENTAL STUDIES Definition Animal or human studies (clinical trials). Controlled or uncontrolled Controls: Placebo, conventional procedure
Clinical Trial Treatment Placebo Population Sample No Yes No Yes Disease Random selection Blinded administration
OUTCOME NOT CURE CURE TOTAL THERAPY A B A+B CONTROL C D C+D TOTAL A+C B+D A+B+C+D THE EFFECT OF A THERAPY: RELATIVE RISK, RISK REDUCTION, RELATIVE RISK REDUCTION, ODDS RATIO, NUMBER NEEDED TO TREAT (NNT) THE PRECISION : P < 0.05; 95% CONFIDENCE INTERVAL
A VARIETY OF METHODS CAN BE USED TO DESCRIBE RESULTS EXPERIMENT EVENT RATE=EER : A / A+B CONTROL EVENT RATE = CER : C / C+D RELATIVE RISK (RR): (A / A+B : C / C+D) RATIO OF THE RISK IN THE INTERVENTION Vs IN THE CONTROL IF THERAPY HAS NO EFFECT, RR = 1 IF THERAPY DECREASE THE EVENT RATE, RR < 1 NOT CURE CURE TOTAL THERAPY 40 (A) 60 (B) 100 (A+B) CONTROL 50 (C) 50 (D) 100 (C+D) RR = 40% / 50% = 0, 8
RELATIVE RISK REDUCTION (RRR): |CER – EER| / CER PERCENTAGE REDUCTION IN EVENTS IN THE INTERVENTION GROUPCOMPARED WITH THE CONTROL GROUP NOT CURE CURE TOTAL THERAPY 40 (A) 60 (B) 100 (A+B) CONTROL 50 (C) 50 (D) 100 (C+D) RRR = 50% - 40% / 50% = 20% NOT CURE CURE TOTAL THERAPY 4 (A) 996 (B) 1000 (A+B) CONTROL 5 (C) 995 (D) 1000 (C+D) RRR = 0,5% - 0,4% / 0,5% = 20% RRR cannot discriminate huge treatment effects from small one
ABSOLUTE RISK REDUCTION (ARR): THE ABSOLUTE ARITHMETIC DIFFERENCE BETWEEN THE CER AND EER IF CER 50% & EER 40%, ARR = 10% IF CER 0,5% & EER 0,4%, ARR = 0,1% ARR IS MORE MEANINGFUL MEASURE OF TREATMENT EFFECTS THAN IS THE RRR THE INVERSE OF THE ARR (1/ARR) HAS THE USEFUL PROPERTY OF TELLING US THE NUMBER OF PATIENTS THAT WE NEED TO TREAT (NNT) WITH THE EXPERIMENTAL THERAPY IN ORDER TO PREVENT ONE ADDITIONAL BAD OUTCOME ARR = 10%, NNT = 1 / 10% = 10, MEAN ? ARR = 0,1%, NTT = 1 / 0,1% = 1000, MEAN ?
Lancet. 2005 Jul 7;365(9477):2098-104. Long-term survival and vascular event risk after transient ischaemic attack or minor ischaemic stroke: a cohort study. van Wijk I , Kappelle LJ , van Gijn J , Koudstaal PJ , Franke CL , Vermeulen M , Gorter JW , Algra A ; LiLAC study group . Department of Neurology, Rudolf Magnus Institute of Neuroscience, University Medical Centre Utrecht, Utrecht, Netherlands. BACKGROUND: Determinants of survival and of risk of vascular events after transient ischaemic attack (TIA) or minor ischaemic stroke are not well defined in the long term. We aimed to restudy these risks in a prospective cohort of patients after TIA or minor ischaemic stroke (Rankin grade< or =3), after 10 years or more. METHODS: We assessed the survival status and occurrence of vascular events in 2473 participants of the Dutch TIA Trial (recruitment in 1986-89; arterial cause of cerebral ischaemia). We included 24 hospitals in the Netherlands that recruited at least 50 patients. Primary outcomes were all-cause mortality and the composite event of death from all vascular causes, non-fatal stroke, and non-fatal myocardial infarction. We assessed cumulative risks by Kaplan-Meier analysis and prognostic factors with Cox univariate and multivariate analysis. FINDINGS: Follow-up was complete in 2447 (99%) patients. After a mean follow-up of 10.1 years, 1489 (60%) patients had died and 1336 (54%) had had at least one vascular event. 10-year risk of death was 42.7% (95% CI 40.8-44.7). Age and sex-adjusted hazard ratios were 3.33 (2.97-3.73) for age over 65 years, 2.10 (1.79-2.48) for diabetes, 1.77 (1.45-2.15) for claudication, 1.94 (1.42-2.65) for previous peripheral vascular surgery, and 1.50 (1.31-1.71) for pathological Q waves on baseline electrocardiogram. 10-year risk of a vascular event was 44.1% (42.0-46.1). After falling in the first 3 years, yearly risk of a vascular event increased over time. Predictive factors for risk of vascular events were similar to those for risk of death. INTERPRETATION: Long-term secondary prevention in patients with cerebral ischaemia still has room for further improvement.
P S Investigation S Sampling P Value Confidence intervals!!! Inference Results
The probability that any particular outcome would have arisen by chance. Standard scientific practice, which is entirely arbitrary, usually deems a P value of less than 1 in 20 (expressed as P<0.05, It means that this result would have arisen by chance on less than one occasion in 20) as "statistically significant" and a P value of less than 1 in 100 (P<0.01) as "statistically highly significant." P VALUE AND CONFIDENCE INTERVAL P VALUE
CONFIDENCE INTERVAL The range of numerical values in which we can be confident (to a computed probability, such as 90 or 95%) that the population value being estimated will be found. Confidence intervals indicate the strength of evidence; where confidence intervals are wide, they indicate less precise estimates of effect.
The larger the trial's sample size, the larger the number of outcome events and the greater becomes the confidence that the true relative risk reduction is close to the value stated. Thus the confidence intervals narrow and "precision" is increased. In a "positive finding" study the lower boundary of the confidence interval, or lower confidence limit, should still remain important or clinically significant if the results are to be accepted. In a "negative finding" study, the upper boundary of the confidence interval should not be clinically significant if you are to confidently accept this result. CONFIDENCE INTERVAL
R 300 220 300 218 Standard, n= 5000 Experimental, n=5000 Cholesterol level, mg/dl t = df = 9998 p = 0.0023 Clinical Statistical Clinical importance vs. statistical significance
Yes No Standard 0 10 New 3 7 Fischer exact test: p = 0.211 Clinical importance vs. statistical significance Absolute risk reduction = 30% Clinical Statistical
Level 1 of Evidence Level Therapy/Prevention, Aetiology/Harm Prognosis Diagnosis 1a SR (with homogeneity* ) of RCTs SR (with homogeneity* ) of inception cohort studies; CDR† validated in different populations SR (with homogeneity* ) of Level 1 diagnostic studies; CDR† with 1b studies from different clinical centres 1b Individual RCT (with narrow Confidence Interval‡ ) Individual inception cohort study with > 80% follow-up; CDR† validated in a single population Validating** cohort study with good††† reference standards; or CDR† tested within one clinical centre 1c All or none§ All or none case-series Absolute SpPins and SnNouts††
Level Therapy/Prevention, Aetiology/Harm Prognosis Diagnosis 2a SR (with homogeneity* ) of cohort studies SR (with homogeneity* ) of either retrospective cohort studies or untreated control groups in RCTs SR (with homogeneity* ) of Level >2 diagnostic studies 2b Individual cohort study (including low quality RCT; e.g., <80% follow-up) Retrospective cohort study or follow-up of untreated control patients in an RCT; Derivation of CDR† or validated on split-sample§§§ only Exploratory** cohort study with good††† reference standards; CDR† after derivation, or validated only on split-sample§§§ or databases 2c "Outcomes" Research; Ecological studies "Outcomes" Research Level 2 of Evidence
Level Therapy/Prevention, Aetiology/Harm Prognosis Diagnosis 3a SR (with homogeneity* ) of case-control studies SR (with homogeneity* ) of 3b and better studies 3b Individual Case-Control Study Non-consecutive study; or without consistently applied reference standards 4 Case-series (and poor quality cohort and case-control studies§§ ) Case-series (and poor quality prognostic cohort studies*** ) Case-control study, poor or non-independent reference standard 5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Level 3,4,5 of Evidence
Grades of Recommendation A consistent level 1 studies B consistent level 2 or 3 studies or extrapolations from level 1 studies C level 4 studies or extrapolations from level 2 or 3 studies D level 5 evidence or troublingly inconsistent or inconclusive studies of any level
Comparing the old world (before evidence based practice) and the new world (with evidence based practice) Old world EBM world Source of knowledge Expert opinion analysis of the evidence Essential skills Clinical Clinical plus ability to appraise evidence Essential information sources Experts Textbooks Selected journals Electronic access to all research evidence Cochrane Library Importance of statisticians, epidemiologist, economists, etc Low High Importance of gathering new evidence on patients Low High Consultant to Juniors Dictatorship Democratic Importance of keeping up to date Optional Essential Importance of access to research evidence Low High Relationship to patients Expert to pupil Potentially much more equal