Non Randomised Control Trial

non rct, quasi trial , time series design, uncontrolled trials, before and after trials

1. Non-randomized control trials Presented by - dr karishma s halageri

2. Contents Introduction Reasons For The Use Of Nonrandomized Studies Examples Of Nonrandomized Studies Types Of Non Randomized Trials Quasi-experimental Designs Threats To Establishing Causality When Using Quasi-experimental Designs Threats To Internal Validity Sources Of Bias In Non Randomized Trials Case Mix Adjustment Methods Implications For Using Non Randomized Trials Conclusion References

3. Introduction The ultimate goal of the evaluation of healthcare interventions is to produce a valid estimate of effectiveness, in terms of both internal and external validity. Internal validity concerns the extent to which the results of a study can be reliably attributed to the intervention under evaluation W hereas external validity concerns the extent to which a study’s results can be generalized beyond the given study context.

4. Although experimental method is almost always to be preferred, it is not always possible for ethical, administrative and other reasons to resort to randomized control trial in human beings. Secondly, some preventive measures can be applied only to groups or on community wide basis . Thirdly when disease frequency is low and natural history long, RCT require follow up of thousands of people for a decade or more. The cost and logistics are often prohibitive. In such situations we must depend on other study designs such as Non-Randomized trials Ex: community trials on water fluoridation. Ex: cancer cervix Introduction

5. As there is no randomization in non-experimental trials, the degree of comparability will be low and chances of spurious result higher than where randomization had taken place.

6. REASONS FOR THE USE OF NONRANDOMIZED STUDIES

7. 1. Nonrandomized studies are sometimes the only ethical way to conduct an investigation If the treatment is potentially harmful, it is generally unethical for an investigator to assign people to this treatment . An example of this is, 1. A study of the effects of malnutrition, where we simply cannot assign subjects to intolerable diets. Thus we compare malnourished populations with those on adequate diets. 2. A study of the effects of carbonated drinks on tooth erosion, where we cannot assign subjects to such habits. Thus we compare population with regular consumption of carbonated drinks and population who don’t consume such drinks.

8. 2. Nonrandomized studies are sometimes the only ones possible. Certain investigations require the implementation of treatments that may affect people's lives . In a democratic society randomized implementation of such treatments is not always feasible. Example: The question of fluoridating a town's water supply . We would have a series of towns, some of which have elected fluoridation and others which have not. The dental experience of the children in these towns can provide a great deal of useful information if properly analysed.

9. 3. Nonrandomized studies are usually less expensive. An advantage of nonrandomized studies is that they usually cost less per subject and may not require the extensive planning and control that are needed for randomized studies . This makes nonrandomized studies particularly attractive in the early stages of any research effort.

10. 4. Nonrandomized studies may be closer to real-life situations. To the extent that randomization differs from natural selection mechanisms, the conditions of a randomized study might be quite different from those in which the treatment would ordinarily be applied . Example: A program may be very successful for those who choose it themselves on the basis of a media publicity campaign but ineffective when administered as a social experiment.

11. Examples of non-randomized trials

12. 1. Uncontrolled trials These are trials with no comparison group. Initially uncontrolled trials may be useful in evaluating whether a specific therapy appears to have any value in particular disease to determine an appropriate dose To investigate adverse reactions Even in these uncontrolled trials, one is using implied “historical controls”, i.e., the experience of earlier untreated patients affected by the same disease.

13. 1. Uncontrolled trials It is becoming increasingly common to employ the procedures of a double-blind controlled clinical trial in which the effect of new drug are compared to some concurrent experience. (either placebo or currently utilized therapy)  Uncontrolled trials may be useful in evaluating whether a specific therapy appears to have any value in a particular disease, to determine an appropriate dose, to investigate adverse reactions, etc.

14. 2. Natural experiments Where experimental studies are not possible in human populations, the epidemiologist seeks to identify “natural circumstances” that mimic an exper iment. For example: in respect to cigarette smoking People have separated themselves “naturally” into 2 groups, smokers an non-smokers. Other population involved in natural experiments comprise the following groups: a) Migrants b) religious or social groups c) famines d) Earthquakes

15. 2. Natural experiments A major earthquakes in Athens in1981 provided a natural experiments to epidemiologists who studied the effects of acute stress on cardiovascular mortality. They showed an excess of deaths from cardiac and external causes on the days after the major earthquake, but no excess deaths from other causes. John Snows discovery that cholera is a water borne disease was the outcome of a natural experiment.

16. Example

17. Quasi-experimental designs Quasi-experimental studies encompass a broad range of nonrandomized intervention studies. These designs are frequently used when it is not logistically feasible or ethical to conduct a randomized controlled trial . These studies aim to evaluate interventions but that do not use randomization. Similar to randomized trials , quasi-experiments aim to demonstrate causality between an intervention and an outcome. Quasi-experimental studies can use both pre intervention and post intervention measurements as well as non randomly selected control groups .

18. R esearchers often choose not to randomize the intervention for one or more reasons: Ethical Considerations Difficulty Of Randomizing Subjects Difficulty To Randomize By Locations (E.G., By Wards) Small Available Sample Size Quasi-experimental designs

19. WHEN IS IT APPROPRIATE TO USE QUASI-EXPERIMENTAL METHODS ? Quasi-experimental methods can be used, i.e., after the intervention has taken place (at time t+1). In some cases, especially for interventions that are spread over a longer duration, preliminary impact estimates may be made at mid-term (time t). It is always highly recommended that evaluation planning begins in advance of an intervention, however. This is especially important as baseline data should be collected before the intended recipients are exposed to the programme / policy activities (time t-1).

20. Different Quasi-experimental Study Designs ( i ) Before And After Studies without Control; (Ii) Time Series Designs; And (Iii) Before And After Studies with Control.

21. 3.Before and after comparison studies

22. Before and after comparison studies without control . These studies centre round comparing the incidence of disease before and after introduction of preventive measure. The experiment serves as its own control; this eliminates virtually all group differences . The events which took place prior to the use of the new treatment or preventive procedure are used as a standard for comparison.

23. Classic examples of “before and after comparison studies” were

24. This table gives an example of a "before and after comparison study" in Victoria (Australia) following introduction of seat-belt legislation for prevention of deaths and injuries caused by motor vehicle accidents.

25. before and after studies without control The intervention is confounded by the Hawthorne effect (the non-specific beneficial effect on performance of taking part in research ) which could lead to an overestimate of the effectiveness of an intervention. In general, before and after studies without control should not be used to evaluate the effects of guideline implementation strategies, and the results of studies using such designs have to be interpreted with great caution.

26. In order to establish evidence in before and after comparison studies , we need: Data –regarding incidence of disease, before and after introduction of preventive measure must be available. Introduction or manipulation of only one factor or change relevant to the situation, other factors remaining the same. Ex; addition of fluoride to drinking water to prevent dental caries. Diagnostic criteria of the disease should remain the same. Adoption of preventive measures should be over a wide area Reduction in the incidence must be large following the introduction of the preventive measure, because there is no control . Several trials may be needed before the evaluation is considered conclusive

27. Time series designs Time series designs attempt to detect whether an intervention has had an effect significantly greater than the underlying trend. They are useful in guideline implementation research for evaluating the effects of interventions when it is difficult to randomize or identify an appropriate control group.

28. Data are collected at multiple time points before and after the intervention; the multiple time points before the intervention allow the underlying trend to be estimated, the multiple time points after the intervention allow the intervention effect to be estimated accounting for the underlying trend. Time series designs

29. Time series designs increase the confidence with which the estimate of effect can be attributed to the intervention, although the design does not provide protection against the effects of other events occurring at the same time as the study intervention, which might also improve performance . Furthermore , it is often difficult to collect sufficient data points unless routine data sources are available . Currently , many published interrupted time series have been analysed inappropriately, frequently overestimating the effect of the intervention. Time series designs

30. Single-Group Interrupted Time-Series Design In this design, the researcher records measure for a single group both before and after a treatment. Group A O------O------O------O------- X ------O-----O-----O------O

31. Control-Group Interrupted Time-Series Design This is a modification of Single-Group Interrupted Time-Series Design in which two groups of participants, not randomly assigned, are observed over time. A treatment is administered to one of the group ( i.e. group A) Group A O------O------O------O------- X ------O-----O-----O------O Group B O------O------O------O------- O------O-----O-----O------O

32. c . Before and after comparison studies with control In the absence of control group, comparison between observations before and after the use of a new treatment or procedure may be misleading. In these situation, the epidemiologist tries to utilize a “natural” control group i.e., the one provided by natural or natural circumstances. If preventive programme is to be applied to an entire community, we would select another community as similar as possible, particularly with respect to frequency and characteristics of the disease to be prevented.

33. I n this example, the existence of a control with which the results in victoria could be compared strengthens the conclusion that there was definite fall in the number of deaths and injuries in occupants of cars after the introduction of compulsory seat-belt legislation.

34. Data are collected in both populations contemporaneously using similar methods before and after the intervention is introduced in the study population. A ‘between group’ analysis comparing performance in the study and control groups following the intervention is undertaken, and any observed differences are assumed to be due to intervention .

35. Non-equivalent (Pretest and Post-test) Control-Group Design In this design, the experimental Group A and the control Group B are selected with random assignment. Both groups take a pre-test and post-test. But only the experimental group receives the treatment. Group A O------- X ------O Group B O----------------O

36. Threats to Establishing Causality When Using Quasi-experimental Designs The lack of random assignment is the major weakness of the quasi-experimental study design.

37. Threats to internal validity Ambiguous temporal precedence Lack of clarity about whether intervention occurred before outcome Selection Systematic differences over conditions in respondent characteristics that could also cause the observed effect History Events occurring concurrently with intervention could cause the observed effect Maturation Naturally occurring changes over time could be confused with a treatment effect Regression When units are selected for their extreme scores, they will often have less extreme subsequent scores, an occurrence that can be confused with an intervention effect Attrition Loss of respondents can produce artifactual effects if that loss is correlated with intervention Testing Exposure to a test can affect scores on subsequent exposures to that test Instrumentation The nature of a measurement may change over time or conditions

38. Sources of bias in nonrandomized studies F our main sources of systematic bias in trials of the effects of healthcare as being: Selection Bias Performance Bias- if there are errors and inconsistencies in the allocation , application and recording of interventions Attrition Bias - will occur if there are dropouts, Detection Bias - if the assessment of outcomes is not standardized and blinded All of these biases can also occur in RCTs, but there is perhaps potential for their impact to be greater in non-randomized studies which are usually undertaken without protocols specifying standardised interventions, outcome assessments and data recording procedures

39. Selection bias Randomized and Non-Randomized studies is, the risk of selection bias , where systematic differences in comparison groups arise at baseline. It is sometimes referred to as case-mix bias, or confounding. The term selection bias can be misleading as it is used to describe both Biased selection of participants for inclusion in a study (which applies to both experimental and observational studies) - classified as an issue of external validity Biased allocation of patients to a given intervention (which occurs where randomization is not used) - is an issue of internal validity.

40. WHEN SELECTION BIAS WILL BE INTRODUCED IN NON RANDOMIZED CONTROL TRIALS .. when participants chosen for one intervention have different characteristics from those allocated to the alternative intervention ( or not treated ). The choice of an intervention under these circumstances will be influenced not only by a clinician’s own personal preference for one intervention over another but also by patient preference , patient characteristics and clinical history.

41. Protopathic bias is a term coined by Horwitz and Feinstein15 to describe situations where the first symptoms of a given outcome are the reason for treatment initiation: “ Protopathic bias” occurs “when a pharmaceutical or other therapeutic agent is inadvertently prescribed for an early manifestation of a disease that has not yet been diagnostically detected” (our emphasis ). For example , a drug given for abdominal pain may be wrongly associated with hepatic injury, as abdominal pain may be one of the prodromal symptoms. A drug given for persistent mouth ulcer may be wrongly associated with oral cancer, as persistent mouth ulcer may be one of the prodromal symptoms.

42. Case-mix adjustment methods In the absence of information on factors influencing allocation, the traditional solution to removing selection bias in non-randomized studies has been to attempt to control for known prognostic factors, either by design and/or by analysis . STANDARDISATION Participants are analysed in groups (strata) which have similar characteristics, the overall effect being estimated by averaging the effects seen in each of the groups

43. implications for those producing, reviewing and using nonrandomized studies An investigator planning to undertake a nonrandomized study should first make certain that an RCT cannot be undertaken . The ability to eradicate bias at the design stage is crucial to establishing the validity of a study. In particular, investigators should not assume that statistical methods can be used reliably to compensate for biases introduced through suboptimal allocation methods. A prospective non-randomized study should be undertaken according to a protocol that is carefully followed to ensure consistent inclusion criteria , that all relevant factors are measured accurately for each participant and that participants are all monitored in a standard manner and blinded to treatment if possible.

44. In some situations it may even be possible to match prospectively treated and control patients on important prognostic factors Healthcare decision-makers should be cautious not to over-interpret results from non-randomized studies. Importantly, checking that treated and control groups appear comparable does not guarantee freedom from bias, and it should never be assumed that case-mix adjustment methods can fully correct for observed differences between groups .

45. Conclusion Non-randomized studies are sometimes but not always biased, The results of non-randomized studies can differ from the results of RCTs of the same intervention. Statistical methods of analysis cannot properly correct for inadequacies of study design. Systematic reviews of effectiveness often do not adequately assess the quality of non-randomized studies. Non-randomized studies provide a poor basis for treatment or health policy decisions.

46. References K Park. Park’s textbook of preventive and social medicine.2019;25 th ed:61-78 Gordis L. Text book of Epidemiology. 5th ed. Elsevier Roger Detels et al. Oxford Text Book of Public Health. 5th ed. New york (U.S.A): Oxford University Press; 201 JJ Deekset et al. Evaluating non-randomized intervention studies : Health Technology Assessment 2003; Vol. 7: No. 27 Friis RH, Sellers TA. Epidemiology for Public Health Practice . 4th ed. Sudbury, MA: Jones and Bartlett Publishers; 2009. MacMahon B, Pugh TF. Epidemiology Principles and Methods . Boston, MA: Little, Brown; 1970. Merrill.M . Introduction to Epidemiology.2010;5 th ed:83-153. Bonita R, Beaglehole R, Kjellstrom K. Basic Epidemiology.2006 Jan;2 ND ed Bhalwar R. Text Book of Public Health and Community Medicine. 1st ed. Pune: Dept of Community Medicine, AFMC. 2009. P. 144 D’Agostino RB, Kwan H. Measuring effectiveness: what to expect without a randomized control group . Med Care 1995;33:95–105 . Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Family practice. 2000 Feb 1;17(suppl_1):S11-6.