BIAS IN EPIDEMIOLOGICAL STUDIES. INTERNAL AND EXTERNAL VALIDITYppt
mitraharauwati
56 views
124 slides
Jul 12, 2024
Slide 1 of 124
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
About This Presentation
Bias in Epidemiological Studies
Size: 1.57 MB
Language: en
Added: Jul 12, 2024
Slides: 124 pages
Slide Content
Bias in Epidemiological Studies
Ratna Djuwita
Departemen Epidemiologi
FKMUI
Basic Question in Analytic Epidemiology
•Are exposure and disease linked?
Exposure Disease
Basic Questions in Analytic Epidemiology
•Look to link exposure and disease
–What is the exposure?
–Who are the exposed?
–What are the potential health effects?
–What approach will you take to study
the relationship between exposure and
effect?
Wijngaarden
Studi Kohort
•Memilih sekelompok orang (kohort) tidak sakit
(tanpa penyakit yang diteliti)berdasarkan ada /
tidaknya suatu faktor risiko (konsumsi red meat)
•Kelompok tersebut kemudian diikuti selama
periode waktu untuk melihat terjadinya penyakit
yang diteliti (kanker kolon)
•Prospektif dan retrospektif (menggunakan data
yang sudah ada)
unexposed
exposed
Cohort studies
unexposed
exposed
Incidence among
exposed
Incidence among
unexposed
Cohort studies
Kekuatan Studi kohort
Pilihan desain untuk studi dimana pajanan
jarang
Dapat mempelajari asosiasi antara satu
pajanan dgn outcome lebih dari satu
Dapat memperlihatkan hubungan
temporal antara pajanan dan outcome
Bias pengukuran pajanan dapat diperkecil
Dapat mengukur incidence
Keterbatasan Studi Kohort
Tidak efisien untuk meneliti penyakit yg
jarang terjadi
Jika prospective, biaya besar dan waktu
lama
Jika retrospective, perlu catatan yg
adekuat
Kemungkinan losses to follow-up besar
dan merupakan ancaman validitas hasil
penelitian
10
•Are the results believable? (internal validity)
•Can results from study participants be
extrapolated to the broader population?
(external validity)
Why do we need to talk about bias?
EXTERNAL VALIDITY
(generalizability)
The degree to which the results of an
observation hold true
If the study was repeated in the same
population using the same methods,
approximately the same results would be
obtained
13
External validityof a study refers to
the appropriateness by which its
results can be applied or generalized
to non-study patients or populations.
External Validity
INTERNAL VALIDITY
The degree to which the results of an
observation are correct for the subjects
being studied
15
Internal validityof a study refers to
the integrity of the experimental
design.
Internal Validity
Any trend in the collection, analysis, interpretation,
publication or review of data that can lead to
conclusions that are systematically different from
the truth (Last, 2001)
A process at any state of inference tending to
produce results that depart systematically from
the true values(Fletcher et al, 1988)
Systematic error in design or conduct of a study
(Szklo et al, 2000)
What is Bias?
A
D
C
B
VALIDITY
The degree to which the results of a
measurement corresponds to the true state
of the phenomenon being measured
If the findings can be taken as being a
reasonable representation of the true
situation
24
The extent to which a measurement
measures what it is supposed to
measure or accomplishes what it is
supposed to accomplish.
Validity
RELIABILITAS
KEMAMPUAN TEST ATAU
PENGUKURAN UNTUK
MENGHASILKAN NILAI YANG
SAMA PADA INDIVIDU DAN
KONDISI YANG SAMA
ERRORS
IF THERE IS ANY DEVIATION OF
RESULTS FROM THE “TRUE”VALUES
Quality of an estimate
Precision
& validity
No precision
Random
error!
Precision but
no validity
Systematic
error (Bias) !
32
Example
Measuring height:
•Measuring tape hold differently by different
investigators
→ loss of precision
–error
•Tape shrunk/wrong
→ systematic error
–bias (cannot be corrected afterwards!)
179
177
178
175
176
173
174
180
ERRORS
A. RANDOM ERROR
error arising from random variation;because
of random variation the characteristics of people in a
particular samples are different from others in the
population from which they were taken
any trend in the collection, analysis, interpretation,
publications or review of data that can lead to
conclusions that are systematically different from the
truth
B. SYSTEMATIC ERROR = BIAS
0
2
4
6
8
10
12
14
0 5 10 15 20 25 30 35 Size of induration (mm)
Per Cent
Random Error
WHO (www)
0
2
4
6
8
10
12
14
0 5 10 15 20 25 30 Systematic Error
Per Cent
Size of induration (mm)
WHO (www)
Chance vs Bias
Chance is caused by random error
Bias is caused by systematic error
Errors from chance will cancel each other out in the
long run (large sample size)
Errors from bias will not cancel each other out
whatever the sample size
Chance leads to imprecise results
Bias leads to inaccurate results
Errors in epidemiological studies
THREATS TO VALIDITY
A study’s internal validity, or how close its
findings are to the TRUTH, can be
compromised by three things….
•CHANCE
•BIAS
•3rd VARIABLES -CONFOUNDINGor
INTERACTION
Threats to Validity -CHANCE
CHANCE
•the unpredictable and uncontrollable element
of an event or occurrence
•how likely is it that an association between an
exposure and disease is the result of a
random set of events
Threats to Validity -BIAS
BIAS
•any systematic error (not random) in a study
that leads to an incorrect estimate of the
association between exposure and disease
•can occur in the design, implementation, or
analysis stages of a study
Threats to Validity-THIRD VARIABLES
CONFOUNDING OR INTERACTION
•the influence of third variables in a study
which lead to an incorrect estimate of the
association between the exposure and
disease variables
•confounding and interaction ARE NOT THE
SAME!…..stay tuned…..
EVALUATING CHANCE
To evaluate the role of CHANCEin a study we
use hypothesis testing and confidence
intervals…
•HYPOTHESIS TESTING -performing tests to
determine the probability (i.e. the p-value)
that an association observed in a study is due
to chance alone
•CONFIDENCE INTERVALS -the range within
which the true observation in a study will lie
given a specified degree of assurance
BIAS
45
Definition of bias
Any systematic errorin an epidemiological
study that results in an incorrect estimate of
the association between exposure and
disease
Systematic variation of measurements
from the true value (Last J.)
46
Review -Model for Clinical Epidemiology
Exposure
(tobacco
smoke)
Outcome
(lung cancer)
ASSOCIATION
Causation?
Bias?
Chance?
Confounding?
In the 1940’s, Alton Ochsner, a surgeon, observed
that virtually all patients on whom he operated for lung
cancer gave a history of cigarette smoking.
47
Sample
Exposed
Group
Unexposed
Group
Outcome
Outcome
Target
Population
Prospective
Sampling Bias
Selection Bias
& Confounding
Three Types of Bias
Information BiasInformation Bias
48
Sample
Exposed
Group
Unexposed
Group
Outcome
Outcome
Target
Population
Prospective
Sampling Bias
Sampling Bias
49
From target population to sample
The target populationis
the group to which we
would like to apply the
results (extrapolate).
The study (or sampling)
populationis the group
from which the sample
was actually drawn.
The sampleis a subset of
the study population.
If the study population is not the same as the
target population, sampling bias mayresult.
Target Population
Study Population
Sample
50
Sampling Bias
•Sigmund Freud advanced theories about all adults based on
his observations of his patients.
•While his conclusions may have been valid with regard to his
patients (internal validity), extrapolation to all adults (external
validity) is problematic because his patients may not be
representative of all adults.
Target Population
(all adults)
Study
Population
(Freud’s
clinical
practice)
Unbiased Sample Biased Sample
Study A (200 subjects) Study B (2000 subjects)
0 5 10 15 0 5 10 15
Risk (percent) Risk (percent)
Fig. 10-3.The effect of sample size on precision of risk
estimates. Point estimates are shown as dashed vertical lines
and 95% confidence intervals are shown as solid horizontal lines.
In both studies, the 5-year risk of mycordial infarction was 9%
among persons receiving dietary therapy and 6% among
persons treated with a Cholesterol-lowering drug. In the larger
study, however, the 95% confidence intervals are narrower, and
the difference in risk between treatment groups is statistically
significant.
Drug
Diet
Drug
Diet
53
Sampling biasoccurs when the
people in a study (i.e., the sample)
are not representative of the target
population.
Sampling Bias
54
Sampling Bias
•If the results of a study are influenced by sampling bias,
then the results may be valid for the study population
(internal validity), but not for the target population
(external validity).
•If the sample is representative of the target population
–there is no sampling bias
–the results may be extrapolated to the target
population
–the results have external validity
55
Sampling Bias –Common Examples
•Extrapolation of results in younger adults to older adults
or children may not be appropriate.
•Extrapolation of results in men to women may not be
appropriate.
•Extrapolation of results in hospitalized or
institutionalized patients to community-dwelling, non-
hospitalized patients may not be appropriate.
•Extrapolation of results in patients attending specialty
clinics or services to patients seen by primary care
physicians may not be appropriate.
56
Avoiding Sampling Bias
•Random samplingand probability samplingare the best
methods for assuring that the sample is representative of
the target population with regard to important variables.
–Random sample
•Each member of population has equal chance of
being selected for sample.
•Subgroup proportions in the sample equal subgroup
proportions in the population.
–Probability sample
•Each member of population has known probability of
being selected for sample.
•Some subgroups are over-or under-sampled, but
calculations are weighted to adjust for this.
Bias
SelectionInformation
Types of Bias
•Selection bias
–Concerned about who is in your study
•Information bias
–Concerned about the information you
elicit from your subjects
Selection bias
Errors in selecting the study population
•When ?
-Inclusion in the study
•How ?
-Preferential selection of subjects
related to their
Disease status cohort
Exposure status case control
REDUCING BIAS
To reduce the role of BIASin a study we
examine and try to eliminate its potential
sources…
•in general, bias is a result of an ERRORin
the designor implementationof a study
•main types of bias include...
–selection bias
–information bias
•misclassification bias
•recall bias
SELECTION BIAS
Selection Bias:Error due to a systematic difference
between those selected for a study and those NOT
selected for a study
•distorts the true strength of association
–RR
true -RR
obs= RR
due to bias
•can occur in both case-control and cohort study designs
–difference in way cases vs. controls selected
–difference in way exposed vs. non-exposed selected
955
45
RR
50% of cases that smoked
lost to follow up
Selection Bias Examples
(www)
EXAMPLES OF SELECTION BIAS
•significant loss to follow-up in a cohort study
–results in differential drop-out rates
•EXAMPLE:
–GOAL:study saccharin exposure and cancer outcomes
–STARTING POPN: 1000 6th grade students
–RESULTING POPN: 300 40-year old adults
–BIAS: because so few adults returned at follow-up it is
possible that they have unique qualities (a heightened
interest in the study due to cancer histories, a more
stable lifestyle, etc.) and thus may be different from
those who dropped-out of the study
EXAMPLES OF SELECTION BIAS
ACTUAL OR IF TOTAL FOLLOW-UP: 0.30
sacc nosacc
CANCER 300 200 500
NO CA 400 100 500
700 300 1000
STUDY OR WITH POOR FOLLOW -UP: 2.00
sacc nosacc
CANCER 100 75 175
NO CA 50 75 125
150 150 300
CHANGE IN OR:
OR
study> OR
true
SB: Loss to follow-up
Difference in completeness of follow-up between comparison groups
-e.g. study of disease risk in migrants
–migrants more likely to return to place of origin when
having disease
lost to follow-up
lower disease rate among exposed (=migrant) Cases of
serious
disease
Non-cases
Migrants a b
Not migrants c d
b
cd
a
EXAMPLES OF SELECTION BIAS
•at the start of the study you fail to know the
demographics and/or the epidemiology of disease
in the total population
–selection bias can occur because study population is not
representative of target population
•EXAMPLE:
–GOAL:study drug use in teenagers and divorce
–TARGET POPN:teenagers STUDY POPN:high schools
–BIAS:using high schools as your study population
allows you to miss drop-outs in the target population
that may have high drug use
EXAMPLES OF SELECTION BIAS
ACTUAL OR WITH DROP-OUTS INCLUDED: 1.16
divorce nodivorce
DRUGS 200 150 350
NO DRUGS 400 350 750
600 500 1100
STUDY OR WITHOUT DROP-OUTS: 1.5
divorce nodivorce
DRUGS 150 100 250
NO DRUGS 350 350 700
500 450 950
CHANGE IN OR:
OR
study> OR
true
EXAMPLES OF SELECTION BIAS
•significant refusals at start of study (differential
participation)
•EXAMPLE:
–GOAL:study smoking habits in Americans and lung cancer
–TARGET POPN:Americans STUDY POPN:few minorities
–BIAS: minorities have lower participation rates in scientific
studies and therefore allow you to miss minority smokers
in your target population that may have unique qualities
EXAMPLES OF SELECTION BIAS
ACTUAL ORINCLUDING MINORITIES: 3.0
smoke no smoke
LUNG CA 300 50 350
NO CA 300 150 450
600 200 800
STUDY ORWITHOUT MINORITIES: 0.8
smoke no smoke
LUNG CA 300 75 375
NO CA 250 50 300
550 125 675
CHANGE IN OR:
OR
study<<OR
true
CONTROLLING FOR SELECTION BIAS
•In a cohort study…
–instill mechanisms for high participation
rates
–get basic information on those who refuse to
participate
–develop effective follow-up mechanisms
Minimising Selection Bias
•Be aware
–Potential sources of selection bias
•Equal opportunity for participation and follow-up
–Cases / Controls
–Exposed / Unexposed groups
–Intervention / Control groups
•Tactics for high participation / follow-up rates
–Reminders / Postcards / Phone calls
Information bias
Systematic error in the measurement
of information on exposure or outcome
•When?
During data collection
•How?
Differences in accuracy
-of exposuredata between cases and controls
-of outcomedata between exposed and unexposed
Information bias
•When?
•How?
•Consequences?
Misclassification:
Study subjects are classified
in the wrong category
Cases / controls
Exposed / unexposed
INFORMATION BIAS
Information Bias: a flaw in measuring
exposure or outcome data that results in a
differing quality (accuracy) of information
between comparison groups
•also called Observation Bias
•distorts the true strength of association
•occurs in all study designs but often described
as RECALL BIAS in case-control studies
Information Bias
•What information are you getting from subjects?
•Concern
–Are there systematic differences in what is
being collected, between study groups?
–Does each subject have an equal chance of
providing the same information?
•Sources
–Observer bias
–Recall bias
86
Two main types of
information bias
•Reporting bias
–Recall bias
•Observer bias
–Interviewer bias
–Biased follow-up
Attention Bias
•Hawthorne Effect
An increase in worker productivity produced by
the psychological stimulus of being singled
out and made to feel important
•People may respond differently if they think they
know what is being studied
•Potential effect
–↑↓ prevalence of disease
–↑↓ relationship under examination
Minimising Attention Bias
•Mask true study question from participants
–Ethics
–Informed consent
–“Health” study
•Collect information about several outcomes
–Difficult in a case-control study
•Collect information about several exposures
•Ensure anonymity
Surveillance Bias
•Can arise if one group is over-researched, in
comparison with the other
•Case-control study
–Tendency to examine more closely those with
outcome of interest
–Association: alcohol consumption vs
oropharyngeal cancer
•Cohort study / Randomised trial
–Tendency to follow more closely (or for longer)
those with exposure of interest
–Association: CBT vs low back pain
Minimising Surveillance Bias
•Ensure identical methodological
procedures for all study participants
•Where possible, blind researchers
–To study question
–To case / control status
–To exposure / non-exposure status
–To treatment / non-treatment group
Observer Bias
•Interviewer knowledge may influence structure of
questions
•Preconceived expectations of study outcome
•Study methods may change over time
•Different investigators may examine different
subjects
•Times / locations of interviews may vary
Minimising Observer Bias
•Standardised techniques / instruments / etc
–Thorough training of data collection staff
–Test agreement between interviewers /
instruments
•Use objective measurements where possible
•Where possible, researchers should be
–Randomly allocated to subjects
–Blind to study question
–Blind to case / control status
Observer Bias
•Interviewer knowledge may influence structure of
questions
•Preconceived expectations of study outcome
•Study methods may change over time
•Different investigators may examine different
subjects
•Times / locations of interviews may vary Attention Bias
•Hawthorne Effect
–Western Electric Co, Illinois
–An increase in worker productivity produced by the
psychological stimulus of being singled out and made to
feel important
•People may respond differently if they think they
know what is being studied
•Potential effect
–?? prevalence of disease
–?? relationship under examination Surveillance Bias
•Can arise if one group is over-researched, in
comparison with the other
•Case-control study
–Tendency to examine more closely those with outcome of
interest
–Association: alcohol consumption vsoropharyngealcancer
•Cohort study / Randomised trial
–Tendency to follow more closely (or for longer) those with
exposure of interest
–Association: CBTvslow back pain Recall Bias
•Major concern where exposure data measured
retrospectively
–Case-control studies (including case-control analysis of
cross-sectional survey)
•Concern
–Differential recall between cases and controls
EXAMPLES OF INFORMATION BIAS
•the way information is extracted from medical
forms
–cases or exposed subjects may have more medical
records or more detailed records
•EXAMPLE:
–GOAL:evaluate workplace hazards and exposures
–INFORMATION SOURCE: nationwide, uniform coding of
occupation and industry on death certificates (Lilienfeld, 1988)
–BIAS:those who die of non-smoking related lung
cancer, or other “non-traditional”causes, may have
more or more detailed occupational information in their
medical records thus over-emphasizing their exposure
relative to those who die of “traditional”causes
EXAMPLES OF INFORMATION BIAS
ACTUAL OR WITHOUT CODING INEQUALITY : 3.0
workexp nowkexp
CANCER 350 100 450
NO CA 350 300 650
700 400 1100
STUDY OR WITH INEQUALITY: 1.16
workexp nowkexp
CANCER 200 150 350
NO CANCER 400 350 750
600 500 1100
CHANGE IN OR:
OR
study < OR
true
EXAMPLES OF INFORMATION BIAS
•the way in which an interviewer asks questions or
interprets answers or observations
–may ask questions or interpret answers differently for
different participant groups based on preconceived ideas
•EXAMPLE:
–GOAL:assess rates of attention deficit disorder in children
–INFORMATION SOURCE: the Behavior Risk Index Scale
–BIAS: this scale helps interviewers objectively distinguish
and classify children’s habits without interjecting their own
preconceived ideas or judgementsbased on appearance or
prior knowledgethus decreasing info bias (Dean & Smith, 1998)
EXAMPLES OF INFORMATION BIAS
ACTUAL ORUSING SCALE (very similar!): 1.125
habitA nohabit
ADD 250 250 500
NO ADD 400 450 850
650 700 1350
STUDY ORWITHOUT USING SCALE: 1.10
habitA nohabit
ADD 250 250 500
NO ADD 450 500 950
700 750 1350
CHANGE IN OR:
OR
studyOR
true
EXAMPLES OF INFORMATION BIAS
•when using surrogate interviewees
–using parents, spouses or siblings may result in less
accurate data for cases than controls
•EXAMPLE:
–GOAL: assess dietary fat contribution to MI outcome
–INFORMATION SOURCE: spouse
–BIAS: spouses of MI victims may be more likely to
criticize the diets of their deceased and more accurately
recall fat intake whereas spouses of control victims (say
those who died of alzheimers) may not as closely
examine the diet of their deceased
EXAMPLES OF INFORMATION BIAS
ACTUAL OR WITH SPOUSES’100% ACCURATE : 0.33
fat nofat
MI 100 75 175
NO MI 100 25 125
200 100 300
STUDY OR WITH LESS ACCURACY: 2.00
fat nofat
MI 100 75 175
NO MI 50 75 125
150 150 300
CHANGE IN OR:
OR
study > OR
true
CONTROLLING FOR INFORMATION BIAS
•Blinding
–prevents investigators and interviewers from
knowing case/control or exposed/non-exposed
status of a given participant
•form of survey
–mail may impose less “white coat tension”than a
phone or face-to-face interview
•use multiple questions that ask same
information
–acts as a built in double-check
•multiple checks in medical records
–gathering diagnosis data from multiple sources
MISCLASSIFICATION BIAS
Misclassification Bias: the erroneous
classification of an individual, a value, or an
attribute into a category other than that to
which it should be assigned
•often results from an improper “cutoff point”
in disease diagnosis or exposure classification
•can be differential or non-differential (see next
slide!)
MISCLASSIFICATION BIAS
•Differential Misclassification Bias
–rate of misclassification differsbetween study
groups (case/control or exposed/non-exposed)
–can lead to an apparent (false) association
–can fail to detect an existing (true) association
•Non-differential Misclassification Bias
–rate of misclassification does NOT differbetween
study groups
–all study groups equally susceptible to inaccuracy in
classification
–dilutes the true association
–RR or OR shifts towards null (1.0)
Information bias: misclassification
Measurement error leads to assigning wrong
exposure or outcome category
Non-differential
•Random error
•Missclassifcation exposure
EQUAL
between cases and controls
•Missclassification outcome
EQUAL
between exposed & nonexp.
=> Weakensmeasure
of association
Differential
•Systematic error
•Missclassification exposure
DIFFERS
between cases and controls
•Missclassification outcome
DIFFERS
between exposed & nonexposed
=>Measure association
distortedin any direction
EXAMPLES OF MISCLASSIFICATION BIAS
•people who have disease (cases) classified as
controls
–due to inadequate description or criteria for what
constitutes disease
•EXAMPLE:
–GOAL: retrospective analysis of hypertension and stroke
–MISCLASSIFICATION SOURCE: hypertension diagnosis
–BIAS: in the 1960’s and 1970’s medical guidelines
diagnosed hypertension only when diastolic pressure
exceeded 100 therefore many individuals who, by
today’s standards were hypertensive, were
“misclassified”into control groups
EXAMPLES OF MISCLASSIFICATION BIAS
•people who have exposures classified as non-
exposed
–due to inadequate description or criteria for what
constitutes exposure
•EXAMPLE:
–GOAL: sugar consumption and dental caries
–MISCLASSIFICATION SOURCE: interview
–BIAS: study of dietary intake in humans finds that
subjects often disguise the actual quality and quantity of
food consumed, leading to misclassification of those
eating enough sugar to be classified as exposed but
putting them in the unexposed group based on interview
(Lissner et al., 1998)
Nondifferential misclassification
•Misclassification does not depend
on values of other variables
-Exposure classification NOT related to disease status
-Disease classification NOT related to exposure status
•Consequence
-if there is an association,
weakening of measure of association
“bias towards the null”
CONTROLLING FOR MISCLASSIFICATION BIAS
•improving sensitivity and specificity of
diagnostic tests
–raises or lowers the “cutoff point”for diagnosis
•increasing the completeness of medical records
•multiple questions that ask same information
–acts as a built in double-check
•multiple checks in medical records
–gathering diagnosis data from multiple sources
Bias in prospective cohort studies
•Loss to follow up
-The major source of bias in cohort studies
-Assume that all do / do not develop outcome?
•Ascertainment and interviewer bias
-Some concern: Knowing exposure may influence how
outcome determined
•Non-response, refusals
-Little concern: Bias arises only if related to both
exposure and outcome
•Recall bias
-No problem:Exposure determined at time of enrolment
123
Epidemiological association:
true or false?
Association?
Bias in selection andmeasurement?
Confounding?
Chance?
True association
present
absent
likely
likely
unlikely
present
absent
unlikely
False
association
Bias –Summary
•Can be prevented by design
•Can be estimated
•Cannot be overcome by analysis
•May be useful