Validity versus Reliability (Comparisons and Contrasts)

Validity versus Reliability discusses the differences between these two subjects.

1. Reliability & Validity
Reliability
“dependability”
is the indicator consistent?
same result every time?
Does not necessary measure what you think it measures
(ex. May consistently measure something other than
the concept)
Validity
measurement validity -how well the conceptual and
operational definitions mesh with each other
does measurement tool measure what we think ?

2. Relationship between Measurement
Reliability and Validity
reliability necessary for validity but does not
guarantee it (“necessary but not sufficient”)
measure can be reliable but invalid

3. Examples of Types of
Reliability
stability
over time
representative
across different subgroups of a population
(ex. of problems: young people may exaggerate
their ages, older people may reduce theirs)
equivalence
Comparable results from multiple indicators
intercoder reliability

4. Improving Reliability
clearly conceptualize constructs
increase level of measurement
use pretests, pilot studies
e.g. use multiple indicators :
Dependent
Variable Measure
Independent
Variable Measure
Empirical
Association?
a2 a3a1 b1 b2
A B
Specific IndicatorsSpecific Indicators
Measurement Using
Multiple Indicators
Neuman (2000: 167)

5. Selected Types of Validity:
Face Validity
judgement by the scientific community that indicator
measures the concept
Construct/concept
Measure
Scientific Community

6. Content Validity
measure represents all the aspects of conceptual definition of
construct.
judgement concerning how adequately a test samples behavior
representative of the universe of behavior the test was
designed to sample.
Love
Beliefs
& Values

7. Criterion
The validity of indicator is verified by comparing it with
another measure of the same construct
Predictive
Relies on occurrence of future event or behavior for
verification of indicator
Concurrent
relies on pre-existing accepted measure to verify
indicator
Construct Validity
A type of measurement validity that uses multiple indicators
convergent and discriminate
Some types of Measurement Validity

8. Other Dimensions
of Validity
Internal Validity
no error of logic internal to research design
External Validity
results can be generalized
Statistical validity
correct statistical methodology chosen ?
assumptions fully met

9. More terms & concepts for assessing
validity (usually in experimental research)
Threats to Internal Validity
Selection bias
History effects (something in context changes)
Maturation
Testing effect
Instrumentation
Mortality
Statistical regression
Contamination
Compensatory behaviour
Experimenter expectanc
Threats to External Validity
Realism
Reactivity

10. Another Aspect of Measurement
Design: Levels of Measurement
1-Nominal (ex. Mother tongue)
different categories (names, labels, images)
not ranked
2-Ordinal (county fair prizewinners ranked by first, second & third prize)
different categories
rank-ordered
attributes indicate relatively more or less of that variable
distance between the attributes of a variable is imprecise

11. Levels of Measurement (cont’d)
3-Interval Measures (age measured by 5 year age groups)
different categories
ranked in order
Can tell amount of difference between categories
Usually no true zero
4-Ratio Measures (age measured by date of birth)
different categories
ranked in order
amount of difference between categories
also possible to state proportion (have a true zero)
Relations between levels --can collapse from
higher into lower, not vice versa

12. Babbie (1995: 101)
The Research
Process