Presentation # 3 - Measurements, Validity, and Reliability.pptx

Evidence Based Medicine. Measurement scales, validity of studies.

1. Evidence-based Medicine I Variables, Measurement Scales, Validity and Reliability in Research

2. Learning Objectives By the end of this lesson, students should be able to: Identify different types of variables (categorical, discrete, continuous) and classify them according to the appropriate measurement scale (nominal, ordinal, interval, ratio). Define key concepts related to measurement, including validity, precision, and accuracy. Differentiate between precision and accuracy, explaining how random and systematic errors affect each. Identify sources of measurement errors, including observer, instrument, and subject variability. Explain the function of random error and describe how it affects precision in repeated measurements.

3. Learning Objectives Explain the function of systematic error and describe how it affects accuracy, using examples like observer bias and instrument bias. Compare and contrast normal vs. non-normal distributions, identifying their characteristics and impact on data interpretation. Apply the 68-95-99.7 rule to interpret data dispersion within a normal distribution. Describe measures of central tendency (mean, median, mode) and dispersion (range, variance, standard deviation) for continuous variables. Explain the importance of using validated measurement tools and questionnaires, discussing their advantages and limitations. Evaluate measurement strategies to minimize errors, including calibration, standardization, training, blinding, and automation.

4. Variables: The Building Blocks of Measurement Definition : Any characteristic of persons or things that takes on different values in research studies ( not constant ).

5. Classification of Variables Independent, manipulated, cause variable Dependent, measured, effect variable https://www.geeksforgeeks.org/dependent-and-independent-variable/

6. Types of Variables Categorical (Qualitative) Numerical (Quantitative)

7. Types of Variables Categorical Variables (Qualitative) Nominal: Unordered categories with no ranking (e.g., race, blood type, medical specialty). Dichotomous (binary): A special case of nominal variables with only two categories (e.g., alive/dead, yes/no, pass/fail). Ordinal: Ordered categories without quantifiable or equal intervals (e.g., pain level, letter grade*, education level). * see notes

8. Types of Variables Numerical variables (quantitative) Discrete : Countable numbers with no intermediate values (e.g., Number of students at FIU, Covid 19 cases, # of cigarettes smoked per day). Continuous : Measured values that can take any number within a range including fractions (e.g., Test score, weight, height, temperature). Typically measured with an instrument that allows infinite precision (e.g., scale, ruler).

9. Types of Variables Note: variable type is important to select appropriate statistics and analytical techniques. 9

10. Measurement Scales 10 Scale Characteristics Examples Nominal Classify into categories ; categories are given names or numbers , but the numbers are arbitrary .  Gender, nationality, religious or political affiliation, blood type Ordinal Rank‑ordered according to relative size or position ( without consistent intervals ). Letter grades, class rank (e.g., freshman, sophomore, junior), SES (e.g., low, middle, high), faculty rank grade level Interval Rank‑ordered and equal differences between values imply equal distances in the attribute ( not true zero ). Temperature on Fahrenheit scale, most assessment devices (e.g., cognitive or psychological tests) Ratio Rank‑ordered, equal intervals, true zero allows ratios to be formed. Speed, height, age, elapsed time, heart rate

11. Descriptive Statistics Descriptive statistics help summarize and interpret data. For categorical variables we may use frequency, cumulative frequency, relative frequency, cumulative relative frequency For continuous variables we may use: Measures of central tendency (mean, median, mode) Measures of dispersion (range, variance, standard deviation)

12. Frequency The count or number of observations within a category. Example : If surveying 100 people about blood type: A: 40 B: 30 O: 20 AB: 10

13. Cumulative Frequency The sum of frequencies up to a certain category. Example : Cumulative frequency of blood types (A → AB): A: 40 A + B: 40 + 30 = 70 A + B + O: 70 + 20 = 90 A + B + O + AB: 90 + 10 = 100

14. Relative Frequency The proportion of observations in a category compared to the total. Example : Blood type A : 40/100 = 0.40 (40%)

15. Cumulative Relative Frequency The sum of relative frequencies up to a certain category. Example : A: 40% A + B: 40% + 30% = 70% A + B + O: 70% + 20% = 90% A + B + O + AB: 90% + 10% = 100%

16. Measures of Central Tendency Mean (arithmetic average): The sum of all values divided by the number of observations. Example : Heights of 5 people: 150, 160, 170, 180, 190 cm Mean = 150+160+170+180+190/5 = 170 cm Median: The middle value when data is ordered from smallest to largest. Example : Data: 150, 160, 170, 180, 190 Median = 170 (middle value) Mode: The most frequently occurring value in a dataset. Example : Data: 150, 160, 170, 170, 180, 190 Mode = 170 (appears twice)

17. Measures of Dispersion (Spread of Data) Dispersion measures how spread out the values in a dataset are. Range: The difference between the highest and lowest value. Formula: Range = Maximum value − Minimum value Example : Heights: 150, 160, 170, 180, 190 Range = 190 - 150 = 40 cm Variance: Measures how far each value is from the mean. Standard deviation (SD): Square root of variance; represents average distance from the mean . Example : Variance = 200 SD ≈14.14

18. The Normal Distribution A normal distribution (bell curve) is symmetrical, with most values clustering around the mean.

19. The Normal Distribution Human height: Mean = 170 cm, SD = 10 cm. 68% of people have heights between 160-180 cm. 95% of people have heights between 150-190 cm. 99.7% of people have heights between 140-200 cm.

20. Non-normal Distribution A non-normal distribution does not follow this pattern and may be skewed, have multiple peaks, or be heavily tailed.

21. Precision and Accuracy Precision (reproducibility of a measurement). Accuracy (how close a measurement is to the true value).

22. Concept Definition Example Precision Reproducibility of results A scale gives 72.1 kg every time, but the actual weight is 70 kg (consistent but incorrect). Accuracy Closeness to the true value A scale shows 70 kg when the true weight is 70 kg (correct but may vary slightly). Both High accuracy and high precision A scale consistently shows 70 kg when the true weight is 70 kg.

23. Measurement Errors Random errors arise from variability in measurements and affect precision . Results fluctuate (move in different directions). Best assessed by repeated measurements. Sources of random error Observer variability (e.g., Skill using an instrument) Instrument variability (e.g., Aging mechanical components) Subject variability (e.g., Physiological differences) Increasing sample size helps reduce random error.

24. Type of Random Error Cause Example Observer Variability Human error Different ways of using BP cuff Instrument Variability Equipment inconsistencies Aging scale shows slight changes Subject Variability Natural fluctuations BP changes based on stress/time

25. Measurement Errors Systematic errors introduce consistent bias (measurement is consistently incorrect in the same direction) and affect accuracy. Best assessed by comparing to a "gold standard.“ Sources of systematic error Information bias (Errors in how data was collection, recorded, or reported). Observer bias (e.g., Leading questions; subconsciously recording lower BP values in medication research) Instrument bias (e.g., Uncalibrated scale) Subject bias (e.g., Recall bias) Blinding helps reduce systematic errors (not by increasing sample size).

26. Type of Systematic Error Cause Example Information Bias Errors in data collection Patients underreport smoking habits in a study. Observer Bias Researcher influences results Doctor records lower BP for patients on treatment. Instrument Bias Faulty equipment An uncalibrated scale consistently adds 2 kg. Subject Bias Participant misreports data A patient forgets how many cigarettes they smoked last week (recall bias).

27. Strategies to Reduce Measurement Errors Standardizing measurement methods Training and certifying observers Refining and automating instruments Repeating measurements (random error) Blinding (systematic error) Making unobtrusive measurements (systematic error) Calibrating instruments (systematic error)

28. Validity Definition: how well a measurement represents the phenomenon of interest (whether a measurement truly reflects what it is supposed to measure). Example: kidney function measurement Creatinine vs. Cystatin C (creatinine influenced by muscle mass) Using validated instruments/questionnaires improves consistency with previous research. Advantage: comparable results with previous studies Disadvantage: may be outdated or suboptimal

29. Concept Explanation Example Validity Measures what it is supposed to measure A thermometer for temperature, not weight Example: Kidney Function Creatinine is affected by muscle mass, Cystatin C is not Cystatin C is a more valid biomarker Using Validated Instruments Ensures reliable and comparable data PHQ-9 for depression screening Advantage Results are consistent with previous studies Easier to compare findings Disadvantage Tools may become outdated or less precise Newer, better diagnostic tests may exist

30. Final Thoughts

31. Application Identify independent and dependent variables from assigned papers Identify measurement scales used Describe reported descriptive statistics Identify sources of measurement error and categorize them as random or systematic