An Introduction Patient Reported Outcome Measures (PROMS)

An introduction to the key concepts of patient Reported Outcome Measures, including reliability and validity, generic versus disease specific,selection criteria and their adaptation for different cultural groups.

1. Copyright DHP Research & Consultancy Ltd 2010
An introduction to Patient
Reported Outcome Measures
(PROM’s)
Dr Keith Meadows
DHP Research & Consultancy Ltd
July 2010

2. Copyright DHP Research & Consultancy Ltd 2010
Overview
•PROM’s What are they and what are they
used for?
•Generic and condition specific - strengths
and weaknesses
•What should we look for when choosing a
PROM
•Adapting PROM’s for cultural and ethnic
groups
•Interpreting PROM data

3. Copyright DHP Research & Consultancy Ltd 2010
Definitions
•Patient reported outcomes (PROM’s)
are outcomes known only to the patient
•Patient reported outcome measures
(PROM’s) are tools we use to measure
patient outcomes

4. Copyright DHP Research & Consultancy Ltd 2010
Variations on a theme
•Health status
•Health-related quality of life (HRQoL)
•Well-being
•Health outcomes
•Quality of life
•Satisfaction

5. Copyright DHP Research & Consultancy Ltd 2010
But all are based on…
Self-assessment

6. Copyright DHP Research & Consultancy Ltd 2010
Common content e.g. SF-36
•Physical functioning
•Role limitations due to physical health
•Bodily pain
•General health perceptions
•Vitality
•Social functioning
•Role limitations due to emotional problems
•Mental health

7. Copyright DHP Research & Consultancy Ltd 2010
The underlying principle
To measure across the complete
continuum of a given construct
No pain Severe pain

8. Copyright DHP Research & Consultancy Ltd 2010
Something for everyone
•Patient choice
•Audit
•Quality improvement (Clinician & Provider)
•Research
•Training
•After Black N. 2008

9. Copyright DHP Research & Consultancy Ltd 2010
General applications
•Measurement of the patient’s health status or
health-related quality of life (HRQoL) at a single
point in time
•PROMs are used to derive measures of the
outcomes of specific interventions.
•Changes in health status at two different points in
time (e.g. before and after an operation) can be
used to derive a measure of the impact of health
care interventions.
•Certain PROMs suitable for purposes of economic
evaluation (e.g., estimation of quality-adjusted
life years – QALYs)

10. Copyright DHP Research & Consultancy Ltd 2010
Some specific applications
•Personalised care planning
•Self-assessment
•Annual review
•Informed decision making

•Population health

11. Copyright DHP Research & Consultancy Ltd 2010
Why consider the patient’s
perspective?
If quality is to be at the heart of everything we do , it must be
understood from the perspective of the patient’
‘Just as important (as clinical measures) is the
effectiveness of care from the patient’s own
perspective which will be measured through
patient-reported outcome measures’
Next Stage Review 2008

12. Copyright DHP Research & Consultancy Ltd 2010
And also…
•Patients know better – ‘We know little of
the clinical outcomes of NHS services from
the patient’s perspective. PROM’s fill this
gap’ DH 2007
•Clinical outcomes not always related to
how the patient feels
•Patients like to be asked

13. Copyright DHP Research & Consultancy Ltd 2010
What can PROM’s tell us?
•Which is the best treatment for the condition?
•Is one subgroup of the population sicker than the
others?
•Is an individual patient’s condition getting better?
After Coulter A 2008

14. Copyright DHP Research & Consultancy Ltd 2010
What should we look for when
choosing a PROM?
…evidence that it is a measure of what it is
supposed to measure

15. Copyright DHP Research & Consultancy Ltd 2010
….across the complete continuum
of a given construct
No limitations Mobility Severe limitations

16. Copyright DHP Research & Consultancy Ltd 2010
….and locating individuals by
providing meaningful scores on
that continuum
Very anxious anxiety No anxiety

17. Copyright DHP Research & Consultancy Ltd 2010
Therefore…
•It is important we choose the right PROM
for our clinical practice, patients, study,
trial
•Check the information we get and don’t
get from a PROM

18. Copyright DHP Research & Consultancy Ltd 2010
The need for a conceptual
model
•A diagram of proposed causal linkages among a
set of concepts believed to be related to a
particular disease (Earp & Enmett 1991).
•A taxonomy of patient outcomes according to the
underlying health concepts they represent and
proposes specific and causal relationships
between different health concepts (Wilson &
Cleary 1995)

19. Copyright DHP Research & Consultancy Ltd 2010
Diabetes Health Profile Conceptual Model
© DHP Research & Consultancy Limited 2010© DHP Research & Consultancy Limited 2010
Impact
Management Treatment Symptoms Diet Complications
Diabetes
Emotional Behavioural
Anxiety Mood
Limitations in
Social/work
functioning
Eating
behaviour
DHP

20. Copyright DHP Research & Consultancy Ltd 2010
Development based on….
•Literature review
•In-depth interviews with patients/clinicians
family etc.
•Thematic analysis of qualitative data

21. Copyright DHP Research & Consultancy Ltd 2010
Conceptual framework
A representation of the expected relationship of
items within a domain within a PROM concept
Item A
Item B Domain score 1
Item C

Item C
Item E Domain score 2
Item F

22. Copyright DHP Research & Consultancy Ltd 2010
We therefore must look for
evidence of…
•Validity
•Reliability
•Responsiveness
•Precision

23. Copyright DHP Research & Consultancy Ltd 2010
Validity
•Does it measure what it is meant to?
-Content validity - does the content reflect the
concept/construct measured, is the content representative?
-Face validity - do the individual items look as if they are
measuring what they should?
-Criterion validity - can the construct be measured
accurately?
-Construct validity - Is there a conceptual model or
theoretical underpinning?

24. Copyright DHP Research & Consultancy Ltd 2010
Reliability

Are the results stable over time when
applied to the same people at different
time periods? (Test-retest reliability)

25. Copyright DHP Research & Consultancy Ltd 2010
Precision
Does the measure discriminate between
different patient groups, health states,
treatments etc?

26. Copyright DHP Research & Consultancy Ltd 2010
Responsiveness
Is the measure responsive to change when
change is present?

27. Copyright DHP Research & Consultancy Ltd 2010
Anything else?
- Acceptability – will people fill it in?
-Response rates
-Item completion rates
-Missing cases

28. Copyright DHP Research & Consultancy Ltd 2010
Feasibility – how easy will it be to
use?
-Cost
-Time
-Ease of scoring
-Interpreting scores
-Supporting documentation (Manual, norm-
reference scoring etc.)

29. Copyright DHP Research & Consultancy Ltd 2010
Does the name of the PROM
reflect what is being
measured?
•Not all PROM’s have a conceptual
model/theoretical underpinning
•Some PROM’s are named by the items
that are grouped together

30. Copyright DHP Research & Consultancy Ltd 2010
Implications of choosing the
wrong PROM
•Fail to identify significant outcomes
•Mislead clinical practice
•Misrepresent treatment and disease
impact
•After Cano S. 2008

31. Copyright DHP Research & Consultancy Ltd 2010
Generic and condition specific -
strengths and weaknesses
Generic
•Suitable for the general
population
•Comparisons with other
conditions/disease groups
•Content may be redundant
for certain
condition/illnesses
•Not sensitive to detecting
disease-specific issues
Condition-specific
•Specific to disease group
•Sensitive to detecting
clinically significant
changes
•Content relevant to target
group
•Cannot compare with
general population

32. Copyright DHP Research & Consultancy Ltd 2010
Disease-specific or generic?
‘Go for a combined approach?’

33. Copyright DHP Research & Consultancy Ltd 2010
Index v Multidimensional
•Overall score (but can
be graded)
•Less information
•?Easier to score
•Appropriate for cost-
benefit analysis
•Provides a profile
•Reflects the
important/different
components of the
illness
•Provides more
information
•Can be long

34. Copyright DHP Research & Consultancy Ltd 2010
Approaches to interpreting
PROM data
•Minimally Important Difference (MID) to change
•Known groups
•Response to treatment
•Normative and reference groups
•Statistical significance
•Effect size

35. Copyright DHP Research & Consultancy Ltd 2010
Effect size
•Effect size is a simple way of quantifying the
difference between two groups that has many
advantages over the use of tests of statistical
significance alone. Effect size emphasises the size
of the difference rather than confounding this
with sample size.
≥ 0.8Large
0.5 – 0.7Moderate
0.2 - 0.4Small

36. Copyright DHP Research & Consultancy Ltd 2010
Using PROM’s for cultural and
ethnic groups
•Technical problems
•Conceptual problems
•Linguistic problems

37. Copyright DHP Research & Consultancy Ltd 2010
What is our aim?
To achieve cross-cultural equivalence:
•Semantic equivalence – equivalence in meaning
of words
•Conceptual equivalence – validity of the concepts
in the target language
•Idiomatic equivalence – equivalent
idioms/expressions in target language
•Experiential equivalence – situations should fit
target language

38. Copyright DHP Research & Consultancy Ltd 2010
Overview
•PROM’s must be fit for purpose
•Are they valid and reliable with a clear
conceptual/theoretical underpinning
•Don’t choose PROM’s on basis of popularity alone
•Take an evidence-base approach in selecting the PROM
•Be sure what you want to measure
•Don’t assume the PROM is accurate in telling you what it
measures
•Translation alone does not result in cultural equivalence

39. Copyright DHP Research & Consultancy Ltd 2010
Overview cont’d
•Statistical significance does not mean clinical significance
•p values can be misleading (large samples can result in
high p values)

40. Copyright DHP Research & Consultancy Ltd 2010
This presentation is a selection of slides taken from our
training course ’Patient Reported Outcome Measures
(PROMs) in Clinical Trials and Health Care – An
introduction.
If you would like more information on our training
courses and or the Diabetes Health Profile email:
[email protected]
Visit our website www.dhpresearch.com