The use of ‘colloquial evidence’ in HTA: the experience of NICE

The use of ‘colloquial evidence’ in HTA: �the experience of NICE�

1. The use of ‘colloquial evidence’ in
HTA:
the experience of NICE
Health Technology Assessment international (HTAi) Annual Meeting
26 June 2012: 17:30-19:00
(Abstract ID: 381)
Presenter: Thomas Wilkinson
Authors: Tarang Sharma, Bindweep Kaur, Moni Choudhury, Bhash
Naidoo, Sarah Garner, Lucy Connor (NICE)

2. Evidence for developing guidance
•NICE produces guidance
on best available evidence.
•Evidence:
–“findings from research
and other knowledge
that may serve as a
useful basis for decision
making in public health
and health care.”
(Health evidence
network, WHO in
Lomas et al, 2005)

3. Colloquial evidence
•Definition:
“anything that establishes a
fact or gives reason for
believing something.”
(Oxford Dictionary, 1980)
•Provides context and
complements scientific
evidence
•Increasingly used in HTA
•Controversial
Adapted from Lomas et
al (2005)

4. Aims and objectives
•Aim:
•To explore the use of colloquial evidence in the
process of developing a guidance across all
guidance-producing centres at NICE.
•Objectives:
–To understand the context of colloquial evidence
from the available literature.
–To identify consistencies and variations in the use of
colloquial evidence across all centres at NICE.
–To map the use of colloquial evidence at NICE to the
conceptual framework outlined by Lomas et al 2005.

5. Methods

6. Results: Mapping colloquial
evidence at NICE
•All guidance-producing centres utilise colloquial
evidence with scientific evidence, as per Lomas et al
2005.

7. Results: Colloquial evidence is utilised
at all stages of guidance production
Expert/
Patient/Carer
input
Grey
Literature
Stakeholder
views

8. E.g. CG105 Motor Neurone Disease - Information and
support needs of patients with motor neurone disease
and their families and carers
•Systematic review searches
identified 11 eligible studies
and a set of interviews from
health talk online.
•All were critically appraised via
qualitative studies checklist
and analysed through a
‘Thematic-Conceptual Meta
Matrix’ (Miles and Rosenblum
1987).
•Example of one of the
evidence profiles looking at
one outcome (timing, level of
information and ways of
communication).
NICE clinical guideline 105 – Non-invasive ventilation for motor neurone disease:
http://publications.nice.org.uk/motor-neurone-disease-cg105/other-versions-of-this-guideline#quick-reference-guide

9. Discussion and Conclusion
•Colloquial Evidence is utilised alongside scientific
evidence across NICE.
•Methodology for integrating all kinds of evidence:
–Deliberative process: “an integration of technical
analysis and stakeholder and lay public
deliberation, contrasting with the traditional ‘top-
down’ or ‘bureaucratic rationalistic’ policy
orientation” (Culyer et al, 2006)
–Lack of methodology?
–MCDA approach?

10. Next Steps
•No formal tools for critically appraising colloquial evidence
utilised across NICE:
–Proposing SART Model
•Source
•Accuracy
•Relevance
•Timeliness
•Following changes may impact the way Colloquial Evidence is
utilised currently at NICE:
–Methods and Process Manuals Update
–Changing NICE’s remit – introduction of social care
guidance
–Value based pricing
•Developing further methods of identifying and capturing
colloquial evidence.

11. References
•Culyer AJ, Lomas J (2006) ‘Deliberative processes and evidence informed decision making in
healthcare: do they work and how might we know?’ Evidence and Policy Vol. 2(3) pp 357-71.
•Lomas J, Culyer T, McCutcheon C et al. (2005) Conceptualizing and combining evidence for
health system guidance: final report. Ottawa: Canadian Health Services Research Foundation
•Reay CA, Colechin ES, Bousfield DR et al (2011) ‘Review of published literature relating to
methods for identifying, synthesis and integration of colloquial evidence’. Newcastle upon Tyne
Hospitals NHS foundation trust
•20 National Institute for Health and Clinical Excellence (2009). Centre for Clinical Practice. The
guidelines manual 2009 Available at:
http://www.nice.org.uk/aboutnice/howwework/developingniceclinicalguidelines/clinicalguidelinedevelopmentmethods/GuidelinesManual2009.jsp?domedia=1&mid=5F238D80-19B9-E0B5-D4CB1191544B5D45
•National Institute for Health and Clinical Excellence (2009) Centre for Public Health Excellence.
Methods for development of NICE public health guidance 2009 Available at:
http://www.nice.org.uk/aboutnice/howwework/developingnicepublichealthguidance/publichealth
guidanceprocessandmethodguides/public_health_guidance_process_and_method_guides.jsp?
domedia=1&mid=F6A97CF4-19B9-E0B5-D42B4018AE84DD51
•National Institute for Health and Clinical Excellence (2008) Implementation support process
manual Available at:
http://www.nice.org.uk/usingguidance/niceimplementationprogramme/nice_implementation_pro
gramme.jsp?domedia=1&mid=656BA97E-19B9-E0B5-D437A317C324122F
•National Institute for Health and Clinical Excellence (2011) Citizen’s Council
http://www.nice.org.uk/aboutnice/howwework/citizenscouncil/citizens_council.jsp

12. Links/Contact details
Links:
Research and Development, NICE:
http://www.nice.org.uk/aboutnice/howwework/researchanddevelopment/about.jsp
Authors/Contributors:
•Ms Tarang Sharma, Senior Analyst
•Ms Bindweep Kaur, Analyst
•Miss Moni Choudhury, Analyst
•Dr Bhash Naidoo, Associate Director
•Dr Sarah Garner, Associate Director
•Ms Lucy Connor, Programme Manager
Contact:
•Sarah Garner Tel: [phone];
•Email: [email protected]