Analytic Frameworks

Understanding the use and application of analytic framework evidence models within systematic medical/health reviews for outcomes based research.

1. Analytic Frameworks
Prepared for:
Agency for Healthcare Research and Quality (AHRQ)
Training Modules for Systematic Reviews Methods Guide
www.ahrq.gov

2. Systematic Review Process Overview

3. To understand the use of analytic frameworks
within systematic reviews
To develop an understanding of principles for
developing analytic frameworks
To understand which elements of a review can
be represented in an analytic framework
Learning Objectives

4. A type of evidence model that links and defines
clinical concepts, evidence, and populations as
they relate to outcomes
Sometimes referred to as causal pathways
Alternative and related concepts are:
Conceptual frameworks
Influence diagrams
Theoretical frameworks
Logic models
What Is an Analytic Framework?

5. To provide clinical, biological, or health services
underpinnings for the mechanism of action
To clarify implicit assumptions
To identify logical flaws as key questions are
developed
To provide a starting point for discussions with
key informants
To identify distinct bodies of evidence to be
included in the review
Why Use Analytic Frameworks?

6. Specifies populations, interventions, and
outcomes, and sometimes comparators, timing,
and settings
Identifies potential modifiers and mediators of
effectiveness
Clarifies links between intermediate and health
outcomes
The Importance of an Analytic Framework

7. Complex bodies of literature with multiple key
questions:
Analytic frameworks help define key questions, and thus direct
specific literature searches.
Complex chain of logic between intermediate
and clinical outcomes:
Analytic frameworks help users understand decisional context.
When To Use Analytic Frameworks

8. Population of interest
Intervention
Linkages that demonstrate key questions
Intermediate outcomes
Ultimate health outcomes (including harms)
Components

9. The population, intervention, and outcomes should
be clearly identified in the analytic framework.
All key questions should be represented clearly on
the framework.
Contextual information about modifiers and
mediators may be appropriate.
Analytic frameworks are drafted with the purpose of
clarifying complex questions and will need iterative
revisions through the topic refinement process (see
the module, “Topic Refinement”).
Revising the Framework

10. Depicting Essential Components: An Example
Graphical chain of logic
Arrows
Linkages
Preventive service or
treatment
Questions
Dotted lines
Associations
Rectangles
Intermediate outcomes
Rounded corners
Health states
Square corners
Curved arrows
Lead to ovals
Harms
Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35; Woolf SH, et al. Annu Rev Public Health 1996;17:511-38.

11. Key Research Questions
The numbers that represent the key questions should be
placed in the relevant position within the framework.
11
22
33
44
Whitlock EP, et al. Am J Prev Med 2002;22:267-84.

12. Sample Working Framework
Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35.

13. Sample Screening and Treatment Framework
Hartmann KE, et. Evidence Report/Technology Assessment No. 187. Available at:
http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.

PopulationPopulation InterventionInterventionOutcomesOutcomes

14. Capturing the Context
Analytic
frameworks
can provide a
means of
capturing
contextual
issues
important to a
review
Vanderbilt University
Evidence-based Practice
Center. Systematic review
protocol: traumatic brain
injury and depression.
Available at:
http://www.effectivehealth
care.ahrq.gov/ehc/products
/77/367/TBI%20and
%20Depression
%20(2-9-2010).pdf.

15. Do not build the framework and forget it
Use the framework throughout the review
process to:
Revisit inclusion/exclusion criteria
Keep a handle on scope
Reconsider the appropriateness of key questions
Guide interviews with key informants
Let the framework help in structuring the report
and results
Using the Framework

16. Analytic frameworks help reviewers clarify
specific key questions and direct literature
searches of complex bodies of literature.
Analytic frameworks help end-users understand
the decisional context of key questions when
there is a complex chain of logic.
Analytic frameworks can help the iterative
process of identifying specific researchable key
questions in the context of complex clinical or
policy issues.
Key Messages

17. Agency for Healthcare Research and Quality. Topic development. In: Methods
reference guide for effectiveness and comparative effectiveness reviews.
Version 1.0 [Draft posted Oct. 2007], Chapter 2, Finding evidence. Rockville,
MD: Agency for Healthcare Research and Quality; Draft Posted October 2007.
p. 10-15. Available at:
http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pd
f.
Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic
reviews: Synthesis of topics related to the delivery, organization, and
financing of health care. Ann Intern Med 2005;142(Pt 2):1056-65.
Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive
Services Task Force: a review of the process. Am J Prev Med
2001;20(Suppl):21-35.
Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive
Bladder in Women. Evidence Report/Technology Assessment No. 187
(Prepared by Vanderbilt University Evidence-based Practice Center under
Contract No. 290-2007-10065-I). Rockville, MD: Agency for Healthcare
Research and Quality, August 2009. AHRQ Publication No. 09-E017. Available
at:
http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.
References (I)

18. Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of
evidence in systematic reviews. Ann Intern Med 1997;127:989-95.
Vanderbilt University Evidence-based Practice Center. Systematic review
protocol: traumatic brain injury and depression. Rockville, MD: Agency for
Healthcare Research and Quality, Posted December 9, 2009. Available at:
http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and
%20Depression%20(2-9-2010).pdf.
Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral
counseling interventions: an evidence-based approach. Am J Prev Med
2002;22:267-84.
Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical
practice guidelines: lessons learned by the US Preventive Services Task
Force. Annu Rev Public Health 1996;17:511-38.
References (II)

19. This presentation was prepared by Melissa L.
McPheeters, Ph.D., M.P.H., a member of the
Vanderbilt University Evidence-based Practice
Center.
The module is based on an update of chapter 2 in
version 1.0 of the Methods Guide for Comparative
Effectiveness Reviews (available at:
http://www.effectivehealthcare.ahrq.gov/ehc/p
roducts/60/294/2009_0805_principles1.pdf) .
Author