Evaluation of health program.

powerpoint presentation regarding evaluation of health program

1. EVALUATION
OF HEALTH
PROGRAM
Dr Sumita Sharma
Post-graduate student
Department of Community Medicine
IMS and SUM Hospital, Bhubaneswar
1

2. EVALUATIONHEALTHPROGRAM
2

3. EVALUATION:
The collection and analysis of information to determine program performance,
i.e. “ finding out the value of something”
Evaluation of health program means a systematic and scientific process of determining the
extent to which an action or sets of objectives are achieved.
The process of evaluation involves 3 distinct stages –
vSelecting the attributes for evaluation
vSynthesizing of evidences
vFinal judgement
3

4. DIFFERENCE BETWEEN MONITORING AND EVALUATION
4

5. HEALTH:For every evaluation there is a need of statistics which is usually called as health statistics
It is an aggregated data describing and enumerating attributes, events, behavior, services, resources, outcomes or cost related to health, disease and health services
A state of complete
physical, mental and
social well-being and not
merely an absence of
disease or infirmity
5

6. PROGRAM:
“an organized response to eradicate or eliminate or reduce one or more problems where
the response includes one or more objectives and the expenditure of resources”
Program is a set of procedures to conduct an activity, e.g. control of malaria
Project Appraisal:
Analysis undertaken prior to project implementation to estimate net benefit in relation to costs
PROGRAM PROJECT
Programs will typically span multiple functional units
within an organisation
Projects are typically confined to a single functional
unit within an organisation
Programs are executed over a much longer timescale than
projects, often several years
Projects are typically of a shorter duration often just
a few weeks, and by definition have a finite duration
Programs have a wide scope, focussing on benefits, and
may have to change scope dramatically during their
execution to meet the changing needs of the organisation
The scope of projects is tight –they are limited to
producing deliverables
6

7. PURPOSE OF EVALUATION
vTo improve health plans, health programs, health infrastructure and health services delivery
vTo assess the need for funding or further coordination of fund
7

8. RESPONSIBILITY FOR EVALUATION
vThe planners
vAd hoc research groups
vThose responsible for health development
vThose responsible for implementation
vThe community itself, especially wherever primary
health programs are being carried out
8

9. TYPE OF EVALUATION
Based on the purpose, progress and
components and activities, evaluation can
be divided into:
Total Partial
Time related
a)Initial
b)Mid-term
c)Final/ terminal
In relation to planning process:
a)Baseline or bench mark evaluation
b)Appraisal evaluation
c)Concurrent evaluation
According to agency
a)Internal evaluation(sometimes called self
evaluation), in which people within a
program sponsor, conduct and control the
evaluation.
b) External evaluationin which someone from
Outside the program.
9

10. Relevance evaluation
Progress evaluation
Effective Evaluation
Impact Evaluation
Efficiency Evaluation
According to Dimensions:
10

11. Formative evaluation:
Evaluate the degree of need for the program, or the activities usedby the program to achieve
its desirable outcomes, but without evaluating the degree ofoutcome
Summativeevaluation:
Evaluation of the degree to which a program has achieved its desired outcomes, and the
degree to which any other outcomes (positive or negative) have resulted from theprogram
EVALUATION CAN BE FORMATIVE OR SUMMATIVE
11

12. EVALUATION CAN BE DESCRIPTIVE OR ANALYTICAL
12

13. EVALUATION CAN BE PROSPECTIVE OR RETROSPECTIVE
Prospective Evaluation: when a new program being introduced
Retrospective Evaluation: when programs have been functioning
for some time
13

14. 1. Review of records from National Health Information systems,
2. Monitoring: Input/efforts analysis, Management by Objectives, Gantt
Charts, Program Evaluation and Review Technique (PERT), and the Critical
Path Methods (CPM) of analysis,
3. Case Studies,
4. Qualitative studies,
5. Sample surveys: Cross-Sectional studies,
6. Cohort studies,
7. Panel studies.
8. Controlled experiments and intervention studies
TOOLS OF EVALUATION
14

15. A
Framework
for
Program
Evaluation
STEPS AND STANDARDS OF EVALUATION
15

16. FISHBONE DIAGRAM:ROOTCAUSEANALYSIS
16
“Root cause analysis is a structured team process that
assists in identifying underlying factors or causes of an
adverse event or near-miss”

17. RCA TOOLS ANDTECHNIQUES
17

18. DELAY FOROPDCONSULTATION
PEOPLEPROCESS
EQUIPMENTMANAGEMENT
Delay in MRDfile
Appointment system (only 10patients)
WrongreportingBusyschedule
ofdoctors
Communication
gap between
Doctor andStaff
OT
CasesRounds
EmergencyCases
Walk inpatients
Patients take time in filling
registrationform
Languageproblem
Breakdown ofequipment
HIS system isslow
Non availability ofpen
Unexpected leave byconsultant
Difficulty in taking lift and findingplace
Queue
system
not
followedNon
availabil
ity of
Queue
barriersHeight of the deskis
more
Delay in Registrationprocess
Doctors will be available in IMS instead of
HRC OPD(Sometimes)

19. All the steps and standards of evaluation help in fulfilling the major objective of any
health program.
MAJOR OBJECTIVES OF HEALTH PROGRAM:
vAccessibility
vEquitability
vQuality
vEffectiveness
vEfficiency
vSustainability
19

20. ACCESS:
Measure of the extent to which a population can reach the health services it
needs delivered by either the public or private sector.
vFinancial access-(also referred to as economic access) measures the extent
to which people are able to pay for health services.
vPhysical access-(also referred to as geographic access) measures the extent
to which health services are available and reachable.
20

21. EQUITY:
Gender
equalityEquityHuman
Rights
It is a normative issue that refers to fairness in the allocation of resources or the
treatment of outcomes among different individuals or groups
vHorizontal equity-“equal treatment of equal needs”
vVertical equity-concerned with the extent to which individuals with different
characteristics should be treated differently
21

22. QUALITY
Which is expected to maximize an inclusive measure of patient welfare.
Three aspects of quality ‘structural’, ‘process’ and‘outcome’
‘Structural’ quality refers to whether appropriateresources are in place to provide health care
of a minimum standard (personnel trained for their tasks, well maintained equipment and
buildings, a regular drugsupply)
‘Process’ quality generally refers to activities occurring during the interaction between the
health system and theclient (i.e. Whether goodquality Care is actually delivered).
‘Outcome’ quality in addition to health status, can include patient satisfaction and perceived
quality.
Outcome flows from process and refers to “changes in a patient’scurrent and future health
status that can be attributed to the antecedent healthcare.”
22

23. 23

24. Effectiveness
Effectiveness is the extent to which planned outcomes, goals, or objectives are
achieved as a result of an activity, strategy, intervention or initiative intended to
achieve the desired effect, under ordinarycircumstances
Efficiency
Refers to obtaining the best possible value for the resources from all stakeholders and sectors used
or using the least resources to obtain a certain outcome
-Technical efficiency(operational level , methods, manpower ,finance ,facilities etc)
-Cost efficiency(cost of vaccine, staff time, transportation, publicity is Rs.1000
a total of 100 children got vaccination, then cost efficiency ratio is Rs.10/Vac 24

25. We must understand what is cost and various types of costs involved in program
Cost :factor of production
It is a monetary valuation applied to an asset or service that has been obtained by an
expenditure of cash by a commitment.
Cost-Direct
Indirect Cost
Fixed Cost
Incremental Cost
Cost Operating
Opportunity Cost
Production Cost
Recurrent Cost
Cost effective Analysis
Cost-benefit Analysis
Cost-utility Analysis
25

26. SUSTAINABILITY
Capacity of the system to continue its normal activities well into the future
vFinancial sustainability is the capacity of the health system to maintain an
adequate level of funding to continue its activities
vInstitutional sustainability refers to the capacity of the system, if suitably
financed to assemble and manage the necessary resources to successfully carry
on its normal activities in the future.
26

27. LOGIC MODEL OF PROGRAM EVALUATION
PROCESS
Short intermediate long
OUTCOMES
Inputs
ASSUMPTIONS/ CONTEXTUAL FACTORS
ActivitiesOutputs
27

28. EXAMPLE: EVALUATION OF NPCDCS
28

29. Inputs: resources that go into a program or intervention—what we invest.
They include financial, personnel, and in-kind resources from any source.
e.g. Various funding sources for program
Your partners
Staff time
Technical assistance
Activities:events undertaken by the program or partners to produce desired outcomes—
what we do, e.g. Create a state-level partnership
Train health care partners
Staff in clinical guidelines
Develop a community health communication campaign on signs and symptoms of
stroke, and to call 9-1-1.
29

30. Outputs are the direct, tangible results of activities—what we get.
These early work products often serve as documentation of progress.
e.g. State-level partnerships created
Health care professionals trained in clinical guidelines
Community health communication campaigns developed.
Outcomes are the desired results of the program—what we achieve.
Describing outcomes as short, intermediate, or long term depends on the objective, the length
of the program, and expectations of the program or intervention.
Short-term outcomes are the immediate effects of the program or intervention activities.
They often focus on the knowledge and attitudes of the intended audience
e.g. Increase partner knowledge of HDSP priorities and strategies
Increase physician knowledge of clinical guidelines
Increase knowledge of signs and symptoms of stroke and of the need to call 9-1-1.
30

31. Intermediate outcomes are behaviour, normative, and policy changes.
e.g. HDSP State Plan has been developed and published with partner involvement.
Health systems implement clinical guideline
Decrease transport time to treatment for stroke victims.
Long-term outcomes refer to the desired results of the program and can take years to accomplish.
e.g. Increase in state wide policy and environmental strategies for HDSP.
Increase in blood pressure control in a health centre population.
Increase in early treatment for stroke.
Impacts refer to the ultimate impacts of the program. Take 10 or more years to achieve.
e.g. Decrease in the rate of death due to heart disease.
Eliminate disparities in treatment for stroke between general and priority populations.
31

32. Assumptions are the beliefs we have about the program or intervention and the resources involved.
Assumptions include the way we think the program will work—the "theory" we have used to develop
the program or intervention.
Examples • Funding will be secure throughout the course of the project.
• Because we teach information, it will be adopted and used in the way we intended.
• Professionals will be motivated to attend learning sessions.
• External funds and well-placed change agents can facilitate institutional change.
• Staff with the necessary skills and abilities can be recruited and hired.
• Partnerships or coalitions can effectively address problems or reach into areas we cannot.
Contextual Factors describe the environment in which the program exists and external factors that
interact with and influence the program or intervention. These factors may influence implementation,
participation, and the achievement of outcomes. Contextual factors are the conditions over which we
have little or no control that affect success.
32

33. Is the program successful?
33

34. SWOT
ANALYSIS
34

35. CompetitiveAdvantagesGrowthGoals
StrengthOpportunities
Area ofImprovementWatchItems
WeaknessesChallenges
35

36. STRENGTHS
vTeam of trainedand qualifiedtrainers.
vInnovation andexecution
vJustifying support
function for all field
WEAKNESSES
vScattered focus/
involved in several
unrelatedactivities.
vShortage ofAptitude
trainer.
vShortage of
communication
vPoor infrastructureOPPORTUNITIES
Government tie up
with
NGOs for
evaluation and
implementation of
health program
CHALLENGES
vHave the chance
of being overcome
vHow program
need to be handled
appropriately
36

37. THANK YOU
37