Evaluation[1].pdf from planning and evaluation PHD

evaluation from public health dentistry

1. EVALUATION
Presented by: Bhakti Pattewar

2. INTRODUCTION
01# Evaluation measure the degree to which objective and targets
are fulfilled and the quality of the results obtained.
# It measures the productivity of available resources in achieving
clearly defined objectives.
# It measures how much output or cost effectiveness is achieved.
# It makes possible the reallocation of priorities and of resources
on the basis of changing health.
-by WHO 1967

3. Evaluation is defined as the collection and analysis of
information to determine program performance
concerned- final outcome and factor related to
02
Determine the value of program
and to make sure the program was
carried out as thought.
Whether all the objectives and
performance was fulfilled. Final outcome
Direct result- assess the
changes made due to
program.
Indirect result- it is not
practicable to measure
directly the change
made.

4. 1.Effectiveness: have a stated objectives been achieved?

2.Efficiency: what has been the cost in manpower or finance
in relation to outcome of program?
03Criteria used for evaluating
dental services (WHO-1972)

5. 3. Appropriateness: is the program acceptable to both
consumers and providers and do the priorities reflect a
proper interpretation of the need of the population?
4. Adequacy: has tthe intended coverage of the target
population been achieved and are the services readily
available?
04

6. " The ratio between the achievement of the program
activity and the desired level which, during the planning
process, the planners has proposed would result from the
program. "
05
Effectiveness:

7. 1. Resources: are evaluated by dividing the actual expenditure of
resources by the planned performance.
2. Activities: are evaluated by dividing actual performance by
planned performance.
3. Objectives: are evaluated by dividing actual attainment of
objectives by planned attainment.
06
Three variables are useful in evaluating
effectiveness:

8. "The result that might be achieved by expenditure of
a specific amount of resources and the result that
might be achieved through minimum of expenditure."
07
Efficiency:
# It is a measure of the resources spent ( money, personal
material and time) in the process of providing the health
care program.

9. It is decided by lay decision makers. They will weigh up
weather the problem defined by the program personal is a
problem for the community.
Evaluation of appropriateness can be carried out at 2 levels:
08
Appropriateness:
1.Whether the aims and objectives of the program are
appropriate.
2.Whether the strategy of the program is appropriate.

10. A measure of adequacy is the extend to which the
population in need was covered by the services or the
extend to which the services covered the various aspects
of the underlying problems.
09
Adequacy:

11. 10
Types of evaluation
1. Based on the time duration
Formative and summative evaluation
2.Based on fulfilment of objectives
Effective, efficiency and impact evaluation
3. Based on the output
process, relevance evaluation

12. Formative evaluation:
(PILOT PHASE AND CONTROL PHASE)
“the internal evaluation of program”. It is an examination of the
activities of a program, as they are taking place. Usually carried out to
aid the development of a program at its early phase.
Used primarily by the program developers and program staff members
as to whether they are workable or whether changes should be made
to improve the activities.

13. Summative evaluation
(ACTUALIZATION PHASE AND FINAL OPERATIONAL
PHASE)
Judges the merit or worth of a program after it has been in
operation. It is an attempt to determine whether a fully
operational program is meeting the goals for which it has been
developed.

14. Summative evaluation aimed at program decision makers, who
will decide as to program continues or is terminated and also at
decision makers from other programs who might be considering
adoption of the program.

15. Relevance evaluation:
It refers to activities designed to determine whether the
program is needed or whether the program is targeting its
efforts at the individuals in need.

16. The seven key questions around which relevance evaluations may be organized
are:
1. What problem does the program address?
2. How adequate is the definition of the problem?
3. What is the level of need for services associated with the problem?
4. How accurate is the information about the problem?
5. How adequate is the definition of the program?
6. Is the program appropriate to the defined problem?
7. Are those identified to be in need of services receiving the program?

17. Process evaluation:
It refers to efforts made to assess the extent to which program
implementation complies with the program plan.
The assessment of whether a program is being provided in a
fashion consistent with the planner’s original intentions should be
helpful to program managers in making early adjustments of the
program and in making decisions concerning program
continuation and expansion.

18. Four types of recommendations might logically follow from a process
evaluation:
Terminate the project
Reorganize the project
“Fine tune” the project
Proceed with the project as it has been implemented.

19. Effectiveness evaluation:
It refers to whether program results meet predetermined
objectives.
Here, the emphasis is on immediate outcomes of program activities
and whether these outcomes meet the activities specified by the
program planners.

20. Questions answered by an effectiveness evaluation include:
1. Did the program meet its stated objectives?
2. Were the program providers satisfied with the effects of program
activities?
3. Were program beneficiaries satisfied with the effects of program
activities?
4. Is the problem reduced or eliminated as a result of the program?

21. Impact evaluation:
It refers to the long term outcomes of the program.
It considers whether the intervention had any long lasting effects on the
ultimate problems that the program is intended to remedy.
It is an expression of the overall effect of a program on health status and
socio-economic development.

22. Efficiency evaluation
Attempts to relate the results obtained from a specific program to the
resources expended to maintain the program.
Questions related to efficiency evaluation include:
Do program benefits exceed the costs initiated?
Are program benefits more or less costly per unit of outcome?

23. Dishonest evaluation can be of different types:
Eyewash – appraisal limited to good parts.
Whitewash – covering up the failure by avoiding objectives.
Submarine – aimed without knowing the worth of the program.
Postponement ploy – need to seek facts in hope that crisis will be over by
the time the facts are known.

24. General steps of evaluation
1. Determine what is to be evaluated.
2. Establish standards and criteria.
3. Plan the methodology to be applied.
4. Gather information.
5. Analyze the results.
6. Take action.
7. Re-evaluate.

25. 1. Determine what is to be evaluated:
there are 3 types of evaluation.
(a) Evaluation of structure:This is evaluation of whether facilities, equipment,
manpower and organization meet a standard accepted by experts as good.
(b) Evaluation of process:The process of medical care includes the problems of
recognition, diagnostic procedures, treatment and clinical management, care
and prevention. The way in which the various activities of the program are
carried out is evaluated by comparing with a predetermined standard.
(c) Evaluation of outcome:This is concerned with the end result, that is, whether
persons using the services experience measurable benefits or not.

26. 2. Establishment of standards and criteria:
Standards and criteria should be established, so that they can be compared against.
This is necessary to determine how well the desired objectives have been attained.
Standard and criteria must be developed in accordance with the focus of evaluation.
Structural criteria. Eg: Physical facilities and equipment.
Process criteria. Eg: Every school going child should receive dental check up once in
6 months.
Outcome criteria. Eg: Alterations in health status (positive, negative) or behavior
resulting from health care (satisfaction, dissatisfaction) or the educational process.

27. 3. Planning the methodology:
A format must be prepared for gathering the desired information.
4. Gathering information:
The type and amount of information required will depend on the purpose
of the evaluation.

28. 5. Analysis of results:
The analysis and interpretation of data and feedback to all individuals
concerned should take place within the shortest time feasible.
Opportunities should also be provided for discussing the evaluation
results.

29. 6. Taking action:
For evaluation to be truly productive, actions designed to support,
strengthen or otherwise modify the services involved, need to be taken.
7. Re-evaluation:
Evaluation is an ongoing process aimed mainly at rendering health
activities more relevant, more efficient and more effective.

30. Thank you!!!