MNEA_Presentation_RevisedFinal June22.ppt

self improvement

1. Program Evaluation
Essentials

2. WHAT is Program
Evaluation?

3. Think/Draw/Pair
Share your image withtheperson
sitting next to youand noticecommon
themes
Draw the first image that comes to
mind when you hear or see the word
evaluation
Label your drawing with a
word or two that describes
the image

4. Evaluation defined…
•Systematic determination of the merit or worth
of an object. (Scriven, 1967)
•Utilization-focused program evaluation: done
for and with specific intended primary users for
specific, intended uses. (Patton, 2008)
•What? So what? Now what? (Patton, 2008)
•Numerous additional definitions.

5. How is evaluation different
from research?
•Similarities:
oDescribe objects studied
oUnderstand the relationships between
variables
oUse qualitative andquantitative methods

6. How is evaluation different
from research?
•Differences:
oPurpose
oUtility
oContext
oJudgmental quality

7. Common misconception #1
Evaluation is only about numbers
(quantitative data)

8. Common misconception #2
RESEARCH is about numbers;
evaluation is NOT about numbers.
Instead, evaluation is about words, stories, and
anecdotes (qualitative data)

9. Rather. . .
Research ANDevaluation
use BOTH
quantitative ANDqualitative methods

10. Evaluation is a profession
guided by standards
The Joint Committee
Program Evaluation Standards
Utility
Feasibility
Propriety
Accuracy

11. WHY would I use
Program Evaluation in
my organization?

12. Top 10 reasons you
might use evaluation…
To establish the impact of a program
To see if a program achieved its outcomes
To make informed decisions about a program,
policy, product
To determine the need for a program
To improve a program

13. To figure out how a program can be or has been
implemented
To determine a program’s merit, worth, or value
To examine howa program works
To distinguish what works from what doesn’t
And thus, to make the world a better place
Top 10 reasons you
might use evaluation…

14. What's in it for me?
•Useful skills
•Support from others
•Ability to impress people
•Good food
•It’s fun!

15. What's init for my
organization?
•Clarifies purpose & direction
•Develops common language
•Provides opportunities for collaboration
•Contributes to credibility & helps secure funding
•Promotes motivation & excitement
•Promotes organizational learning & improvement
•Increases effectiveness
•Furthers good decision making

16. What evaluation isand
what you and your
organization getfrom
evaluation depend on
the purpose you
establish for the
evaluation

17. Evaluation Purposes
Two main purposes:
•Valuing (Michael Scriven)
•Decision-making (Daniel Stufflebeam and Michael Quinn Patton)
Accountability
Development
More recently discussed purposes:
•Evaluation for learning(Torres & Preskill, 2001; Preskill, 2008)
•Evaluation for social betterment(Mark & Henry, 2003; 2004)
•Evaluationfor deliberative democracy(House & Howe, 2000)

18. HOWcan I begin use of
evaluation?

19. So it all starts with a
Question…
What do you want to know aboutyour program?
How does your program work?
What is theimpact of your program?
Whatopportunities doyou have to
expand your program to serve others?
How or to what degree do the resources and
activities meet the needs of stakeholders?

20. Answering the question
requires…
Systematic collection and analysis of data
oThis might mean using quantitative measures,
qualitative measures, or both

21. Possible ways to collect
data for an evaluation
Quantitative:
oSurveys
oParticipant
assessments
oCost-benefit analysis
oStatistical analysis of
existing program data
oSome kinds of record
and document review
Qualitative:
oFocus groups
oInterviews
oObservations
oAppreciative inquiry
oCase studies
oSome kinds of record
and document review

22. What are the best
methods for your
evaluation??
It all goes back to the
question you started with…
Some data collection methods
are better than others at
answering your questions
And some tools are more appropriate for the
audience you need to collect information from

23. The Evaluation Process
1.Determine the object (program, process,
etc.) for evaluation and available resources
2.Develop guiding evaluation question(s)
3.Design the evaluation, including methods
4.Implement the evaluation: collect
information, analyze and interpret findings
5.Report and use evaluation

24. Questions?

25. This presentation was developed for
the Minnesota Evaluation
Association
June, 2009
All rights reserved
Presentation prepared by (in alphabetical order)
Edita Bucinca, Ann Mavis, Laura Pejsa, Cindy Reich,
Denise Roseland, and Robert Tornberg
Under the direction of Dr. Jean A. King, University of Minnesota