Choosing questions and planning the evaluation

Questionnaires and evaluation

1. Copyright © 2014 by The University of Kansas
Choosing Questions and
Planning the Evaluation

2. Copyright © 2014 by The University of Kansas
What do we mean by choosing
questions?
•Evaluation questions are the questions your
evaluation is meant to answer about your
work.
•Evaluation questions help set the direction of
the work, as well as assess its effectiveness.
•Ideally, choosing evaluation questions is part
of the planning of the overall program.

3. Copyright © 2014 by The University of Kansas
Questions to ask yourself as
you choose:
•What do you want to know?
•Why are you interested?
•Is the issue you’re addressing important to the
community or to society?
 
•How does the issue relate to the field?
•Is the issue general, rather than specific to your
population or community?
•Who might use the results of your evaluation?
 
•Whose issue is it?

4. Copyright © 2014 by The University of Kansas
Why is it necessary to choose
evaluation questions carefully?
•It helps you understand what effects different parts of your
effort are having.
 
•It helps you clearly define what it is you’re trying to do.
•It shows you where you need to make changes.
•It highlights unintended consequences.
•It guides your future choices.
•In participant evaluations, it involves stakeholders in setting the
course of the program, thus making it more likely that it meets
community needs.
•It provides focus for the evaluation and the program.
•It determines what needs to be recorded in order to
gather data for evaluation.

5. Copyright © 2014 by The University of Kansas
When should you choose questions
and plan the evaluation?
•If possible, choosing questions and planning
the evaluation should be an integral part of
planning your program.
•If your reality makes that impossible, choosing
questions and planning the evaluation should
take place as soon as possible after the
program starts.

6. Copyright © 2014 by The University of Kansas
Who should be involved in
the process?
•To the extent possible, the process should
involve all stakeholders, including program
participants/beneficiaries.

7. Copyright © 2014 by The University of Kansas
 How do you choose questions and
plan the evaluation?
•Choosing questions:
•Describe the issue or problem you’re addressing.
•Describe the importance of the problem.
 
•Describe those who contribute to the problem.
•Assess the importance and feasibility of changing
those behaviors.
•Describe the change objective.
 
•Make sure that the expected changes would
constitute a solution or substantial contribution
to the problem.

8. Copyright © 2014 by The University of Kansas
•Planning:
•Take into account the issues raised by multiple or
very different settings.
•Take into account the issues raised by participant
groups that differ in culture, ability to complete the
program, geographical location, and other factors.
 How do you choose questions and
plan the evaluation?

9. Copyright © 2014 by The University of Kansas
•For outside evaluators, specifically:
•Choose a setting.
•Learn as much as you can about the organization
you’ve chosen.
•Contact the appropriate person(s) and request an
interview.
 
•Plan and prepare for the initial meeting.
 How do you choose questions and
plan the evaluation?

10. Copyright © 2014 by The University of Kansas
•For all evaluators:
•Find out all you can about the context.
•Establish trust with program administrators,
staff, and participants.
•Aim for a participatory evaluation.
•Plan the evaluation, in collaboration with
stakeholders.
 How do you choose questions and
plan the evaluation?

11. Copyright © 2014 by The University of Kansas
•Consider in your planning all the elements of an evaluation:
•Information gathering and synthesis.
•Designing an observational system.
•Developing and testing a prototype intervention.
•Selecting an appropriate experimental design.
•Collecting and analyzing data.
•Gathering and interpreting ethnographic information.
•Collecting and using archival data.
•Encouraging participation throughout the research.
•Refining the intervention based on the evaluation
•Preparing the evaluation results for dissemination.
 How do you choose questions and
plan the evaluation?