Assessments and Kirkpatrick Model | Evaluating Training Programs

An insightful presentation on the Kirkpatrick Model, a widely recognized framework for evaluating training programs. This session will delve into the four levels of the model: Reaction, Learning, Behavior, and Results. We’ll explore the relationships between these levels and discuss the types of assessments used at each stage. Additionally, we’ll examine how learning and behavior are measured and the impact these assessments have on overall training effectiveness. Whether you’re an educator, trainer, or HR professional, this presentation will provide valuable insights into optimizing your assessment strategies.

1. Slide 1
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Assessments and the
Kirkpatrick Model
Using Assessments to
Evaluate Training

2. Slide 2
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Donald Kirkpatrick
Kirkpatrick developed a model of training
evaluation in 1959
Arguably the most widely used approach
Simple, Flexible and Complete
4-level model

3. Slide 3
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
The Four Levels
Level I: Evaluate Reaction
Level II: Evaluate Learning
Level III: Evaluate Behavior
Level IV: Evaluate Results
Fifth level was recently “added” for return on
investment (“ROI”) but this was not in
Kirkpatrick’s original model

4. Slide 4
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Relationship Between Levels
Each subsequent level is
predicated upon doing
evaluation at lower level
A Level 3 will be of
marginal use, if a Level 2
evaluation is not
conducted
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - Knowledge
Did they learn anything
Level 3 - Behavior
KSA being used on the job?
Level 4 - Results
Was it worth it?

5. Slide 5
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Improve
Learning Environment
Improve
Knowledge/Skill transfer
Check
Performance Environment
Check Requirements,
Systems and Processes
Only by assessing each level can we
yield actionable results
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - Knowledge
Did they learn anything
Level 3 - Behavior
KSA being used on the job?
Level 4 - Results
Was it worth it?

6. Slide 6
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Types of Assessments Used at Each
Level
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - Knowledge
Did they learn anything
Level 3 - Behavior
KSA being used on the job?
Level 4 - Results
Was it worth it?
Type Form
Summativ
e
Correlation of business
results with other
assessment results
Summativ
e
Observation of Performance
360° Survey
Diagnostic
Summativ
e
Self-assessment
Test
Reaction
Formative
Survey
Real-time Polling
Quizzing

7. Slide 7
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reaction - What Is It?
How favorably participants react to the
training (“Customer satisfaction”)

Collects reactions to instructor, course, and
learning environment

Communicates to trainees that their feedback is
valued

Can provide quantitative information

8. Slide 8
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reaction - What It Looks Like
Questionnaire - Most common collection tool

Content

Methods

Media

Trainer style

Facilities

Course materials

9. Slide 9
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reaction - How to Perform
Determine what you want to find out
Design a form to collect/quantify reactions
Do Immediately
Develop acceptable scoring standards
Follow-up as appropriate

10. Slide 10
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Learning - What Is It?
Knowledge
Skills
Attitudes

11. Slide 11
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Learning - What It Looks Like
Media used to measure learning:

Text

Voice

Demonstration
Methods used to measure learning:

Interviews

Surveys

Tests (pre-/post-)

Observations

Combinations

12. Slide 12
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Learning - How to Perform
Use a control group, if feasible
Evaluate knowledge, skills, and/or attitudes
before and after
Get 100% participation or use statistical
sample
Follow-up as appropriate

13. Slide 13
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Behavior - What Is It?
Transfer of knowledge, skills,
and/or attitude to the real
world

Measure achievement of
performance objectives

14. Slide 14
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Behavior - What It Looks Like
Observe performer, first-
hand
Survey key people who
observe performer
Use checklists,
questionnaires, interviews,
or combinations

15. Slide 15
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Behavior - How to Perform
Evaluate before and after training
Allow ample time before observing
Survey key people
Consider cost vs. benefits

100% participation or a sampling

Repeated evaluations at appropriate intervals

Use of a control group

16. Slide 16
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Results - What Is It?
Assesses “bottom line,”
final results
Definition of “results”
dependent upon the
goal of the training
program

17. Slide 17
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Results - What It Looks Like
Depends upon objectives of training program

Quantify
Proof vs. Evidence

Proof is concrete

Evidence is soft

18. Slide 18
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Results - How to Perform
Use a control group
Allow time for results to be realized
Measure before and after the program
Consider cost versus benefits
Be satisfied with evidence when proof is not
possible

19. Slide 19
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Summary
Level I: Evaluate Reaction
Level II: Evaluate Learning
Level III: Evaluate Behavior
Level IV: Evaluate Results

20. Slide 20
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Readings to Consider
Kirkpatrick, Donald L. (1998). Evaluating Training Programs: The Four
Levels. Berrett-Koehler Publishers.
Worthen, Blaine R, James R. Sanders, Jody L. Fitzpatrick (1997). Program
Evaluation: Alternative Approaches and Practical Guidelines (Second
Edition).Addison, Wesley, Longman, Inc.
Kirkpatrick, Donald L. (1998). Another Look at Evaluating Training
Programs. American Society for Training & Development.
Sieloff, Debra A. (1999). The Bridge Evaluation Model. International
Society for Performance Improvement.