Important question time:
#1: How do you know if your online learning program is truly effective?
#2: How do you know what you should be measuring to gauge effectiveness?
Answer:
The only way to be sure is to use a proven framework for evaluating your program. One of the most popular and widely-used learning evaluation models was created by Dr. Don Kirkpatrick almost 70 years ago, and yet it’s still being used today because it’s so effective.
The original Kirkpatrick Model includes four separate levels of evaluation:
- Reaction
- Learning
- Behavior
- Results
These are the four primary areas you should be evaluating every time someone completes one of your courses.
Obviously, we’ve come a long way in terms of technology and systems since the original Kirkpatrick model was created, and this means we now have a NEW model for evaluating learning programs.
The new Kirkpatrick model includes several additions that are much more appropriate for the learning environment (and the learners) of today’s corporate environment.
For example, in the Level I evaluation of the past, reaction was simply measured according to the level of satisfaction that learners were getting from the training.
“Did you like the course?”
“Sure.”
That’s it.
Essentially you were asking the learner, “Are you satisfied with the training?” and if they were, you could check that off the list.
However, now we also measure engagement and relevance, both of which add a deeper dimension to the overall learning experience. In a world of “always on” social media and information exchange, engagement and relevancy are essential to maintaining attention.
So if you use the Level I model and you determine that your learners are satisfied, engaged, and the material is perceived as being relevant, then you can move on to the second level of evaluation.
Level II: Learning
During this level you’re going to be evaluating how much of the intended material has actually been retained and utilized after your learners have gone through the training. Are they actually using what you taught them - or did they simply memorize the information without truly learning to apply it?
To find out, you will break things down into subcategories and evaluate:
- Knowledge
- Skills
- Attitudes
- Confidence
- Commitment
You won’t just be testing to see if your learners have retained the knowledge, you’ll also be trying to determine their beliefs and feelings about the material they are going through.
If your learners score highly in those five areas of evaluation then you can confidently move on to the third level of your evaluation process.
Level III: Behavior
At this point you will be monitoring performance to determine how much of the learned material is actually being applied in the workplace.
If your learners love the material but they are not applying it correctly on the job, for example, then you will need to take a close look at your program to find out where things are falling through the cracks so you can fix it.
One of the more interesting additions to the Kirkpatrick model is the inclusion of “required drivers” as an evaluation point. This is where you take a close look at your processes for encouraging and rewarding learners for their behavior.
This is one of the keys to getting the material to actually “stick” and ultimately become utilized by your team in the workplace.
Once you’ve thoroughly evaluated the behavioral aspects of your program, then you can move on to the final level and measure the most important aspect of your program.
Level IV: Results
This is where you’ll be taking a look at your metrics and determining the overall efficacy of your program.
(Interestingly enough, this is also the critical point where many developers completely drop the ball in the course design.)
What many course developers don’t realize is the fact that your job does not end once the learners have gone through your program – the truth is that right here is where the real learning “golden nuggets” can be found.
After your learners have gone through the program you will want to design follow-up activities and evaluations to continuously reinforce the material and make sure that the learners are using it appropriately.
You might use robust Learning Management Systems or you might choose to use simple surveys and email follow-ups at this point. What matters is that you’re following up and helping your learners retain and utilize the information continuously over time.
So there you have it - four primary areas to evaluate your online learning program and find out just how effective it truly is. Once you’ve put your program through this framework, you’ll then be able to troubleshoot various aspects of the training and optimize for performance. Do this and you’ll be surprised how much more effective your systems will become.
At the end of the day you’re looking for a return on your learning investment that you can measure over time. This might be an increase in sales or customer service satisfaction, or it might be a decrease in employee turnover. Whatever your success metrics are, this model can help you optimize your learning program so you can hit your goals and see a positive return.
