Defending the costs of training is the challenge of every training professional. Even at the more “enlightened” companies (who don’t regard their training department as a cost center but rather as an investment center) quantifying the value of their training “investment” is a difficult task.

Donald Kirkpatrick’s Four Levels of Training Evaluation has been the industry standard for measuring training success since 1979. Level One measures the participant’s reaction to the training program (smile sheets) and Level Two measures the learning that has occurred, usually through pre- and post-testing.

But Level Three (behavior changes back on the job) and Level Four (ROI) are rarely, if ever, measured. In fact, according to the Learning Resources Network, 77 percent of organizations use reaction measures; 36 percent use learning evaluations; 15 percent measure behavior change; and only eight percent measure results.

That’s right – fewer than 15 percent of our colleagues measure behavior changes or return on investment from their training programs. Yet, behavior changes and ROI are the very things almost every organization wants from their training department. Despite this justifiably high level of interest, 85 percent of us are not able to provide them with any hard data in these areas. Why not? Well, because it’s pretty hard to do.

One of the key challenges is quantifying soft data. For instance, how do you put a numerical measure on a new process’s ease of use? How do you measure changes in attitude? Work habit? Effectiveness of communication?

While no system is perfect, some new tools have been developed to help us get around these obstacles. To remove the guesswork, we wanted to find a method for evaluating L3 and L4 that uses known data. We recently discovered a tool that uses participant salary and benefits costs to show one type of training impact. This information, which most training organizations already have or can easily obtain, is combined with a self-assessment by the training participant.

Self-Assessment Of Training Results – with only 3 questions!

A self-assessment method for Kirkpatrick Levels Three and Four is reported to have been validated. This technique came to us from Alan Scott, a Chicago associate, who read about the technique in the book Elevators by John Noonan. According to Alan, research cited in the book supports the claim that this method may be used to estimate training effects on the behavior of participants (Level K3) and to begin to quantify organizational impact (Level K4).

The following is an example of the type of questions that may be used for self-assessment. These questions were written to assess the impact of our one-day Systematic Project Management workshop.

[T] What percent of your total working time will you spend on tasks that require the skills or
knowledge of the Systematic Project Management workshop? (circle one)

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100%

[P1] Rate your productivity before the Systematic Project Management workshop on job tasks that require the skills and knowledge of this course. (circle one)

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100%

[P2] Rate your productivity after the Systematic Project Management workshop on job tasks that require the skills and knowledge of this course. (circle one)

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100%

Once these percentages have been determined, we can plug them into the impact formula below:

self assessment formula

Here is a sample calculation for a hypothetical team of four members who attended a job related workshop. (V results should be multiplied by .73.)

self assessment table

*Multiply salary by 1.5 to estimate salary and benefits

More good news! This formula method is likely to drastically underestimate the real impact of training. It assesses only labor costs. But, labor is used to produce revenue. If someone is made more effective at producing revenue (throughput) the impact is an increase in revenue at the same salary and benefits.

If someone is being paid $50,000 and their productivity improves by 10% the method shows a $5,000 net benefit through labor cost savings. However, if being 10% more productive earns the organization new revenue, that is not accounted for. The formula sums reduced costs assuming zero revenue gains.

Example A: Suppose your job involved solving sales problems and then teaching 100 salespeople to sell more effectively. Making you more effective in solving sales problems would allow you to improve sales leveraged 100 fold. Solving one sales problem could dwarf your entire salary and benefits package year after year!

Example B: (Based on a real project but using hypothetical figures): A five-person team responsible for installing a new radiology unit at a hospital attended a BPI Critical Thinking workshop after the project was launched. They were estimating they would be 45 days late. Using the BPI Planning process, they were able to complete the project on time. The organization realized a $450,000 gain in revenues from using the unit 45 days sooner. This would be in addition to the recapture of the team’s labor during those 45 days. BENEFIT: The workers devoted 50% of their time to the project. So the organization gained an estimated 50% of their time (45 days) to be devoted to other work plus $450,000 in lost revenue or an estimated $150,000 in net profit on that revenue.

Conclusion: This impact formula has been validated for labor cost effects and is an easy way to demonstrate Level K3 and K4 results. Extending this formula to include revenue/profit effects would demonstrate results one or two orders of magnitude higher.

Shameless Plug: Document saving using BPI workshops! One method we’ve used to document training impact is to have participants bring issues to class with known value to the organization. Resolving these issues in class demonstrates cost avoidance and revenue gains. Our workshops have consistently demonstrated multiple returns on the training investment within the workshop itself. The benefit of resolving just one major issue easily repays all training costs including the salaries and benefits covering the time away from work of everyone attending the workshop several times over. We guarantee a ten-fold return on your Critical Thinking training investment from problems resolved and decisions made during the workshop itself.

After the workshop, further gains can be assessed using the methods discussed in this article.