What’s Brewing in the Learning and Development Space?

by Rahul Sinha, 17 Nov, 2016

“One of the principles of the future employee will be shifting from being a “knowledge worker” to being a “learning worker” – Jacob Morgan, “The Future of Work.”

Learning in the workplace is a constantly evolving space. Today, every organization aspires to be a “Learning” organization giving significant emphasis on “self-learning”. A robust, technology-driven and flexible learning platform that enables continuous learning Continue Reading


Implementing Kirkpatrick’s model in SuccessFactors Learning Management System (LMS) for training evaluation

By Sarayu Raghavan, August 05, 2016

Expenditure on Corporate training is soaring higher than ever. According to a Deloitte research, the corporate training expenditure worldwide has risen up to over $130 Billion. Employees are the most valued assets and their proficiencies are a priority. Organizations spend a great deal of time and money on devising and conducting trainings to develop employee capabilities.

But are the trainings adding value to the employee and organization? Are the trainings being evaluated?

Talent Development Teams believe that their employees are receiving the right training and think that their work ends after the course is created, delivered, and recorded.  Evaluating the effectiveness of training offerings is a critical step that is often overlooked.  Training and development is a substantial investment of both money and effort. The results of the training must therefore be evaluated in the context of impact on your organization.

Most Organizations struggle with Training Evaluation because their Learning and Management Systems do not have the capabilities to measure effectiveness of training. It consumes lot of resources and is not efficient in such organizations.

How to perform an effective Training Evaluation?

There are several approaches to it. One of the most renowned approaches is Dr. Donald Kirkpatrick’s model called “The Kirkpatrick model”. There are four levels to it. But in order to measure these four levels, you would a need a competent system.

Well, SAP SuccessFactors’ Learning and Management System (LMS) is based on this successful evaluation model.  In this blog post, I want to highlight how organizations can leverage the SuccessFactors LMS to efficiently evaluate the performance of their training programs.

What are the four levels of Kirkpatrick’s evaluation model?

Fig 1: The four levels of Kirkpatrick’s model

Fig 1: The four levels of Kirkpatrick’s model

  • Reaction: What learners thought and felt about the training
  • Learning: The resulting increase in knowledge or capability
  • Behavior: Extent of behavior and capability improvement and implementation/application
  • Results: The effects on the business or environment resulting from the learner’s performance.

Implementing the model in SuccessFactors

Fig 2: Implementing the training evaluation model

Fig 2: Implementing the training evaluation model

The following tools are available in SAP SuccessFactors (SF) Learning and Management System (LMS) relative to Kirkpatrick’s evaluation model.

Table 1: Tools in SF LMS

Table 1: Tools in SF LMS

  • Reaction- Item Evaluation (Smile/Happy sheet)-

This level measures how your trainees reacted by soliciting their opinions of the learning experience after a training course. It helps you understand how well the training was received by your audience and helps improve future training.

Item evaluation is a questionnaire survey and is most effective if delivered immediately after training.

Fig 3: Sample questionnaire survey

Fig 3: Sample questionnaire survey

With SF LMS, this survey can be structured with one or more pages and one or more questions per page.  There are four types of questions namely –

  • Rating Scale
  • One Choice
  • Multiple Choice
  • Open Ended
Fig 4: Structure- one or more pages with questions

Fig 4: Structure- one or more pages with questions

Organizations can use this feedback to know the effectiveness of the training, the trainee’s perception, identify the areas that are missing and make future improvements thus doing justification for the training expense.

  • Learning- Pre/Post Exams (Tests)

This level measures the degree to which your trainees have gained knowledge, and skills as a result of the training. It determines if the training objectives are met.

In SuccessFactors LMS, you can create pre and post tests (exams) to measure the increase in intellectual capability from before to after the training experience.  It is relatively simple to set up, and is clear cut for quantifiable skills. You get to know if the trainees learnt what was intended to be taught, and was the advancement in the intended direction and so on.

Fig 5: Sample pre test question

Fig 5: Sample pre-test question

Fig 6: Sample post test question

Fig 6: Sample post test question

Measuring learning evaluation is important as knowing it helps you analyze if there is a demonstrative increase in their skills and also helps improve future trainings.

  • Behavior- Follow up evaluation survey

Well, your trainees might have scored well in the post test but are they actually implementing what they learnt remains as a question. How to measure it? Typically, this level measures the change in behavior of your trainees as a result of training- whether they are applying what they learnt on the job.

Ideally, in SuccessFactors LMS, the follow up survey is conducted 3-6 months post the training program. This time period gives the trainees an opportunity to utilize their knowledge.

SuccessFactors LMS allows survey questions to be best fit to the evaluation methodology.  Surveys are set up for managers and users based on how you want to set it up.

Fig 7: Sample follow up survey for the manager

Fig 7: Sample follow up survey for the manager

Survey Evaluation Analysis Reports

There are specific reports to support analysis of each level of the training evaluation process.  The following reports can be derived –

  • Item Evaluation Report – mean score for each survey, survey page and survey question and the percentage of users who selected each response
  • Item Evaluation by Individual Response Report – shows each user’s responses to the survey questions
  • Item Evaluation by Instructor Report – mean score for each survey and survey page grouped according to the instructor
  • Learning Evaluation Report – shows training effectiveness of items based on the average pre and post test scores
  • Follow up Evaluation Report – mean score for each follow-up survey, survey page and survey question
  • Follow up Evaluation by Individual Response Report – mean score for each follow-up survey and survey page
Fig 8: Sample follow up evaluation

Fig 8: Sample follow up evaluation

Investing in Human Capital is a wise decision for every organization.  Knowing the importance of training alone isn’t sufficient- you must know the role Training evaluation plays. Surely, you wouldn’t want to spend your time, effort and money on something which is not yielding right? SuccessFactors LMS’ training evaluation feature helps you streamline your training process.   The various levels of evaluation not only helps gather data but also helps improve training based on the feedback received.

If you are an organization looking for a complete LMS solution or if you are already a SuccessFactors customer and want help with implementing the Kirkpatrick’s evaluation model, please reach out to us at irene.jones@neeyamo.com.