07 - Training Evaluation Models

 

Introduction

Training employees incurs costs and is a time-consuming task. Therefore, comprehending the effectiveness of training is essential to ensure allocated resources are not squandered (StaffCircle, 2023). In a corporate setting, training program effectiveness plays a critical role in organizational growth and success. This is where the significance of training evaluation models becomes apparent as they present structured frameworks for assessing the impact, efficiency and value of training initiatives.  Training evaluation models provide valuable insight into the effectiveness of training for learners, its impact on behavior change, and its alignment with organizational goals (Gupta, 2023).

Training Evaluation Models

1. Kirkpatrick Model

This is also referred to as Kirkpatrick's Four Levels of Training Evaluation, a crucial methodology for appraising the effectiveness and outcomes of employee learning programs. Widely acknowledged as one of the most successful methods of assessing training, organizations can employ this model to appraise both habitual and non-formal instruction and correspondingly adopt any style of teaching (Andreev, 2023).

The Kirkpatrick model consists of 4 levels,

• Reaction - Determine the level of satisfaction, engagement, and job relevance among learners using feedback forms.
• Learning – Verify whether the learner has acquired the intended knowledge, skills, attitude, confidence, and commitment from the training program.
• Behavior - Assess if employees are applying the acquired knowledge from the training program to their daily work.
• Results - The focus of this study is to ascertain whether the intended results were achieved through the training program, with the aid of support and accountability from organizational members (Andreev, 2023).

2. The CIRO Model

This model, which was developed by Peter Warr, Michael Bird, and Neil Rackham and published in their book Evaluation of Management Training" in 1970, is used to evaluate the effectiveness of management training courses.  It provides businesses with an effective method of assessing their management training requirements and outcomes (Deller, 2021).

CIRO is an abbreviation that stands for the four levels. And they are,

1. Context – Evaluate the operational status of the organization to identify and assess training needs and objectives. Conduct a training needs analysis to identify areas in which the organization lacks proficiency. Once training needs have been defined, they are categorized into the following three levels:

• The ultimate objective - which aims to eliminate organizational deficiencies, such as poor performance, low profits, low productivity or poor customer service.

• Intermediate objectives - Require changes in employees' work behavior to achieve the defined ultimate objectives.

• Immediate objectives - Involve acquiring new skills, knowledge, and even changing employee attitudes to influence their behavior (Deller, 2021).

2. Input – In this model practitioners should gather information on potential training techniques and methods. This will aid in identifying the most suitable option to achieve the desired training objectives.  The design, planning, management, and delivery of the training course must also be considered. An analysis of the organization's resources is necessary to determine the optimal use of these resources in achieving the desired aims (Deller, 2021).

3. Reaction – In this level, views and the suggestions from the participants about the training they received will be collected. Information were gathered under following aspects, such as program content, approach, value-added. And collected information will use to identify the possibilities of improvement of the training program (Deller, 2021).

4. Output - This involves presenting information about the results of the training. And these results are classified into three different levels. Immediate (consist how the trainees got on and whether they managed to complete the training successfully), intermediate (results that may take some time to implement, such as changes to the course design or acquiring new training resource) and ultimate level (main goals for the organization, ones that have a far-reaching impact on the organization (Deller, 2021).

3. The Phillips ROI Model

The Phillips ROI Model is a revised variant of the Kirkpatrick model by Jack Phillips in the 1970s. Phillips sought to improve Kirkpatrick's work, addressing some of its inadequacies. This approach is commonly referred to as the Phillips ROI Model and should be seen as an expanded edition of the Kirkpatrick Model (Deller, 2021).

Following are the five levels of the Phillips ROI Methodology

1. Reaction

As with the Kirkpatrick model, the Phillips ROI Models begin with evaluating the participants' reaction to the training which they received. The most common approach is to use short questionnaires or surveys to gather data on what the workforce thought about their training (Deller, 2021).

2. Learning

This level is exactly the same as the Kirkpatrick Model and usually requires giving the participants MCQ or quiz before or after the training (Deller, 2021).

3. Application and Implementation

The Kirkpatrick model falls short in providing adequate data for enhancing training. Its ultimate purpose is to determine only whether the training was applied or not.  Therefore, by utilizing this model, it becomes easier to ascertain whether the training has impacted any alterations in the workplace. However, the Phillips ROI model has made significant strides in this regard by extending the measuring phase to encompass the application and implementation stages. The Phillips ROI model enables identification of any problems with learning application or implementation (Deller, 2021).

4. Impact

The Phillips ROI model focus on the impact of the training on the organization in much broader way. So this allows to analyze the impact of training content and other factors that contribute to participants’ final performance (Deller, 2021).

5. Return on investment (ROI)

This is an evaluation of the return on investment, using cost-benefit analysis to determine the worth of training programs. The goal is to enable organizations to gauge whether funds allocated to training have resulted in measurable returns and, if so, to what extent (Deller, 2021).

4. The Brinkerhoff Model

In 2003, Robert O. Brinkerhoff introduced a novel evaluation method for measuring training effectiveness known as the Success Case Method (SCM). It rapidly gained popularity as a practical way of assessing the influence of training, coaching, and other organizational interventions. SCM primarily concentrates on qualitative analysis. It can be applied to scrutinize any significant business alteration, such as the adoption of a new procedure or the acquisition of novel equipment (Deller, 2021).

Key steps of this model,

1. Identify success cases

Identify individuals or groups who have demonstrated exceptional performance as a direct result of the training. These instances of success serve as evidence that the training program has fostered positive outcomes (Gupta, 2023).

2. Identify failure cases

Identify individuals or groups who have not achieved the required performance improvements despite participating in the training. These failure represent the potential barriers or limitations of the training program (Gupta, 2023).

3. Conduct interviews

Conduct detailed interviews with both successful and unsuccessful cases to gather comprehensive information about learners' experiences with the training program. This should include identifying both effective training aspects and potential challenges faced during the application of newly acquired knowledge (Gupta, 2023).

4. Analyze findings

Analyze the data gathered from the interviews in order to identify common themes, patterns and trends. This analysis will help to determine the critical success factors as well as the factors that hindered the effectiveness of the training (Gupta, 2023).

5. Action planning

Develop action plans based on responses and recommendations to improve the training program. These plans may involve improving the content, delivery methods, and support systems or addressing specific challenges faced by participants (Gupta, 2023).

5. Kaufman's Model of Learning Evaluation

Roger Kaufman and John M. Keller’s published the "Levels of Evaluation beyond Kirkpatrick," was featured in the winter edition of Human Resource Development Quarterly in 1994. This publication introduced Kaufman's Model of Learning Evaluation, which is one of many models of learning evaluation that build upon the Kirkpatrick Model, one of the most successful training evaluation models. Kaufman’s model comprises five levels of training evaluation (Gupta, 2023).

1a - Input

This encompasses training materials, including digital resources that facilitate training and coaching  (Deller, 2021).

1b - Process

The second section of the primary level evaluates the suitability and efficiency of the procedure. Simply put, it assesses the actual provision of the learning encounter (Deller, 2021).

2. Acquisition

Kaufman's second level analyses the payoffs for individuals and small groups objectively. The primary focus is on the learner as the 'micro-level client,' examining whether they acquire knowledge and how they apply it on the job. The language used is clear, value-neutral, and concise, with logical progression between statements and accurate technical term usage. Additionally, the text is free from grammatical, spelling, and punctuation errors, and adheres to common academic formatting requirements (Deller, 2021).

3. Application

The third tier assesses the extent to which participants apply their acquired knowledge to their work performance (Deller, 2021).

4. Organizational payoffs

Kaufman's fourth level quantifies returns that benefit the organization holistically. The "macro-level client" would usually refer to the business or organization that is conducting the assessment. This level encompasses evaluations of performance improvements and an analysis of cost-benefit and/or cost-consequence (Deller, 2021).

5. Societal Outcomes

Kaufman’s fifth level focused on what he termed ‘mega-level clients’. This could refer to a business’ clientele and/or to society as a whole (Deller, 2021).

6. Anderson’s Model of Learning Evaluation

The Anderson model of learning evaluation is a distinctive three-stage learning evaluation cycle designed for use at an organizational level. Unlike other training evaluation models that concentrate on targeted learning interventions, Anderson's model prioritizes aligning training objectives with the strategic goals of the organization (Deller, 2021).

Anderson's model is a three-stage cycle, which assists an organization in determining the optimal training strategy for its demands. The three stages are,

Stage 1 – Assess the present alignment of training with regard to the strategic priorities of the organization.

Stage 2 – Assess and evaluate the contribution of learning using a variety of methods.

Stage 3 – Establish the most pertinent approaches for your organization.

Training evaluation models are frameworks that are designed to measure the effectiveness, efficiency, and outcomes of training programs within organizations. It consists of feedback from learners, comparing the performance before and after the program, effectiveness of achieving the organization strategic. Also this models will help organizations collect data, analyze outcomes, and make informed decisions to optimize training strategies. Evaluating training program is an effective method where it requires extra resources. But the benefits of training evaluation will provide greater result than the difficulties. Such as saving money, by evaluation the organization can identify what training programs will be beneficial for the company and which are not towards achieving company objectives. Then another benefit will be identifying gaps in training program. Organization can identify the gaps in particular area by reviewing the evaluation results, so they organization can arrange more training on that specific area (Bloom, 2022).

List of References.

Andreev, 2023. The Kirkpatrick Model valamis.com [Online]. Available at https://www.valamis.com/hub/kirkpatrick-model Accessed on 25th of August, 2023

Bloom, 2022. Training Program Evaluation: How To Achieve Perfection ispringsolutions.com [Online]. Available at https://www.ispringsolutions.com/blog/how-to-evaluate-a-training-program Accessed on 25th of August, 2023

StaffCircle, 2023. Training Evaluations Model: The Complete Guide staffcircle.com [Online]. Available at https://www.staffcircle.com/blogs/training-evaluations-model-the-complete-guide/ Accessed on 25th of August, 2023.

Deller, 2021. CIRO Model: The Definitive Guide kodosurvey.com [Online]. Available at https://kodosurvey.com/blog/ciro-model-definitive-guide Accessed on 25th of August, 2023

Deller, 2021. Brinkerhoff Model 101: Methodology and Goals kodosurvey.com [Online]. Available at https://kodosurvey.com/blog/brinkerhoff-model-101-methodology-and-goals Accessed on 25th of August, 2023

Deller, 2021. Kaufman's Model of Learning Evaluation: Key Concepts and Tutorial kodosurvey.com [Online]. Available at https://kodosurvey.com/blog/kaufmans-model-learning-evaluation-key-concepts-and-tutorial  Accessed on 25th of August, 2023

Deller, 2021. Anderson Model of Learning Evaluation: The Comprehensive Guide kodosurvey.com [Online]. Available at https://kodosurvey.com/blog/anderson-model-learning-evaluation-comprehensive-guide Accessed on 25th of August, 2023

Gupta, 2023. 6 Best Training Evaluation Models in 2023 whatfix.com [Online]. Available at https://whatfix.com/blog/training-evaluation-models/. Accessed on 25th of August, 2023.