Innovating workplace learning: Training methodology analysis based on content, instructional design, programmed learning, and recommendation framework

Kim, Sehoon

doi:10.3389/fpsyg.2022.870574

ORIGINAL RESEARCH article

Front. Psychol., 22 July 2022

Sec. Organizational Psychology

Volume 13 - 2022 | https://doi.org/10.3389/fpsyg.2022.870574

This article is part of the Research Topic COVID-19 and Digital Transformation of Information through Innovation Technologies. View all 9 articles

Innovating workplace learning: Training methodology analysis based on content, instructional design, programmed learning, and recommendation framework

$\r\nSehoon Kim,,*$ Sehoon Kim^1,2,3*

¹Seoul School of Integrated Sciences and Technologies, Seoul, South Korea
²Business School Lausanne, Chavannes-près-Renens, Switzerland
³The Institute for Industrial Policy Studies, Seoul, South Korea

The quality of sales processes is crucial in automotive and directly related to the firm’s competitive advantage and financial success. Sales training is the most prevalent intervention to guarantee quality and productivity. Extant literature has attempted to measure training effectiveness adequately, and the Context, Input, Process, and Product evaluation (CIPP) model has been a popular approach. This study endeavored to advance current literature and suggest a novel effectiveness framework, Content, Instructional design, Programmed learning, and Recommendation (CIP-R). The framework was applied to examine three different methodologies—traditional, pure digital, and hybrid training—collecting 583 instances from the automotive sales training conducted from 2019 to 2020 in South Korea. The findings advocate the importance of human elements, the role of efficacy, and self-determination in generating learning transferability, leading to performance in the digital age.

Introduction

Coronavirus disease 2019 (COVID-19) created a global mechanism that “forced” online learning to be a reasonable alternative to traditional ways of developing people (Lands and Pasha, 2021). HR managers began to rethink development initiatives, providing flexible, personalized online learning opportunities (Li and Lalani, 2020). Training online could comfort the employees, reducing their skill gaps (Zhang et al., 2018) under isolated situations, and global institutions started providing digital workforce development and upskilling programs (Lands and Pasha, 2021).

A skilled workforce is substantial when it comes to the sales function. A firm’s economic success depends on the relationships between the customers and the employees that function as the interface that connects the company’s value chain to the market (Davenport, 2012). Ever-elevating customer expectations upon better services and products, complex buying and selling processes would draw more pressure and let businesses pay attention to the quality and competencies of their sales organizations (Zoltners et al., 2008; Williams et al., 2017; Arli et al., 2018; Hartmann et al., 2018; Hartmann and Lussier, 2020; Lim, 2020). The automotive industry is the largest sector in overall retail spending (Palandrani, 2020) and a vital customer that plays a crucial role in the industry ecosystem (Mathur and Kidambi, 2012; Dweiri et al., 2016). The quality of sales processes is crucial in auto retail since that is where the customers perceive the brand’s quality (Hoffmeister, 2018). Hence, automakers are expected to focus on human resource development (HRD) in retailing. Sales training is the most prevalent and widespread intervention to guarantee quality and increase productivity (Singh and Venugopal, 2015; Kodwani and Prashar, 2019). Today’s sales organizations proactively adopt digitalized training to enhance efficiency and effectiveness (Zoltners et al., 2021). However, it remains uncertain to determine the most optimized way of adopting technology in sales training.

Maximizing the effectiveness is the primary goal for any training initiative (Sitzmann and Weinhardt, 2015) and measuring the outcomes is essential to ensure the continuation and improvement of interventions (Kirkpatrick and Kirkpatrick, 2006). The Context, Input, Process, and Product evaluation (CIPP) model is one of the most frequently used frameworks (Adedokun-Shittu and Shittu, 2013; Finney, 2019). However, its limitations, such as time, cost, complexity, and lack of the participants’ voice, hinder practitioners and researchers from adopting the framework despite its advantages (e.g., Robinson, 2002; Tan et al., 2010). Furthermore, defining the training effectiveness, which includes learning transfer and post-training changes, is vital in the existing training literature (Alvarez et al., 2004). Hence, the CIPP model requires further theoretical review to prove itself as a robust and valuable framework (Finney, 2019). In the post-COVID era, more businesses are expected to provide entirely digital or hybrid learning opportunities for their workers, replacing traditional classrooms. It is required to empirically compare and analyze the effectiveness of traditional and technology-enhanced training. The CIPP model requires further effort to reflect the changing trend and theoretical attempt to calculate the “effectiveness” over “evaluation.”

From the abovementioned data, the study seeks to propose a new framework based on the CIPP model, advancing it into a practical, trainee-centered model. Based on the new framework, the study aims to compare the outcomes of three methodologies, namely, traditional, pure digital, and hybrid training and measure the effectiveness of each method. The study then seeks to analyze the factors that generated the differences in the training outcome. In this respect, the study presents the following research questions: (1) What is the optimal way of measuring training effectiveness in the digital age? (2) How does the training effectiveness change as digital technologies involve? and (3) What are the factors that affect training effectiveness?

This article is structured as follows. Section 2 provides theoretical background and a literature review related to the research. Section 3 deals with the methodology used in the study, analytical procedure, and details for data collection. Section 4 addresses the study’s results and Section 5 summarizes and concludes the research findings. Section 6 proposes theoretical and practical implications. Finally, Section 7 presents limitations and suggestions for future researchers.

Theoretical background

Training of sales organization

Modern consumers expect extended service and consumer-centric behaviors from salespeople (Lassk et al., 2012). Businesses focus on keeping the quality of the sales by investing in training (Kodwani and Prashar, 2019). Firms strive to meet consumers’ escalating expectations and manage organizational resources to provide quality service (Shabani et al., 2017). Managers expect the training to result in several returns, such as salesforce motivation and sales competence (Kauffeld and Lehmann-Willenbrock, 2010; Panagopoulos et al., 2020), leading to overall performance enhancement and goal achievement (Salas-Vallina et al., 2020). Sales training is the most frequent and universal intervention to improve sales productivity or customer orientation (Singh and Venugopal, 2015; Kodwani and Prashar, 2019). Effective sales training interventions may enhance sales organizations’ knowledge, skill, and performance (Lichtenthal and Tellefsen, 2001; Singh et al., 2015).

The most common challenges that organizations undergo are (1) constant changes in salespeople’s role and definition, (2) emphasis on accountability and reliability of sales function, and (3) reskilling and upskilling of salesforce facing new technologies (Lassk et al., 2012). However, one of the less explored issues is the (1) understanding of differences in training methodologies and (2) measurement for training evaluation and effectiveness (e.g., Lupton et al., 1999; Attia et al., 2021). This article discusses two primary challenges in the following sections of the literature review, namely, training methodologies and the training evaluation/effectiveness measurements.

Digital technology in a corporate learning context

Using technology in training must conform to the changing environment (Hartmann and Lussier, 2020). Digital transformation of salesforce pursued by firms has been accelerating (Guenzi and Nijssen, 2021) due to the merits, such as elevated efficiency and effectiveness (Zoltners et al., 2021). Online learning is now replacing traditional, instructor-led, and offline settings, becoming a significant corporate learning and development element to upskill employees based on a lower cost (Lands and Pasha, 2021). Advanced technologies adopted in the learning management system provide a flexible learning environment (Traxler, 2018), and technologies such as gamified learning could affect users’ flow and continuous usage (Kim, 2021).

However, a single method (purely online or purely offline) might be challenging to satisfy all learners. Instead, by combining online and offline, learning satisfaction could be increased (Mantyla, 2001). Integrating technology with traditional methods has raised the interest in “Blended learning” (Hrastinski, 2019). Blended learning combines traditional methods with technology-based learning and avoids the demerits of offline and online learning (e.g., Driscoll, 2002; Graham, 2006). At present, the terms blended learning and hybrid learning are interchangeably used (Watson, 2008; Graham, 2009) and are regarded as one of the most suitable modes for training in the digital era (Liu et al., 2020). Consequently, technology-blended learning could lead to best practices for businesses (Singh and Reed, 2001; Driscoll, 2002) since the learners’ cognitive process and the training/teaching strategy might change (Liu et al., 2020).

This study takes the empirical case from a sales training program conducted by Company A in South Korea. The firm is the regional headquarters of a multinational automotive company. The firm’s overall HRD system is shown in Figure 1. The sales training program, which is the main interest of this study, is named sales consultant training and education program (STEP). The company had difficulties conducting face-to-face sales training during COVID-19. Consequently, the firm decided to implement salesforce training using digital technology. In this respect, two alternatives, namely, structured pure online training and hybrid training that mixed offline and digital, were developed and applied.

FIGURE 1

Figure 1. Human resource development at Company A.

Training transfer, evaluation, and effectiveness

Measurement of training outcome contributes to the continuity of training and program improvement (Kirkpatrick and Kirkpatrick, 2006). Training evaluation is an act of evaluating the value or quality of training elements systematically and scientifically. The expectancy theory by Vroom (1964) supports the views of prior researchers that the training’s perceived value or trainees’ beliefs (valence) about the training outcome would be crucial to training success (Colquitt et al., 2000). If the trainees get motivated, their job performance may increase with the feeling of accomplishment, which may also provide the potential for their future growth (e.g., Hanaysha and Tahir, 2016; Ibrahim et al., 2017; Wolor et al., 2020).

One of the most widely accepted performance indicators is the transfer performance of learning (Leach and Liu, 2003). Changes might occur when the employees transfer what was learned to their workplace (i.e., Ployhart and Hale, 2014). Researchers have insisted that training efforts will not bring the desired outcomes if a proper transfer does not follow (i.e., Wilson et al., 2002). Extant training literature has argued that learning transfer is a concern for organizations (Friedman and Ronen, 2015). Learning transferability has been a long-time researched topic in educational psychology (Hung, 2013). Adapting acquired knowledge in changed or modified forms in diverse situations to complete a given task or solve problems based on cognitive processes roughly defines “learning transfer” (Hung, 2013). Training evaluation and training effectiveness have drawn significant recognition in the literature (e.g., Broad and Newstrom, 1992; Kraiger, 2002; Noe and Colquitt, 2002; Holton, 2003). Although two terms, training evaluation and training effectiveness, are interchangeably used, they are independent of each other (Alvarez et al., 2004). Training evaluation means the appropriateness, relevance, and usefulness measured by the reaction to training content and design items. It is more of a “measurement technique” to check if the training programs meet the goals as planned (Alvarez et al., 2004). In contrast, training effectiveness investigates the variables that might affect training outcomes throughout the process (pre/during/post) and could be estimated by transfer measures and post-training attitudes (Alvarez et al., 2004). Several conceptual models (e.g., Tannenbaum et al., 1993; Holton, 1996) were suggested to measure training effectiveness. In summary, training evaluation is a methodological approach for measuring training outcomes, while the latter is a theoretical effort to understand the causalities.

However, the studies dealing with the various effects of variables and timewise training outcome comparison are limited (Grossman and Salas, 2011; Massenberg et al., 2017). As previously discussed, the evaluation of extended training intervention adopting technology should be able to (1) measure transfer performance adequately and (2) explain training effectiveness that could capture the states of trainees.

Advancing the CIPP evaluation model

Researchers have tried to extend the existing models to measure the training outcome in the literature, and the CIPP is one of the most frequently referenced models (e.g., Adedokun-Shittu and Shittu, 2013; Finney, 2019). CIPP is a comprehensive framework for evaluation aimed at long-term, sustainable interventions (Stufflebeam, 2004), standing for context, input, process, and product evaluation. CIPP reviews four areas of training (Figure 2). Context evaluation focuses on the overall objectives. Input evaluation concentrates on resources (infra, curriculum, content, and material), while process evaluation sees the teaching–learning process or other activities. The attitudinal and behavioral changes are regarded as the outcomes during the product evaluation. Generally, the CIPP is often sequentially applied following the orders of context–input–process–product assessment to confirm the accountability of a program and review the quality of each step (Stufflebeam, 2004). Using CIPP, the evaluators could grasp how well and effectively the training outcomes are achieved.

FIGURE 2

Figure 2. CIPP evaluation model.

However, there are several limitations to the CIPP model. First, although it is a robust model to measure training outcomes, several situations would not allow smooth evaluation due to the complicated dynamics of stakeholders (Robinson, 2002; Tan et al., 2010). Second, it requires tremendous effort and resources, leading to slow evaluation. Third, CIPP is a framework for the evaluator, not the trainee, and might lack actual feedback from the trainees unless adequately applied (Tan et al., 2010). Finally, as pointed out in the previous section, the dynamic aspect of training “effectiveness” should be considered (i.e., Alvarez et al., 2004), including transfer measures and post-training attitudes. The CIPP model requires further updates and theoretical concerns to prove itself as a more robust framework.

This study suggests a new conceptual framework based on the CIPP model, attempting to close the theoretical gaps unveiled above. Then, the new framework collects and analyzes empirical data. The unique effectiveness model that (1) is more concise, (2) enables practical evaluation, (3) realizes trainee-centered evaluation, and (4) could measure learning transferability is presented in the following section.

Methods

Analytical procedure

The study includes the following analytical procedures. First, this study suggests a novel CIP-R model that overcomes the limitations of the existing CIPP model. Second, evaluation data for traditional/pure digital/hybrid training methods were collected from the sales training sessions administered in 2019 and 2020. Third, frequency analysis of the collected data and descriptive statistics for each variable are presented. Fourth, the three training methods are comparatively analyzed, and ANOVA confirms the statistically significant differences. Fifth, factors affecting the learning transfer are identified and compared by general linear model analysis. For all the statistical analysis, the jamovi ver. 1.8.2 software (jamovi, 2021) was used.

The CIP-R framework

The primary goal of this article was to analyze and compare the different modes of sales training enabled by digital technology and examine the factors affecting training effectiveness. As discussed in the “Advancing the CIPP evaluation model” section, the article adopted CIPP as the base framework and attempted to provide a new effectiveness model, further improving its limitations.

For decades, organizations have shifted from satisfaction to next-level constructs to measure training effectiveness (Mattox, 2013). Kirkpatrick (1998) argued that organizations should avoid using one simple “satisfaction measure” to measure training’s value. The rest of the stages in Kirkpatrick’s model focus on more critical aspects of learning transfer, which is training effectiveness (Mattox, 2013). Researchers such as Phillips (2012) and Brinkerhoff (2003) also advocated the necessity of “beyond satisfaction” measures in the post-training stages.

Satisfaction and loyalty (recommendation) are two concepts that have become crucial constructs in modern management (Kristensen and Eskildsen, 2011). The positive causal effect of satisfaction on loyalty has been proved in the literature (i.e., Tuu and Olsen, 2010). Tracking promoters propose organizations a compelling way to measure loyalty (Reichheld, 2003), and promoters who recommend services or products to others are loyal enthusiasts (Reichheld, 2006). This basic idea of focusing on recommendations is borrowed from the net promoter score (NPS) concept by Reichheld (2003). NPS measures the “recommendation” by the customers’ rating toward a simple question asking if they want to “recommend” the service or product they experienced to other close people such as friends, family, or colleagues, using a 0- to 10-point scale. The rating, which is believed to present loyalty, helps divide the respondents into three groups, namely, promoters (9–10), passives (7–8), and detractors (0–6). Considering recommendation factor in training is now commonplace across organizational units and utilized frequently as a valuable predictor and effectiveness measure for HR practices (Mattox, 2013) since it is not the driver but an outcome of training. Certainly, if trainees learn new skills and knowledge, they might want to apply the learning to their jobs and encourage others to do the same when successful. In this respect, recommending a training program could be a strong predictor of learning transfer.

Based on the above discussions, the existing CIPP model was reviewed and redesigned regarding the following two issues. First, the new framework should endorse digital training’s particular characteristics, enabling an apple-to-apple comparison. In this respect, the new framework excluded spatial factors to reflect the digital training’s characteristics and offline-only elements from the measurements. Second, ambiguous and almost-duplicative factors should be removed or simplified for clarity. For example, excessively detailed measures for context (e.g., objectives, goals) and input (e.g., content, curriculum) evaluation in CIPP may confuse the trainees while evaluating. As a result, a new training effectiveness framework, the CIP-R model, was presented (Figure 3). CIP-R is an acronym for content, instructional design, programmed learning, and recommendation. Recommendation measures loyalty. It was adopted for learning transferability toward individual and group levels, which is the key to training effectiveness. As discussed in the “Theoretical background” section, the C-I-P could be viewed as the training evaluation, and the R (recommendation) could be understood as a training effectiveness measurement.

FIGURE 3

Figure 3. CIP-R effectiveness model.

Data collection

As explained in the “Digital technology in a corporate learning context” section, the study takes the empirical data from STEP (sales consultant training and education program) of Company A, based in South Korea, an importer of global premium and mass automobile brands. Three HRD experts in automotive sales training and two Ph.D. researchers participated in the analytical procedure. Sales training data from March 2019 to October 2020 were used. Trainees assessed all training sessions through an online survey. To ensure unbiased evaluation, the survey did not ask for personal information. The collected survey results were sent to the firm’s training experience management system. The measurements of the CIP-R framework are shown in Table 1, and the 5-point Likert scale was applied to all items. STEP is a structured program designed to maximize the competence of sales consultants and aims to pursue customer delight through quality sales. The training is a 3-day, off-site, traditional classroom-type program. Due to the suspension of training during COVID-19, the company shifted to pure digital (from April to June 2020) and hybrid training (from July to October 2020) mixed online and offline.

TABLE 1

Table 1. Measurements.

In the traditional training conducted in 2019, 325 trainees participated in the 3-day session. Offline-based lectures, discussions, role-playing, and problem-solving sessions were implemented. A total of 140 sales executives participated in the pure digital training. Lectures and interactions were not provided, and the course depended on the learner’s self-directedness. Hybrid training was a well-blended version of offline and online. A total of 118 employees participated, and real-time live streaming sessions were provided. After the online session, instructors visited the learners’ offices to check their competencies and conducted role-play and problem-solving evaluations. First, the instructors handed out various scenarios, including multiple car purchase scenarios for customers with differentiated needs. Second, they evaluated each personnel based on structured measurement scales and checklists to assess sales consultants’ capabilities expected to be enhanced throughout the prior online learning sessions.

Although the three methods differed in their implementation, there was no significant difference in each course’s learning objectives and curriculum. The traditional method was the highest in terms of cost, the hybrid type was medium, and the pure digital training was the lowest. The details of the three training modes and frequency table are presented in Tables 2, 3. Brand: premium brand salespeople 59.2%, mass brand 40.8%. Grade: certified (42.5%), expert (33.3%), pioneer (23.5%), master (0.7%). Training methods: traditional (55.7%), pure digital (24.0%), and hybrid (20.2%).

TABLE 2

Table 2. Comparison: traditional, pure digital, and hybrid training.

TABLE 3

Table 3. Frequency: brand, sales consultant grade, and training method.

Results

Outcome comparison of three training methods

To compare the effectiveness of the three training methodologies, the evaluation results by training participants were analyzed. A total of 583 data instances were used for the analysis. The basic summary statistics results are as follows (Table 4). The mean value of the items was 4.266–4.501. The item with the highest value was the “Sales and service philosophy” item in content evaluation, and the lowest item was the “Practice and participation” item in instructional design. The results indicate that the training participants were relatively satisfied with the training content delivering the firm’s philosophy and were less content with participatory and practical learning settings. The mean of all variables was 4.419. The average values of each dimension of content, instructional design, and programmed learning were 4.471, 4.386, and 4.409, respectively, and it was found that the content dimension received relatively higher ratings than the other three dimensions.

TABLE 4

Table 4. Summary statistics.

Then, all variables were divided by three training methodologies, and group statistics were derived (Table 5). By comparing the mean value of each item, it was confirmed in which area the differences in the training methodology developed significantly. As for the “recommendation” item that indicates training effectiveness, the traditional method appeared to be the highest (4.523), and the pure digital method was the lowest (4.093), while the hybrid method was in the middle (4.339). In the first half of 2020, when all the training interventions shifted from traditional to pure digital due to COVID-19, the item where the trainees’ rating decreased the most was the “Lecturer and instructor engagement” in the instructional design dimension (traditional = 4.686, pure digital = 4.086, difference = −0.600). The second highly impacted item was the “Duration of training” in the same dimension, showing a steep decrease (traditional = 4.560, pure digital = 4.114, difference = −0.446). The item that showed the slightest change was the “applicability” item in the programmed learning dimension (traditional = 4.520, pure digital = 4.386, difference = −0.134). In the second half of 2020, training shifted from pure digital to the hybrid method, and the overall ratings were improved. The “Lecturer and instructor engagement” item in the instructional design dimension indicated the most improvement (pure digital = 4.086, hybrid = 4.636, difference = + 0.550). The next most improved area was the “Practice and participation” in the instructional design dimension (pure digital = 3.971, hybrid = 4.263, difference = + 0.292). The item with the most negligible difference was “Confidence and overall competence” in the programmed learning dimension (pure digital = 4.286, hybrid = 4.364, difference = + 0.078).

TABLE 5

Table 5. Group descriptive by training method.

Finally, an ANOVA was conducted for each variable. The result displayed that there is a statistically significant difference stemming from training methodology in all items except for two, “Confidence and overall competence” (F = 3.009, p > 0.05) and “Applicability” (F = 2.264, p > 0.05) in the programmed learning dimension (Table 6). Additionally, post hoc test results were provided to check the significant mean differences found in the variables based on the training methodology (Table 7).

TABLE 6

Table 6. One-way ANOVA.

TABLE 7

Table 7. Post hoc tests.

Factors affecting training effectiveness

Then, the study verified the results of the ANOVA omnibus test and the fixed-effects parameter estimates result (Gallucci, 2019) to confirm the variables’ effect sizes on training effectiveness and the coefficient for each term (Tables 8, 9). The three sales training methods—traditional, pure digital, and hybrid—showed statistically significant differences (F = 4.882, p < 0.01). The result confirmed a statistically significant difference between the pure digital method and the traditional training effectiveness (B = −0.121, p < 0.01). The difference between the hybrid training and the pure digital method was not statistically significant (B = 0.040, p > 0.05). The existence of five factors affecting the training effectiveness (recommendation) was confirmed. “Lecturer and instructor engagement” (F = 17.783, B = 0.134, p < 0.001), “Duration of training” (F = 27.798, B = 0.162, p < 0.001) in instructional design dimension and “Personalized learning” (F = 9.231, B = 0.121, p < 0.01), “Confidence and overall competence” (F = 52.304, B = 0.330, p < 0.001), and “Skill level for the job” (F = 11.221, B = 0.168, p < 0.001) in programmed learning dimension confirmed the clear difference by methodology and appeared to affect the dependent variable in a statistically significant way.

TABLE 8

Table 8. ANOVA omnibus test.

TABLE 9

Table 9. Fixed-effects parameter estimates.

Finally, estimated marginal means of the dependent variable were calculated and compared for each training method to compare training effectiveness, reflecting the influence of each term included in the model. The result indicated that the traditional method has the highest effectiveness (mean = 4.429). The effectiveness decreased in the pure digital method (mean = 4.308) and then recovered after shifting to the hybrid training method that blended both (mean = 4.348). The result is presented in Table 10 and Figure 4.

TABLE 10

Table 10. Estimated marginal means comparison.

FIGURE 4

Figure 4. Training effectiveness: traditional, pure digital, and hybrid.

Discussion

This study suggested a novel CIP-R training effectiveness framework to analyze the difference in training outcomes derived from the adoption of digital technology and verify which factors influence training effectiveness. The findings of the study are summarized as follows. First, the results of this study confirmed the effectiveness of blended learning, the hybrid method (Table 10). The overall sales training effectiveness fell as the firm shifted from the traditional to the pure digital (pure digital—traditional, mean difference = –0.121). However, the effectiveness was recovered with the hybrid method (hybrid—pure digital, mean difference = 0.040). Pure digital training required a complete self-directedness of participants and showed strength in terms of time and cost but had the lowest effectiveness. Blending the advantages of digital and offline could enhance learning effectiveness (Mantyla, 2001) and under today’s digital transformation, advancing traditional methods by technology-based learning (e.g., Driscoll, 2002; Graham, 2006) might be a suitable way of talent development (i.e., Liu et al., 2020).

Second, the findings confirmed the importance of human factors in training. The variables that caused visible differences in training effectiveness were primarily related to human involvement. The score of “Lecturer and instructor engagement” in the instructional design dimension decreased the most (pure digital—traditional, mean difference = –0.600) and then recovered to the maximum (hybrid—pure digital, mean difference = 0.550). Also, the “Practice and participation” item of the same dimension showed the second-largest increase (hybrid—pure digital, mean difference = 0.291). In the “Duration of training” item, which evaluates the appropriateness of learning time, the second-largest decrease was confirmed (pure digital—traditional, mean difference = −0.446). This empirical evidence confirms the claims that social interactions with colleagues in a digital learning setting play a crucial role in trainee engagement (i.e., Kim, 2021). Although traditional and pure digital training had the same learning duration (3 days), the perceived duration of online self-directed training might have been felt loosely. In this respect, social relationships might have to be regarded as a substantial factor in building a proper learning environment.

In contrast, the ANOVA result displayed that there were no statistically significant differences by training method in the “Confidence and overall competence” and “Applicability” items of the programmed learning dimension (Table 6). Despite the changed training methodologies, the actual content changed little; accordingly, the training program’s perceived value (valence) may not have changed. However, a statistically significant difference was found in the same dimension, in the “Personalized learning” and “Skill level for the job” items. This finding may support the assumption that the human factors caused a difference positively regarding training effectiveness. In fact, in hybrid training, instructor–trainee interaction was strengthened, and practice and feedback sessions were provided through field visits.

Third, businesses must understand the role of efficacy and self-determination. Factors that influenced the learning transferability were mostly variables that define how the learning is delivered to the participants and variables related to trainees’ benefits (two variables in the instructional design dimension and three in the programmed learning dimension). In fact, in this study, the four variables in the content dimension were found to have little effect on learning transfer (Table 9). If personalized learning is provided, trainees could feel confident in their knowledge and skills, have higher expectations for improved competency, and develop actual work performance based on learning transfer (Colquitt et al., 2000). It can be explained by the self-efficacy of individuals and groups (i.e., Bandura, 2000).

Moreover, it can be expected that self-determined learning will occur through programs that provide a sense of self-efficacy (i.e., Ryan and Deci, 2000). As the training program shifted to hybrid, trainees appeared to have found more value. If intrinsic motivation increases, behaviors to disseminate knowledge and skills acquired during training to peers would increase. Based on empirical evidence, extant literature argued that intrinsic motivators such as self-efficacy in knowledge/skill and enjoyment of supporting others might expedite the transfer of acquired competencies, enhancing the upper-level or group performance (e.g., Lin, 2007; Wen and Lin, 2014; Na-Nan and Sanamthong, 2020). Consequently, it can be assumed that understanding group psychology and reflecting it on talent management might lead to higher workforce performance.

Theoretical and managerial implications

The study provides theoretical implications as follows. First, this study suggested the CIP-R framework, a novel effectiveness model that overcomes the limitations of the CIPP model. In particular, the study differentiated itself from the existing models by defining the trainee’s intention to recommend as a measurement for future learning transferability. The studies that provided empirical evidence of the effects of training variables and learning transferability as training outcomes are limited in the literature (Grossman and Salas, 2011; Massenberg et al., 2017). The CIP-R framework and the empirical evidence presented in this study are expected to contribute to existing domain knowledge and theoretical expansion. Second, by demonstrating the effects of digital technology in training, the study drew academic interest in sub-variables in each dimension. Moreover, the study calls for scholarly attention to organizational psychology theories by shedding light on individual/group efficacy (i.e., Bandura, 2000) and self-determination theories (i.e., Ryan and Deci, 2000), searching for a structural mechanism leading to learning transferability. The findings of this study present a perspective on how existing theories should change and be applied as digital technology expands.

The managerial implications of the study are as follows. First, the importance of digital technology acceptance should be reviewed from the talent development perspective. From a business standpoint, it is unimaginable to return to the traditional ways of developing people. HR professionals should consider development programs and provide flexible, personalized learning online (Li and Lalani, 2020). This study attempted both a blended learning format and an entirely online program. The findings of this research could be a practical example for future HRD professionals. Second, the study devised a practical and efficient framework for workforce development. Constant upskilling would be critical for organizations. Businesses need a useful and ready-to-use recipe for their people development initiatives. Existing effectiveness models are resource-taking and inadequate to measure the dynamic aspects of the learning or unreasonably focus on checking post-training changes (e.g., Robinson, 2002; Alvarez et al., 2004; Tan et al., 2010). Sometimes such academic interests fail to meet the actual needs of the business managers (Mattox, 2013). This study proposed the CIP-R framework as an alternative that efficiently measures learning transferability. It would enable practitioners to explore new opportunities to evaluate and improve training at a lower cost.

Limitations and future research

This study has several limitations. First, it is difficult to generalize the research findings since the study was conducted in a specific context, such as the region and profession. It remains uncertain that the findings of this study could be applied to other cultures or workplace environments. Therefore, further studies are required to test the causalities between variables. Future researchers are invited to conduct repeated research with different subjects and contexts. Second, the conceptual framework presented in this study requires additional validation. It is needed to examine the utility of the CIP-R model through additional empirical studies. The replicated usage can strengthen the model’s validity, and its theoretical/managerial value could be proved. The critical views from subsequent researchers, model adjustments, and measurement changes are also considered meaningful. The study used statistical regression to examine causal relationships and effect sizes. If enough data instances are secured, using the recently preferred machine learning (ML) method could be valuable (Kim et al., 2021). Using ML techniques, non-linear, hidden relationships might be discovered. Third, the study did not track trainees’ post-training behavioral changes or the financial results generated. However, training is not the only driver for organizational performance improvement. Hence, it is recommended to study other related variables, such as the working environment, motivation, and exchange relationships in the workplace, which might be related to sustaining the post-training performance.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the patients/participants was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

SK was responsible for the conceptualization, data collection, methodology, analytical procedures (quantitative), writing – original draft, review and editing.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Adedokun-Shittu, N. A., and Shittu, A. J. K. (2013). ICT impact assessment model: an extension of the CIPP and the Kirkpatrick models. Int. HETL Rev. 3, 1–26.

Google Scholar

Alvarez, K., Salas, E., and Garofano, C. M. (2004). An integrated model of training evaluation and effectiveness. Hum. Resour. Dev. Rev. 3, 385–416. doi: 10.1177/1534484304270820