A study of the potential of training to be transferred to the workplace

Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 89 ISSN 1989 - 9572 A study of the potential of training to be transferred to the workplace Un estudio del potencial de la formación para ser transferida al puesto de trabajo Cristina Granado, Universidad de Sevilla, España Journal for Educators, Teachers and Trainers, Vol. 7 (1) http://www.ugr.es/~jett/index.php Fecha de recepción: 29 de febrero de 2016 Fecha de revisión: 21 de abril de 2016 Fecha de aceptación: 29 de abril de 2016 Granado, C. (2016). A study of the potential of training to be transferred to the workplace.. Journal for Educators, Teachers and Trainers, Vol. 7(1). 89 – 100. Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 90 Journal for Educators, Teachers and Trainers, Vol. 7 (1) ISSN 1989 – 9572 http://www.ugr.es/~jett/index.php A study of the potential of training to be transferred to the workplace Un estudio del potencial de la formación para ser transferida al puesto de trabajo Cristina Granado, Universidad de Sevilla, España [email protected] Abstract In order to improve the effectiveness of the training provided by the agency in charge of training of public servants in the region of Andalusia (Spain), trainers were surveyed about what is the utilization of designrelated transfer factors they do when they design training courses. Thus, the extent to which training is designed to be transferred can be analyzed. The results suggested that trainers focused their interventions more on the trainees’ satisfaction with the level and usefulness of the learning acquired than on the learning transfer to workplace. In addition, this study allowed us to gain better understanding of the perspective of trainers on how training design elements are associated. Four transfer-focused training approaches were detected, whose concomitant use varies depending on training objectives. It is concluded that the study of the transferability of training is useful to detect weaknesses and strengths in training and proposals for improvements and lines of inquiry are suggested. Resumen Con el fin de mejorar la eficacia de la formación ofrecida a los empleados públicos en Andalucía (España) por la organización responsable, se encuestó a los formadores sobre el uso que hacen de los factores de transferencia relacionados con el diseño de la formación cuando planean cursos. Así, es posible analizar en qué medida la formación es diseñada para ser transferida. Los resultados sugieren que los formadores centran más su intervención en la satisfacción de los empleados por el nivel y utilidad del aprendizaje adquirido que en su transferencia al puesto de trabajo. Además, este estudio permitió obtener una mayor comprensión de la perspectiva de los formadores sobre cómo se combinan los elementos del diseño. Así, se detectaron cuatro enfoques de formación centrados en la transferencia, cuyo uso conjunto varía en función de los objetivos de formación. Se concluye que el estudio del potencial de la formación para ser transferida resulta útil para detectar fortalezas y debilidades en la formación y se sugieren propuestas de mejora y líneas de investigación. Keywords Professional training; Transfer of training; Training methods; Evaluation Palabras clave Formación continua; Transferencia de la formación; Métodos de formación; Evaluación. Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 91 1. Introduction Training is seen as an essential tool for the improvement of employee performance and for the survival and success of the organizations in a changing world. Determining if the training really gets these attainments and how it is possible to improve them has been the goal of both training effectiveness studies and training evaluation efforts. Training evaluation is in charge of assessing the benefits derived from training, with a clear vocation towards accountability, while training effectiveness studies have been focused on detecting what variables can be affecting training impact with a clear diagnostic and improvement-oriented vocation (Holton, Bates & Ruona, 2000). Most training effectiveness studies use the transfer of training as the main evaluation measure of the impact of training (Alvarez, Salas & Garofano, 2004). Transfer of training is defined by Baldwin and Ford (1988) as ‘the degree to which trainees effectively apply the knowledge, skills, and attitudes gained in the training context to their jobs’ (p. 63). Although other outcomes are also valuable criteria of training effectiveness, such as trainee satisfaction and level of learning, the transfer of training is considered the most powerful indicator to evaluate the effects of training, because it requires the trainees to learn job-related competencies (Velada & Caetano, 2007) -although learning does not guarantee that the transfer of training happens (Hutchins & Burke, 2007)- and because it leads to improvements in employee competencies and, consequently, in organizational performance (Kauffeld & Lehmann-Willenbrock, 2010). In Spain, since the establishment of Foundation for the Training of Employees (Fundación para la Formación Continua) in 1993, a spectacular growth of training plans and training agencies has taken place. The efforts have been focused on the quantitative development of employee training systems and on the consolidation of training habits throughout working life and in organizations. But the impact of these efforts is unknown, because training evaluation has been a deficient practice (Pineda-Herrero, Moreno-Andrés & Durán-Belloch, 2014). In an context of economic crisis, there is a need for measures aimed at reducing expenditures that has driven the government and the companies to question the cost-effectiveness of the approach to training used until now, more interested in the number of workers who have attended any training initiative or in the number of training courses delivered, and to prioritize training impact in the workplace (Renta-Davis, Jiménez-González, Fandos-Garrido & González-Soto, 2014). The concern for transfer of training and for its evaluation is making a dominant appearance on the stage. In terms of transfer, the improvement of training is not just a matter of introducing new measures linked to transfer rates into training evaluation design, but it is also about introducing changes in the design and delivery of training. For that purpose, the existing training transfer research literature is very useful. The so-called ‘transfer of training problem’ –the gap between training and workplace performance (Burke & Hutchins, 2007) - has driven researchers to detect the factors that contribute to enlarge or to bridge this gap. These factors are categorized in the three training inputs identified by the Baldwin and Ford’s (1988) model: characteristics of participants, training design and work environment. The considerable number of factors affecting the transfer of training detected makes it difficult for organizations to pinpoint and manage the factors that are most critical in their case (Cheng & Hampson, 2008; Grossman & Salas, 2011). Moreover, given the ethical issues involved in selecting trainees on the basis of their personal characteristics, many transfer factors cannot be handled to enhance transfer; another factors linked to work environment are difficult to change in order to increase training impact. If the purpose is improving the training of employees, it is necessary to focus on elements that can be handled through training (Russ-Eft, 2002). In this regard, training designrelated transfer factors can be deliberately managed more easily. As Velada, Caetano, Michel, Lyons and Kavanagh (2007) pointed out, training efforts should be made to ensure the transfer of training and therefore organizations should design training programs to include those training interventions that are likely to increase transfer. The socalled ‘transfer design’ refers to what degree training has been designed in such a way that both meets job requirements, ensures learning outcomes, and provides trainees with the ability to Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 92 transfer learning back to the job (Holton, Bates & Ruona, 2000). The idea of ‘transferability of training’ is derived from the transfer design concept. The transferability of the training provided by any training agency will be determined by the extent to which training is designed for being transferred, that is, by the use of training interventions which are pointed out by research on training effectiveness as transfer enhancers by trainers. The study of transferability of training entails a diagnostic evaluation of the potential that the training provided has for facilitating the training content transfer to the workplace, and consequently it can address proposals of change. Moreover, the improvement of transferability should be a priority for any training agency and should be undertaken upon when data on the impact of training are not available. On the other hand, the research on training effectiveness found many training interventions that, considered on an individual basis, influence the transfer of training to the workplace. But a particular training intervention, no matter how efficient it has proved to be, may be insufficient for learning and transferring in view of the complexity of transfer process (Blume, Kard, Baldwin & Huang, 2010). A series of effective training interventions, supplementing the effects thereof, would be necessary in order to enhance transfer possibilities (Culpin, Eichenberg, Hayward & Abraham, 2014). But there is no research on the way in which these training interventions are combined by the trainers when they plan training courses. In this work, a study of the transferability of training provided by the agency in charge of the public servants training in the region of Andalusia (Spain) – the Andalusian Public Administration Institute (Instituto Andaluz de Administraciones Públicas, IAAP) was carried out. The main goal of this study was to explore how trainers use training interventions affecting the effectiveness of training, i.e., what kind of training interventions trainers decide to use in order to meet training objectives and the degree to which transfer mechanisms are included in the intervention design of the training that they provide (Alvarez, Salas & Garofano, 2004; Velada et al., 2007). Thus, it is possible to detect strengths and weaknesses that guide changes in order to improve the transferability of training provided by this agency. On the other hand, this study attempted to explore if, from trainers perspective, effective training interventions can be grouped in some way, i.e. discover if the training design-related transfer factors are organized by the trainers in a particular way, reflecting latent training methodological approaches. Since the effectiveness of training interventions can depend on learning goals, i.e., whether training is focused on learning job-related skills or on acquiring knowledge aimed at understanding and acting differently in work environment (Lim & Johnson, 2002; Nikandrau, Brinia & Bereri, 2009), another goal of this study was to determine whether the type of training objective affects the use of diverse training approaches by trainers. 2. Method 2.1. Context and participants The IAAP is a public organization which provides a training service to the nearly 50,000 employees of the Public Administration of Andalusia. The IAAP counts with a qualified team of training professionals in charge of the analysis of training needs, overall planning, management and evaluation. In order to develop the overall planning, the IAAP also has a fairly stable team of external professionals from different fields (economics, statistics, information technologies, law and so on), selected as experts, who are hired to act as teachers in the training courses. These external professionals are not professional trainers but they play an essential role throughout the instructional design process (Burke & Hutchins, 2008). Therefore, in order to study the transferability of training provided by this agency, a key task was to know what training interventions facilitating transfer these trainers-teachers make use when they design and deliver training courses. Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 93 All trainers who had acted as teachers during the last three years in the training courses provided by the IAAP received an email invitation to complete an online survey on effective training interventions they tend to use. The questionnaire completed was sent in anonymously. The response rate was calculated at 49.3% (300/609). Of the 300 study participants, 60% were male and 93.3% had university degrees. The average age of the respondents was 44.9. Participants had been working an average of 13.3 years in their professional field and 10.2 years participating as trainers in training courses. It is remarkable that 36.3% of participants have not been trained on training. 2.2. Training design-related transfer factors used as indicators of transferability With a view to developing a questionnaire which allowed us to ask trainers´ use of training practices enhancing transfer -and thus, to study the transferability of the training provided by them-, research literature on training design-related transfer factors was reviewed. Only those effective training interventions that fall within the decision-making spheres of trainer as teacher were selected. Three lines of training intervention design were used to organize the inventory of effective practices used as indicators: a) The trainees’ organization and workplace characteristics (needs, scenarios, challenges and tasks) are used by trainers to design training courses. It includes effective training interventions such as: - Trainers set job performance-related goals, i.e. trainers set training objectives oriented to improve the employees’ performance in their workplace (Hutchins & Burke, 2007). - Trainers clearly inform trainees what performance is expected to occur on the job when the trainees apply training content at their workplace; thus, trainees notice a clear link between the training content and their workplace (Burke & Hutchins, 2007; Gegenfurtner, Veermans, Festner & Gruber, 2009). - Trainers select training content relevant to the job, i.e. the trainees must see a close relationship between the training content offered by the trainer and the tasks they encounter on the job (Burke & Hutchins, 2007; Gergenfurtner, Veermans, Festner & Gruber, 2009; Holton, Bates & Ruona, 2000; Liebermann & Hoffmann, 2008). - Trainers set training objectives which are aligned with organizational goals (Alvarez, Salas & Garofano, 2004; Burke & Hutchins, 2007; Donovan & Darcy, 2011; Lim & Johnson, 2002). - Trainers use job learning situations (cases, examples, problems) that reflect the job context where learning is to be applied (Burke & Hutchins, 2007; Grossman & Salas, 2011). b) Trainers use training strategies in accordance with adult learning principles (Alvarez, Salas & Garofano, 2004; Baldwin & Ford, 1988; Blume et al., 2010; Culpin et al. 2014; Donovan & Darcy, 2011). It contains the following effective training interventions: - Trainers detect trainees’ learning needs to set learning objectives accordingly (Lim & Johnson, 2002; Martin, 2010; Nikandrau, Brinia & Bereri, 2009). - Trainers permit discussion and the exchange of current knowledge and experiences by the attendees (Kauffeld & Lehmann-Willenbrock, 2010; Leberman, McDonald & Doyle, 2016). - Trainers are aware of the diversity of learners and, accordingly, they use a wide range of different methodologies -a complex methodologyin each learning situation (Baldwin & Ford, 1988; Culpin et al., 2014; Machin & Fogarty, 2003). Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 94 - Trainers use teamwork as a learning situation (Leberman, McDonald & Doyle, 2016). In order to improve transfer possibilities, organizations should create work environments that promote collaboration among workers (Hawley & Barnard, 2005; Massenberg, Spurk & Kauffeld, 2015). To that end, the use of group activities during training offers opportunities to practice the skills needed for successful collaboration (Culpin et al., 2014; Homklin, Takahashi & Techakanont, 2014) and, therefore, teamwork-based learning situations improve, indirectly, the possibilities of transfer. c) Trainers use methods and instructional techniques that facilitate the transfer of training when they plan learning interventions (see reviews such as Burke & Hutchins, 2007; Russ-Eft, 2002). It includes the following effective practices: - Trainers model ideas or behaviors that are to be taught by them (Alvarez, Salas & Garofano, 2004; Burke & Hutchins, 2008; Grossman & Salas, 2011; Russ-Eft, 2002). - Trainers use the guided discovery method. This active learning method forces trainees to explore and experiment with the task to infer and learn concepts, rules and strategies (Burke & Hutchins, 2007; Russ-Eft, 2002). - Trainers provide opportunities to practice new skills and knowledge (Alvarez, Salas & Garofano, 2004; Kaufield & Lehmann-Willenbrock, 2010; Libermann & Hoffmann, 2008; Lim & Johnson, 2002; Russ-Eft, 2002). Trainees can be asked in practical tasks to apply training content in the training context. But they can be asked to apply it in their job context or plan how they will use new skills and knowledge at their workplace (development of a plan of action) as follow-up techniques (Martin, 2010). - Trainers use variable examples, including positive and negative examples (Burke & Hutchins, 2007; Machin & Fogarty, 2003). These training interventions lead to improve generalization to novel situations (Russ-Eft, 2002). - Trainers use feedback, i.e. trainers provide information to the learner on his/her current and desired performance in training tasks during training (Burke & Hutchins, 2007, 2008; Culpin et al., 2014; Russ-Eft, 2002; Van den Bossche, Segers & Jansen, 2010) or post-training (Velada et al., 2007; Martin, 2010). - Trainers assess the trainees’ learning outcomes after training. Trainees will be more motivated to learn and learning retention will increase if they know that they are accountable for the training they receive (Burke & Hutchins, 2008; Kontoghiorghes, 2001). 2.3. Procedure for developing the questionnaire With these indicators, an initial questionnaire was elaborated containing 19 Likert-type items to know training interventions facilitating transfer which are used by trainers (1 meaning ‘Never’ and 5 meaning ‘Usually’). Since employee training perspectives may vary depending on business sector (Donovan & Darcy, 2011), critical to this study was the development of survey items representative of culture of employee training in public sector. For this reason, the questionnaire was analyzed by five professional trainers of the IAAP. They proposed to change the wording of several items and two new items were included. One of these items dealt with the use of teamwork as a learning situation because there was a great interest in improving the markedly individualistic culture of Public Administration organizations in Spain. The use of teamwork has an indirect impact on the transfer of training, as it has already been mentioned. The second item dealt with the use of the lecture as instructional technique; it is not a method supporting transfer of training but the professional trainers thought that it was usually used by trainers and the questionnaire needed to reflect the practices of the latter. Finally, the questionnaire was composed of 21 items, but one of them does not reflect any transfer factor. Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 95 One additional question about the importance given to each type of learning objective was added. Trainers were requested to prioritize (main objective, secondary objective or not an objective) both training objectives (acquiring knowledge aimed to understanding and acting differently in workplace and learning job-related skills) in the courses they designed. 2.4. Data analysis The reliability of the instrument was assessed via the calculation of Cronbach’s alpha coefficient for all Likert-scale items, excluding the item dealing with the use of lectures since it is not related to transfer; coefficient alpha was measured at 0.87. Descriptive statistics were computed for all variables. Later, the 21 items from the questionnaire were subject to a principal components analysis (PCA) using SPSS to find if the original variables (training interventions) are organized in a particular way reflecting latent variables (training approaches). Three items with low correlations and low communalities were eliminated (‘I set job performance-related goals’, ‘I clearly inform attendees what they must do and produce on the job as a result of training’, ‘I use the lecture as the main methodological strategy’). Later, a multivariate analysis of variance (MANOVA) was conducted to examine whether the use of the training approaches by trainers varies in function of the priority given to training objectives. 3. Results Descriptive statistics about the training interventions frequency of use by trainers surveyed are reported in Table 1. Three teacher decision-making spheres have been used to display the results and, within each sphere, the items are ranked by mean value. Mean value of four of the five items included in first decision-making sphere was greater than 4, therefore, trainers estimated that they are well aware of the trainees’ organization and workplace characteristics when they design training courses. Nevertheless, 30.4% of trainers never or sometime set training objectives to meet trainees’ organizational goals. Training interventions derived from adult learning principles showed a more moderate frequency of use in training design and delivery than prior dimension, but exhort the exchange of knowledge and experiences among attendees that was the most used intervention in this sphere. The use of teamwork as learning situation and the use of a complex methodology were not used or were utilized scarcely by nearly half of the trainers. Table 1. Means table (and SD) for the 21 items of the questionnaire Use of organization and workplace characteristics Mean SD I clearly inform participants what they must do and produce on the job as a result of training. 4.37 0.64 I use learning situations (examples, cases, problems) that resemble the trainees’ job characteristics (tasks, challenges and scenarios). 4.33 0.68 I set trainees’ job performance-related goals. 4.18 0.68 The training content I select is closely linked to the trainees’ workplace needs. 4.06 0.76 The training objectives I set are aligned with the trainees’ organizational goals. 3.35 0.89 Use of adult learning principles Mean SD I encourage participants to exchange knowledge and experiences. 4.14 0.69 I provide participants with opportunities to discuss problems and perspectives. 3.93 0.85 I detect the trainees’ learning needs in order to set learning objectives accordingly. 3.40 0.85 I use a variety of different teaching methodologies with a view to meeting the different demands of the diversity of trainees. 3.18 0.93 Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 96 I propose assignments to work as a team. 3.14 1.00 Use of methods and instructional techniques Mean SD I make variable examples of how to use training content. 4.37 0.64 I use the lecture as the main methodological strategy. 4.18 0.75 I propose exercises to practice training content in training contexts during training. 4.14 0.84 I model ideas or behaviors I teach (demonstrations). 3.98 0.85 I provide the trainees with feedback about their performance at practical tasks. 3.88 0.87 I make negative examples. 3.65 0.87 I use guided discovery method. 3.64 0.89 I propose activities to practice training content in the trainees’ workplace. 3.33 0.97 I propose exercises in order to assess what trainees have learnt. 3.33 0.99 I ask trainees to plan how they will use new skills and knowledge in their workplace (plans of action). 2.75 0.98 I provide post-training coaching to the participants. 2.40 1,01 As regards the methods and instructional techniques, the frequencies of use were very variable. There were more trainers who used training interventions linked to in-the-room training process (lecture, to make examples, exercises to practice training content in-the room, demonstrations) more frequently. Those training interventions extending teaching action outside the classroom to support transfer process (post-training coaching, plans of action, activities to practice training content in workplace) were used less frequently. These less frequent three training interventions are strategies that make it possible to customize the learning transfer process to the specific characteristics of the trainees’ workplace. This homogeneous treatment of learners can also be seen in a less frequent use of two adult learning principles: the detection of trainees’ learning needs in order to set learning objectives accordingly and the use a variety of methods with a view in meeting diverse demands of the learners. Finally, assessing trainee learning was not a habitual practice among the trainers surveyed. Regarding training objectives, learning new skills was identified as the critical objective by 53.7% of the trainers and gaining knowledge was the main objective for 49.3%. The very small number of trainers who chose the response ‘Not objective’ in each training objective drove us to eliminate these cases in subsequent analysis. PCA was deemed appropriate: the Kaiser-Meyer-Olkin Measure of Sampling Adequacy was 0.87 and the Bartlett’s Test of Sphericity was statistically significant (p< 0.001). PCA revealed the presence of four factors with Eigenvalue above 1, explaining 54.6 per cent of the variance in total. It was decided to retain the four factors and a Varimax rotation was performed. The rotation solution can be seen in Table 2. The first rotated factor, which amounted to 18.3% the total variance, deals with six training interventions that involve carrying out, guiding and assessing practical tasks as a mean to promote learning and transfer. It was named ‘task-focused approach’. The second rotated factor, which amounted to 15.3% of the total variance, includes six training interventions aimed at meeting the demands derived from the needs and the characteristics of specific learners and of their work contexts. It was called ‘responsive approach’. Journal for Educators, Teachers and Trainers JETT, Vol. 7 (1); ISSN: 1989-9572 97 Table 2. Varimax rotated factor matrix Training interventions description Factor 1 2 3 4 Exercises to practice in training context. .78 .09 .17 -.01 Activities to practice at their workplace. .63 .29 .19 -.07 Feedback on their performance of practical tasks. .78 .12 .11 .05 Guided discovery method. .52 .26 .41 .14 Assignments to work as a team. .55 .28 -.06 .44 Exercises to assess what trainees have learnt. .64 .07 .03 .23 Detecting trainees’ needs in order to set objectives. .29 .54 .11 .13 Training content is relevant to workplace needs. -.07 .63 .26 .15 Training is aligned with organizational goals. .02 .66 .14 .18 Development of plans of action. .37 .59 -.01 .17 Post-training coaching. .14 .59 .10 -.15 Meeting demands of diversity of learners through a complex teaching methodology. .23 .52 .15 .10 Models of ideas or behaviors. .20 .32 .53 -.30 Variable examples of how to use learning. .35 .04 .71 .10 Learning situations that resemble job features. .07 .27 .69 .17 Negative examples. .00 .13 .66 .18 Opportunities to discuss problems and perspectives. .11 .09 .19 .72 Encouragement to exchange knowledge and experiences among trainees. .10 .19 .14 .72 Note: The values of rotated factor loadings > .45 are written in bold. The third factor of the Varimax rotated factor matrix, named ‘modeling-focused approach’, amounted to 11.9% of the total variance and included four training interventions aimed at giving trainees an insight of how to use the training content. The fourth and final rotated factor, which amount to 9.1% of the total variance, reflects a participative training approach. It was formed by only two training interventions: the trainer exhorts learners to exchange knowledge and experiences and the trainer provides attendees with opportunities to discuss problems and perspectives. This training approach involves seeing trainees as adults having their own career and formed opinion. The items were grouped together according to the PCA and means scores for the new four factors calculated. The teaching approach more frequently used was the modeling-focused approach (M=4.1, SD= .65), closely followed by the participative one (M=4.04, SD= .77). Mean value of the task-focused approach was 3.74 (SD= .79). The responsive scheme was the least frequently used approach (M=3.36, SD= .69) . Since trainers indicated their intention of seeking both types of training objectives giving different priorities to them, significant effects of training objectives on the set of training approaches were sought through MANOVA. The extent of inequality of sample sizes was very small and the Box´s test showed that the assumption of homogeneity of covariance matrices was met (F=1.227, p= .183). The Shaphiro-Wilk test showed that multivariate normality is not fulfilled in all subgroups. With these conditions, Pillai´s trace was used as multivariate test which showed (anyway, four multivariate tests obtained the same Fand p-values) that the priority given to learning skills (F=7.423, p< .001) or to acquiring knowledge (F=3.863, p= .004), considered on an individual basis, did have a significant effect on all four training approaches together as dependent variables, but the interaction between independent variables had not effect. In other words, the concomitant use of training approaches is different when learning jobrelated skills is the main or a secondary goal of training courses and the same applies when the training objective is acquiring knowledge. But there are not differences when the interaction between knowledge and skills objectives is considered. Univariate tests for the effect of the training objectives on each of the dependent variables (Levene´s test verified the homogeneity of variances for all groups) showed that knowledge objectives had a significant effect on the