A model of good practice in training of trainers.

Moraru, Adela ; Tomuletiu, Elena-Adriana ; Solovastru, Alina 等

1. INTRODUCTION

The large scope of the article is to emphasize the importance of implementing trainings for trainers (TOT) that have as objective not only to teach knowledge, attitudes and skills relevant for pedagogical purposes but also to give the possibility to evaluate, during training, the pre- and post-training competencies in developing a session plan. We obtain in this way an accurate, behaviorally relevant measure of efficiency of TOT, based on facts, not only on subjective opinions by means of questionnaires at the end of the training. The objective of this article is to describe the trainer of trainers program conducted locally with a group of 15 professionals during which such a performance evaluation was given.

Evaluating the effectiveness of the training needs primarily the choice of evaluation criteria. Although there are a lot of models to training evaluation (Day et al., 2001) the most popular model of training evaluation is that of Kilpatrick (1959, 1976, 1996), based on asking the question "Effective in terms of what? Reactions, learning, behavior or results?". Thus the objectives of training determine the most appropriate criteria for assessing the effectiveness of training. A reaction criterion, which uses self-report measures, represents the trainees' affective and attitudinal responses to the training program (Winfried et al., 2003). Although they are largely used in applied settings, they tell nothing to a researcher about: a) how much the participants learned from the program (learning criteria); b) changes in their job-related behavior (behavioral criteria); c) the utility of the program to the organization (results criteria). Learning criteria are measures of the learning outcomes of the training; they are not measures of job performance. They are operationalised by using paper-and pencil tests and performance tests. In contrast, behavioral criteria are measures of the actual on the job performance and can be used to identify the effects of training on actual work performance. They are operationalised by supervisor ratings or objective indicators of performance. Although these concepts are linked, there are situations where behavioral criteria are influenced by other variable that impend the transfer of the new skills to the job. Finally, result criteria (productivity, company profit) are the most distal and macro criteria used, operationalised by using utility analysis estimates (the dollar value gained by engaging in specified personnel intervention including training).

So, there are at least four different ways to operationalise the effectiveness of training. In case of training of trainers we consider important to evaluate not only the subjective impressions of the participants (reaction criteria) but also the learning criteria, because we don't have the opportunity to see the effect of the real job performance of the participants, as in the behavioral criteria. We present therefore a model of good practice in which relevant learning criteria were implemented to evaluate the effectiveness of the training, using an instrument with behavioral aspects of pedagogical skills to be learned.

2. INFORMATION

We present a case study of training for trainers course, developed by a local organization (Tirgu-Mures, Romania). The course lasted for three months, including a number of 96 hours of training, structured in three learning modules: a two week interactive training course, two months of e-learning platform with weekly forum discussion on planned learning material and two weeks dedicated to the evaluation of the pedagogical competencies, during two videotaped simulations of training offered by every participant. The overall goal of the training is to strengthen the participant's knowledge and skills for effective training.

As a specific objective, it was intended that by the end of the course the participants should be able to describe the basic characteristics of the competence-based learning; identify the principles of adult learning; create and maintain a positive learning environment; use verbal and non-verbal facilitation skills to enhance communication in training; develop and use effective audio-visual training tools and equipment; describe interactive training techniques and kinds of learning appropriate to each one; develop a training session design based on criteria for an effective learning experience; develop and use evaluation tools based on knowledge and skills; describe the components of group dynamics; formulate adequately the general and operational objectives for a session plan.

The objectives for e-learning sessions were specific to the content material: identify functions of the trainer, knowledge and use of didactical resources, knowledge and skills in assertivity training, motivation, communication, knowledge of neurolinguistic programming (NLP).

In these courses the training method was completely based on participatory techniques such as small group discussions, role play, case study, brainstorming, and experience-based learning. "Warm-up" sessions were conducted to start morning and afternoon programs. "Ice-breakers" were used to energize and dynamise the learning process and group dynamics.

The program contained sessions on learning styles, adult learning, communication skills, didactical resources, assertivity training, interactive training methods, expositive methods, audio-visual training tools and equipment, demonstration, coaching, evaluation techniques for knowledge/skills and designing training session. There was no formal evaluation at the end of the course, in order to obtain a general satisfaction of the participant with the training. There were periodical forms of evaluation during the course, using specific techniques for knowledge evaluation (true/false questions, for example, multiple choice answers and so on).

The most important was the third module, whose major objective was for participants to develop a 5 minute simulation of training session. After one week there was another simulation session, this time lasting for 15 minutes. In both sessions the participants were allowed to choose their own topic for training. There was a pre-established evaluation form (Tab. 1) that took in consideration all the phases of the training session: elaboration, execution (both introduction and development), conclusion and evaluation of the session.

Each phase has behavioral indicators for execution, noted on a 4 Likert point range, from 1--to improve to 4--achieved. As it can be seen from Table 1, the instrument covers all important aspects during the session, starting with a clear definition of pedagogical objectives, elaboration of the session plan, preparation of evaluation instruments and continuing with elements that facilitate an agreeable atmosphere, assuring group participation and feedback from participants, adequate use of pedagogical auxiliaries and adequate checking of the learning results. Personal characteristics like emotional control and capacity of expression, control of the situation were also included as relevant for a successful training.

This behavioral assessment procedure was administered to each participant after their first and second simulation of training. Both sessions were videotaped and, at the end of their individual presentation each participant was subjected to, through the film content, to the evaluation of group members, self evaluation and trainer's evaluation. In this way multiple feedbacks could be obtained regarding individual performance and skills developed. Using the same instrument at the end of the second simulation gave the participants the opportunity to notice, by comparing the two forms, the improvement they made by practice and feedback.

At the end of the first simulation, there were a few aspects to improve in participants, mainly those related to emotional control because there was an initial anxiety during filming, time management or control of situation due to inexperience in group management. A few of participants did not have time to draw the conclusion and evaluation of the training because of deficient time management. Almost all participants, no matter how inexperienced they were showed great creativity in choosing the exercises, icebreaking and evaluation forms.

At the end of the second simulation there was a clear improvement of previous behavioral deficiencies, all of the participants showed improved performance, greater control of the situation, good time management and improvement in emotional control.

3. CONCLUSION

Most studies concerning effective training of trainers are based on questionnaires given to participants at the end of the session and evaluating the frequencies of their responses on different items (Yolsal et al., 2003; Beir-Holgersen et al., 2004). Although they show a positive impact and a desire of the participants to be involved in similar experiences, these kinds of evaluations shows nothing about their real competencies developed as a result of the training (self progress evaluation). It is important to consider other assessment methods in order to allow not only declarative but also procedural knowledge to be evaluated in training. The present article showed a model of good practice in the field, offering multiple feedbacks (self evaluation and group evaluation) to assess pedagogical competencies of the trainees, in two successive sessions, so that practice and feedback could make a real difference. There seems to be a noticeable improvement in the quality of the training, both in inexperienced and experienced participants.

Teaching of adults is, in many ways, different form teaching of children. Adult learning can be reinforced by reflection--at the end of a session the learner ask himself/herself some basic questions about what has been learned or how a skill has been developed and how this might affect their behavior next time (Dobson, 2007). This is something that a teacher also need to learn to do at the end of the every session, probably more that the learner.

4. REFERENCES

Beier-Holgersen, R.; van Diest, R. & Resti, C.V. (2004). Evaluation of training for trainers courses for Oromia (Ethiopia) health professionals schools' tutors, in International Nursing Review, Vol. 51, pg. 185-192, Available from: http://search.ebscohost.com/ Accessed:

2009-05-21

Day, E.; Arthur, W. & Gettman (2001). Knowledge structures and acquisition of a complex skill, Journal of Applied Psychology, Vol. 86, pg. 1022-1033, Available from: http://search.ebscohost.com/ Accessed: 2009-05-23

Dobson, M. (2007). Training the trainers, in Anaestesia, Vol. 62, Suppl. 1, pg. 96-102, Available from: http://search.ebscohost.com/ Accessed: 2009-05-22

Winfried, A.; Edens, P. & Bennett, J. (2003). Effectiveness of training in organizations: a meta-analysis of design and evaluation features, Journal of Applied Psychology, Vol. 88, No. 2, pg. 234-245 Available from: http://search.ebscohost.com/ Accessed: 2009-05-21

Yolsal, N.; Bulut, A. & colab. (2003). Development of training for trainers programmes and an evaluation of their effectiveness in Istanbul, Turkey, Medical teacher, Vol. 25, No. 3., pg. 319-324 Available from: http://search.ebscohost.com/ Accessed: 2009-05-21

Tab. 1. Trainer's competency evaluation form

Session
phases Behavioral aspects

Preparation 1. Definition of pedagogical objectives
 2. Knowledge of subject
 3. Elaboration of session plan
 4. Preparation of pedagogical means and
 auxiliaries
 5. Preparation of evaluation instruments
Introduction 6. Creation of a favorable atmosphere
 7. Communication of objectives
 8. Contextualization of subjects
 9. Control of pre-requirements
Development 10. Self confidence/emotional control
 11. Methods and techniques
 12. Structure and sequence of the session
 13. Trainees' activity
 14. Motivation of participants
 15. Capacity of active listening
 16. Respect the rhythm of individual learning
 17. Adequate encouragement of participants
 18. Use of pedagogical auxiliaries
 19. Revisions and partial syntheses
 20. Capacity of expression (voice, gesture
 movement, etc.)
 21. Control of the situation
 22. Creativity
 23. Time management
 24. Capacity of answering clearly,
 concisely
Conclusion 25. General synthesis
Evaluation 26. Checking learning results
 27. Communication of learning results