Was it worth the effort? An exploratory study on the usefulness and acceptance of video annotation for in-service teachers training in VET sector

Research on Education and Media Vol. 11, N. 1, Year 2019 ISSN: 2037-0830 – DOI: 10.2478/rem-2019-0014 Was it worth the effort? An exploratory study on the usefulness and acceptance of video annotation for in-service teachers training in VET sector Elena Boldrini a, Alberto Cattaneo b, Alessia Evi-Colombo c aSwiss Federal Institute for Vocational Education and Training, Switzerland, [email protected], https://orcid.org/0000-0003-3405-9740 bSwiss Federal Institute for Vocational Education and Training, Switzerland, [email protected], https://orcid.org/0000-0001-5155-1214 cSwiss Federal Institute for Vocational Education and Training, Switzerland, [email protected], https://orcid.org/0000-0002-2071-3791 Abstract In the field of teachers training of different levels (primary and secondary) and types (in-service and pre-service), exploiting video support for teaching practices analysis is a well-established training method to foster reflection on professional practices, selfand hetero-observation, and finally to improve teaching. While video has long been used to capture microteaching episodes, illustrate classroom cases and practices, and to review teaching practices, recent developments in video annotation tools may help to extend and augment the potentialities of video viewing. Various, although limited, numbers of studies have explored this field of research, especially with respect to in-service teachers training. However, this is less the case for Vocational Education and Training. The study presented here is a pilot experience in the field of in-service teachers training in the vocational sector. A two-year training programme using video annotation has been evaluated and analysed. The dimensions investigated are teachers’ perceptions on the usefulness, acceptance and sustainability of video annotation in teaching practices analysis. Results show a very good acceptance and usefulness of video annotation for reflecting on practice and to deliver feedbacks. Implications for the integration of a structural programme of analysis of practices based on video annotation are presented. Keywords: video annotation; in-service teacher training; Vocational Education and Training; feedback; professional practices Introduction Different scholars have recognized video in general as an effective support for analysing, reflecting and – consequently – improving professional practices (e.g. Avsar et al., 2015 in medicine; Tripp & Rich, 2012 in teacher training). Professional competence development requires in fact constant and explicit reflection on action (e.g. Kember, McKay, Sinclair, & Wong, 2008) to consolidate effective routines, to develop knowledge on actions and to recognize improvements needed for enhancing the effectiveness of professional behaviour (Saedon, Salleh, Balakrishnan, Imray, & Saedon, 2012), with the final intent to reach a given standard and expected performance (Voerman, Meijer, Korthagen, & Simons, 2012). Reflection on action can be fostered through receiving formative feedbacks provided by an agent such as a peer, a tutor or an experienced colleague, and progressively through self-generated feedbacks (Brinko, 1993). In a situation-based instructional approach, the reflective analysis on authentic video-recorded situations experienced by professionals constitutes a valuable means for fostering both hetero-provided feedbacks and self-observation by the professional herself, thus ensuring the development of observational skills on one’s real actions. This process can be additionally reinforced by the use of video as a tool for supporting focused, concrete and situational evidence-based feedback (Brinko 1993, Fukkink, Trienekens, & Kramer, 2011). In the field of teachers training of different levels (primary and secondary) and types (in-service and pre-service), exploiting video support for teaching practices analysis is a well-established training method as in other professions (Colasante, 2011, Bonaiuti, 2012; Gaudin & Chaliès, 2015). Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 101 At this regard, research has pointed out the affordances of this tool as it allows (a) a postponed access to an authentic situation in its complexity and richness (Gaudin & Chaliès, 2012); (b) to see in a slower pace the interactions and actions during the class that otherwise would not have been considered because of the great amount of information the subject manages while practising; (c) to select specific parts and clips out of a more complex situation and to develop the capacity to reflect on a more evidence-based manner (Tripp & Rich, 2012); (d) finally, video viewing enhances the possibility of a greater connectivity and alternation among professional practice (teaching) and training course (Gaudin& Chaliès, 2015). While video has long been used to capture microteaching episodes, illustrate classroom cases and practices, and to review teaching practices, recent developments on video annotation tools may help to extend and augment the potentialities of video viewing. Such tools make practice documentation possible, and support self-analysis using verifiable evidence. In fact, video annotation is a flexible and sustainable tool requiring minimal video-editing competence for inserting and sharing comments and notes, and it allows the following: a) writing and commenting functionalities directly in the video frames, thus reinforcing the reflection on action process (Boldrini & Cattaneo, 2013, 2014); b) peers, tutors and personal remote feedbacks insertion and exchange on a same and specific classroom event; c) the possibility to have access to reports of the feedbacks and analysis to perform an overall reflection process on one’s practice. Nowadays, a wide range of video annotation tools is available (Picci, Calvani & Bonaiuti, 2012; Rich & Hannafin, 2009), both online and offline, providing modelling situations or possibilities to analyse authentic teaching scenarios, thus proving to be a flexible tool for different professional uses (Cattaneo, van der Meij, Sauli, 2018; Sauli, Cattaneo & van der Meij, 2018). Since Rich and Hannafin published their work on video annotation as technology for scaffolding, structuring and transforming teachers’ reflections (2009), various, although limited, numbers of studies have explored this field of research. In their review, Torregosa, Díaz-Martin & Ibáñez-Cubillas (2017) showed that since 2009–2017, there was an increasing numbers of studies on video annotation and teacher education, yet the quantity of them is still scarce, especially if referred to the specific field of teacher education such as the vocational one. Additionally, as the authors reported, the field of research about in-service teachers training is insufficient. On the same vein, Picci, Calvani & Bonaiuti (2012, 2014) and Calvani, Bonaiuti & Andreocci (2011) pointed out that giving the potentialities of video annotation, it is relevant to collect both empirical data on teachers’ perspective on its use and on the implications for teacher’s self-reflection capacity and peers interactions. This paper aims therefore to contribute to fill the gap in knowledge with respect to the use of video annotation in vocational teachers’ training, with a specific focus on in-service teachers, working on an instructional scenario characterized both by collaborative and individual analysis on teachers’ practices. 1.Context of the study In Switzerland, a Federal Law regulates teachers’ training within the Vocational Education and Training (VET) sector, including secondary upper level and vocational tertiary level teachers. The Law on Vocational Education and Training indicates that teachers attend the training while they are already enrolled in schools with an assignment contract. This means that teachers’ training is organized between the alternation of training and professional practice in schools. On this backdrop, the alternation between real practice in VET schools and training courses should be exploited as an occasion to work on the connection between authentic experiences and declarative knowledge inputs, in a perspective of situation-based didactics (Boldrini, Ghisla, Bausch, 2014). The complete training curriculum of VET teachers is characterized by a 3-year (six semesters) part-time diploma course (1800 hours of training for a total of 60 ECTS) consisting of different kinds of modules (for a total of 12, each 5 ECTS). Specifically, 10 modules aim at developing specific skills of the teacher’s competence profile (structuring training sequences, assessing learnings outcomes, collaborating within the school institution, managing communication in the classroom, etc.). The remaining training modules cross the entire training curriculum. They aim at providing the teachers with a transversal development of their professionalism and identity. This path ends with the evaluation of the teaching practice and with the drafting of the final practical project, based on the teacher’s personal training activity conducted based on the teacher’s interests and identified teaching concerns. Within the transversal modules, there is also an individual and personalized pedagogical guidance, that is, training visits in class once a semester by expert tutors and by disciplinary teachers, with the ultimate goal of encouraging the teacher's reflection on their practices and – on this basis – to further develop their competences. On top of that, it has to be considered that within the VET teachers’ training curriculum, while generally very appreciated, teachers can profit from limited visits in class as well as occasions for receiving and providing direct feedbacks on classroom practices. In this perspective, some innovations in the training curriculum have been considered and tested to offer more frequent and direct opportunities to provide teachers with video and video annotation supported feedback on teaching practices. Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 102 2.Aims Based on these premises, authors developed a pilot study within the VET teachers training diploma in Switzerland and particularly within the transversal modules leading to the final evaluation of the teaching practice. The study represents a first exploratory project on the use of a video annotation tool for individual and collaborative analysis of authentic teaching practices, with the final intent to evaluate its effects and appreciation so that it can be adapted, developed and included in the training curriculum in a structural way. In other words, the study first aimed to test the feasibility in terms of perceived usefulness, acceptance and sustainability of the instructional model to be steadily integrated in the teacher education program. Second, the pilot intended to evaluate the impact of video annotation on teachers’ reflective capacity, as well as to assess the effects of using video annotation on the quality and quantity of reciprocal feedbacks among participating teachers and on one’s own practice. In the present paper, authors will focus on the analysis and discussion of the first goal. 3. Sample The participants were 36 VET in-training teachers. They were part of two different diploma groups (18 teachers each), both concluding their training in June 2018. Professional subject teachers and baccalaureate subject teachers composed the two groups. For each diploma group, participants were split into two conditions based on their preferences, but in any case with the indication of a maximum of 10 participants for each condition. Half of the sample (18 participants) provided feedbacks and conducted a self-observation on video-recorded lessons supported by the video annotation tool. The remaining participants (18 teachers) delivered feedbacks and conducted the self-observation task by writing a report through direct in-class examination of peers’ and one’s own lessons. 4. Procedure Teachers took part in the study and attended a training programme over four half training days distributed in four semesters, for a total of 16 presence hours and about of 25 hours of individual work time. The course included first three hetero-directed feedback activities (teachers worked in couples) on colleagues’ teaching situations, and finally a self-analysis activity on one’s own lesson. More in detail and as presented in Figure 1, the participants took part in a training pattern in which training activities are provided in presence (four meetings), and activities of analysis in distance learning mode (three analysis activities in the intersessions). In the following paragraphs, we describe the overall programme. During a first meeting, in addition to a presentation of the pedagogical and ethical contract, in a plenary setting, participant analysed the video of a lecture by a teacher not present in the training group and then defined as “unknown teacher”. The aim was to identify the recurrent activities that a teacher normally carries out in his practice and to define in a shared way a list of about 10–15 relevant didactic activities (e.g. giving feedback to students, introducing the lesson, etc.). Starting from this list, which would serve as a guide for the whole course, we started with the analysis on the first two activities from the list (i.e. introducing the lesson and giving formative feedbacks to students). Each teacher was asked to watch a video on the subject, always by a teacher outside the group, and to identify elements on which to provide feedbacks (the identification of elements was conducted via video annotation in the video annotation condition or via pen-and-paper annotation in the other condition). Feedbacks were then shared and discussed in the group. At this point, a theoretical input was given on how to effectively provide feedback to a colleague, following recommendations by Hattie & Timperley (2007) and by Brinko (1993), as well as tutors’ guidelines. At the end of the first meeting, participants were prepared for the intersession task: teachers collaboratively defined two activities on which to focus their attention for the observation and feedbacks to be delivered during the intersession period. The participants were then separated into work pairs. Each teacher in the couples video-recorded a part of her/his own lesson in which the defined activities were carried out; the video of each teacher was inserted within a video annotation software so that each teacher in the couple could write within an online sessions feedbacks to the colleague and vice versa (Figure 2). When both teachers had finished entering their feedback, one of the two course tutors would enter their feedbacks. In the direct observation condition, participants followed the same procedure, with exception of the use of video annotation software for feedback sharing. They worked in pairs and during the intersessions, they observed an in-class colleague’s lesson. Feedback on the examined lesson were then delivered through a written report on the selected foci of their observation. In the second meeting in presence (which took place in the following semester), the analysis conducted where then shared and generalized to define best practices with respect to the selected activities. Then, a second couple of instructional activities were identified from the list and deemed the object for the next intersession period. In the third meeting, the analysis and the feedbacks of the intersession were shared and consolidated into general guidelines for good practices. Before the last meeting, each teacher video-recorded and self-analysed their own practice, focusing on another couple of teaching activities and a longer section of a lesson (20 minutes). Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 103 In the last meeting, the results of the self-analysis were then thematized in the group and finally all the participants of the two conditions conducted the analysis of a whole lesson by an unknown teacher: this last analysis would function as the final measure of the programme. An online satisfaction and perceived usefulness questionnaires were submitted at the end of the session. This programme is characterized by a progression design, starting from hetero-analysis to self-analysis, and from the analysis of clear-cut actions to generalizable and transferable teaching actions. Fig. 1 The overall training model. The video annotation software used in the program is called ivideo (www.ivideo.education). It allows users to transform traditional videos into interactive ones, through hyperlinks (called active points) that enrich the videos with additional multimedia material, and annotations synchronized with the specific segments of the video. These annotations can be private or, as in the case of online sessions created within this training course, shared with other members of a defined group (in this case, the couple's colleague and the course supervisor). As shown in Figure 2, annotations appear on the right-hand portion of the video screen, in the form of comments generated both by the teachers’ peer and by the supervisor. Fig. 2. The ivideo software and the video annotation functionality. 5. Data collected Stemming from the objectives derived by the exploratory nature of the study, the authors aimed, first, to verify the acceptability, usefulness and sustainability of the training programme in terms of the development of didactic skills and Supervisor’s annotation Peer’s annotation Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 104 engendering of reflective skills on the teaching practice. This was done through an online questionnaire submitted to all participants during the last day of plenary training, and through a discussion among course participants and supervisors. The questionnaire consisted of 22 items, of which 11 were open-ended questions and 11 closed-ended questions. In the closed-ended questions, statements with Likert scale answers (from −3: completely in disagreement, to 3: completely in agreement) were provided. The questionnaire was divided into four main parts. The first part was dedicated to the perception of usefulness in relation to the training programme; the second part focused on the usefulness and ease of observing a teaching practice and delivering feedbacks; the third part was dedicated to the work processes (time invested, writing process, etc.) and results (learning outcomes and final assessment of the experience); in the last part, questions were asked about the added values and perceived limits of the two conditions proposed, as well as suggestions for further development of the training programme. Further data were collected with respect to the analysis of the lesson of the unknown teacher (final measure) and with respect to the quantity and type of feedbacks exchanged in the two conditions among the participants (process measure). As mentioned before, in the present paper, authors focus on the results obtained from the questionnaire with respect to acceptability, perceived usefulness, and sustainability of the programme, with the final aim to assess its conditions of effectiveness and aspects to be developed to be steadily implemented into the training curriculum. 6.Results Usefulness. Generally speaking, the results on the usefulness of the training programme are positive. Teachers declared to agree with the general usefulness of the programme with an average mean of 2.14 (SD.640); moreover, teachers think that the seminar allowed them to improve their teaching practice (M2.17, DS.69) and above all they strongly agree that the seminar allowed them to develop reflective skills on their own teaching practice (M2.33, SD.67). Not as significative – but positive nonetheless – is the belief of having developed a method of analysis (M1.92, SD.94) and a reusable method in the future (M1.69, SD1.12). This overview is not differentiated in the two working conditions: both those who have worked with video annotation and those who have worked with in-class observation of their colleague stated a general satisfaction towards the programme (Table 1), without showing statistical differences among conditions (independent samples T-Test). Table 1. Results about perceived usefulness by condition. I think the training programme Conditions N M SD has been useful Video annotation 18 2.17 .707 Direct observation 18 2.11 .583 has allowed me to learn how to teach Video annotation 18 1.94 1.056 Direct observation 18 1.94 .539 has helped me to improve my teaching practice Video annotation 18 2.11 .832 Direct observation 18 2.22 .548 has allowed me to develop my reflective competence on my practice Video annotation 18 2.28 .752 Direct observation 18 2.39 .608 has allowed me to learn a method of analysis Video annotation 18 1.67 1.188 Direct observation 18 2.17 .514 .. has taught me a method that I can use in the future, too Video annotation 18 1.44 1.504 Direct observation 18 1.94 .416 Selfand hetero-observation. In terms of usefulness of peers’ observation, there has been a positive perception (M2.33, SD.59); even more so with respect to the possibility of observing peers’ teaching practices (M2.44, SD.61). Interestingly, participants scarcely agree with the belief that self-observation is more powerful and useful than observing colleagues (M.033, SD1.87). Nevertheless, they consider that self-observation and self-analysis are more difficult than hetero-observation (M1.56, SD1.68). That is probably the reason why they agree on the progression from self-observation to hetero-observation (“having had the possibility to observe my colleague’s practice made the selfobservation easier”, M 1.75, SD.99). Table 2. Results about heteroand self-observation by condition. Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 105 Conditions N M SD Receiving a written feedback by a colleague has been useful Video annotation 18 2.22 .647 Direct observation 18 2.44 .511 Observing colleagues’ teaching practice has been useful Video annotation 18 2.39 .608 Direct observation 18 2.50 .618 Having had the opportunity to observe a colleagues before observing myself made the self-observation easier Video annotation 18 1.50 1.150 Direct observation 18 2.00 .767 Self-observation has been more difficult than hetero-observation Video annotation 18 1.00 1.847 Direct observation 18 2.11 1.323 Self-observation has been more useful than hetero-observation Video annotation 18 .39 1.883 Direct observation 18 .28 1.904 Splitting this analysis on the basis of the two conditions (Table 2.), no specific statistical difference has emerged (independent samples T-Test). It is, in any case, important to point out that the only statistical difference concerns the item “Self-observation has been more difficult than hetero-observation”: teachers in the video annotation condition perceived self-observation as less difficult, as they regarded it as the opportunity to view their practice and to direct annotate their reflections on it (MVideoAnnotation1.00, SD1.85; MDirectObservation2.11, SD1.32); t(34)2.075, p.046). Differently, the teachers in the direct observation condition did not have a support to link their analysis to their own practice, thus revealing the difficulty of self-reflecting on one’s own practice without concrete elements on which to anchor a posteriori evidence-based considerations. In general, on these items related to self and hetero-observation reveal that direct observation condition gets slightly higher scores. This is probably due to the fact that the observations in the classroom were richer than those in the video condition and therefore considered a bit more useful. Work process and perceived results. In terms of the average time invested in the programme, the video annotation cohort of teachers and the direct classroom observation cohort invested a crucially different amount of time on their tasks. For each analysis and feedback session (both self-analysis and hetero-analysis), those who worked within the video annotation condition indicated an average time investment of one and a half hours of work (M83.82 minutes, SD54.10); on the other hand, those who directly observed a colleagues’ classroom lesson had to invest an average of more than 4 hours of work. This was due to the travel time dedicated to reach the colleague’s school; to the fact that the observer stayed in the classroom for the whole duration of a teaching unit of 45 minutes, and because the drafting of the report required taking notes and relying on memory recall what was observed (M283.24, SD134.90). Standard deviation values show, in any case, a large variability in the time invested in both conditions, probability due to the different level of deepening in the analysis conducted and to the different organizational conditions. Nevertheless, the overall evaluation of the investment in terms of time has been well evaluated. For the video annotation condition, 16 out of 18 participants reported that the cost–benefit ratio for learning was positive. In three cases, it reported a critical load in terms of time invested with regard to the management of video files and to the preparation of the filming in classroom (consent form to be submitted to the students). In the condition of direct observation, 12 out of 18 positively evaluate the cost–benefit ratio; negative observations go in the direction of onerous investment of time for the drafting of the report and for the practical organization concerning the visits to their colleagues. The majority of the participants, regardless of the type of condition they were involved in (VA Video annotation; DO Direct observation), agrees on the tenet that if you want to reflect [1], and especially if you want to go in depth with reflection [2] on professional practice [3], you have to deserve an adequate amount of time; this investment is appropriate – cost–benefit wise – in the light of the learning outcomes attained and will also generate a positively critical attitude [4]. 1. “Since reflection is necessary, the time spent on it is adequate” (VA). 2. “It depends on the quality and depth of the reflections one makes. If there is actually a reflection supporting the comments, then I think it's an adequate amount of time to dedicate to this activity” (VA). 3. “Yes [it is worthy], because in-depth reflection on practices takes time” (VA). 4. “... the drafting of the report requires an evaluation and reflection on what I have observed. It automatically makes you develop some self-criticism towards your own professional activities, in relation to each aspect observed” (DO). A third question concerned the process of writing (writing to communicate a feedback to a colleague, writing to selfanalyse one’s own practice). Within the video annotation condition, writing is widely recognized as a process strongly Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 106 related to reflection [5, 6] but also with a strong structuring and connecting power [7], allowing to put order in one’s thoughts [6]. 5. “Writing the reflections helps focus on the meaning of the activity and enhance the critical thinking process” (VA); 6. “Writing allows to make order between thoughts and to engender reflections” (VA); 7. “Writing an analysis certainly clarifies the ideas behind it, helps to structure them, and also makes it possible to create connections among findings” (VA). Nevertheless, with respect to the writing process, there are different perceptions in the two groups. Teachers working with video annotation reported a more positive experience, as the writing process was facilitated by the video annotation functionality [8]. Three participants made explicit reference to the added value of the ivideo software in supporting the writing process, and would find it useful to further promote similar experiences within the whole training [9]: 8. “Writing with active points is certainly useful and facilitates reflection compared to the simple act of viewing. It is also a useful way to remember, during the meetings, what we had inserted as comments and encourage reflection in the classroom a few weeks later. I didn't find it difficult to write my own feedback” (VA); 9. “I believe that video annotation facilitated personal reflection and that the development of this practice on the ivideo interface was easy to carry out. It will be interesting to further encourage this type of activity” (VA). In the direct observation cohort, on the contrary, 10 out of 18 participants stated that the process of writing the report on the observations made generated some difficulties and mentioned a certain effort connected to the writing process [10]; although they also recognize is usefulness [10], some mentioned their preference for other modalities of analysis, such as oral discussion [11]. 10. “For me the difficulties were those of expressing myself in a writing format; however writing observations was very useful because it allowed me to remember the various moments of the lesson” (DO). 11. “I had some difficulties with writing. I would have preferred an informal verbal discussion” (DO). Learning outcomes, limits and added values. In the final part of the questionnaire, some questions concerned the perceived usefulness related to one’s learning outcomes, as well as the limits and benefits of the two different conditions. With respect to the learning outcomes, participants of the video annotation condition reported to have learned how to give a feedback (13 persons) and what to focus on during the analysis of a teaching practice (eight persons) [12]. These statements are also present in the direct observation condition, but in this last, teachers focused more on the communication and sharing of feedbacks between colleagues [13]. 12. “Before, I used to just describe, now I look, think and try to imagine if there is a better way to do it” (VA). 13. “Yes. Sensitivity in the communication with the colleague and revaluation of the shared reflection” (DO). In terms of limits and added values, these were differentiated with respect to the condition in which the participants were involved. About the video annotation condition, the limits that have been pointed out were mainly about the risk of losing the general context, the overview and the climate of the class (five participants spoke about this issue); this was also due to the fact that 14. “The possibility of stopping and reviewing segments of the video may lead to focus too much on the details risking losing the general overview” (VA). Moreover, the video recording organization was perceived as cumbersome by seven participants. Finally, three participants underlined that one limit concerned the position of the recording camera in the classroom; its placement can influence the whole interpretation of the teaching situation. On the other hand, the added values of the video annotation modality are represented by the possibility to review the recordings of the lessons several times (seven participants agreed on this tenet), to better handle the organizational aspects of the task (seven participants), to have more time to reflect in depth on the video product (six participants), to have the possibility to observe oneself in a video (5 participants), and to use a non-intrusive (three participants) and innovative technological tool (two participants). In the direct observation cohort, one of the reported limits is the impossibility to review what has been observed (five participants); to manage the organization of the visits between colleagues with different schedules (four participants); the direct observation is perceived as less objective than the video observation, since video provides indisputable evidence of the teaching practice (three participants). On the other hand, participants from the direct observation condition pointed out as the main affordances of their modality the advantage of a direct confrontation with a colleague (six participants); the possibility of direct observation (six participants); the “immediately after” feedback opportunity (two participants); the easy organization of the visit (one participant), and the advantage of not seeing oneself in a video (one participant). Finally, participants had to say if they would have changed the modality chosen at the beginning of the programme. Most teachers (14 out of 18) in the video annotation condition would choose the same condition again; one would mix the two modes; one would maybe change and the other two did not answered. In the direct observation mode, 13 out of 18 would choose again this condition, while 5 would switch to the video annotation condition if given the opportunity. Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 107 The general evaluation of the experience is positive for all the participants. Some of them were enthusiastic [15]; some others proposed to enlarge the training time dedicated to this programme and reported crucial understandings and personal competence development, under the basic result about learning to observe and reflect on professional practices [16]: 15. “The whole process of the seminar helped to open up my horizons and enhanced my analytical skills, hopefully in a positive way. I also learned what to look for in a teaching setting, as the seminar helped focus on important factors of teaching” (VA). 16. “I found this seminar useful as I learned first of all to observe and to reflect” (DO). Conclusions This study has been implemented as a pilot experience in the use of video annotation in the training of in-service VET teachers, an area where research is scarce. The authors have drawn several useful elements from this experience, as far as the structural development and innovation of the training programme within the diploma acquisition course for teachers in the professional sector. Both in the direction of introducing corrective measures into the design of the entire seminar and with the goal of confirming the validity of some crucial affordances regarding video annotation as a tool for analysing professional practices. In general, with regard to the proposed training, it should be noted that both conditions were appreciated by the two cohorts, each for its own peculiarities and based on the attitudes of the participants. Teachers reported positive evaluations in regards to both the usefulness and acceptance of the training model and its sustainability in relation to the cost–benefit ratio. All the participants found taking part in this programme worth the time and efforts. These considerations allow the authors to continue to experiment and optimize the training developed so far. In particular, the video annotation mode showed clear potential in relation to facilitating self-analysis and self-reflection processes – which in this mode was less difficult, in relation to the development of more detailed, situated and focused analyses anchored to actual evidence, and in relation to its sustainability in terms of time and financial issues (no travels necessary, the time invested in the tasks is overall less than the one dedicate to direct observation, and all dedicated to develop reflective feedback). Also, the writing process was perceived as more aligned and functional to the training scenario, as well as very appropriate to support a structured reflective process. As far as the two different working methods proposed are concerned, it seems reasonable to continue to allowing the possibility of having each teacher choose the working method that best suits his/her preferences and attitudes, while being mindful of his/her own concerns about being filmed and having to use technological supports. However, it was also observed that a course that proposes a mix of modes could be appreciated, to develop a set of differentiated and complementary skills in the participants. In relation to the limits of the video annotation mode, some improvements may be introduced. Based on this pilot findings, the authors aim at facilitating the recording and editing procedures of the videos with adequate technical support and previous technical training, as well as to provide dedicate training in the use of video annotation and video annotation tool. In addition, the ivideo software will also implement the possibility of including emoticons in feedbacks and comments to insert in the video, with the ultimate goal of respecting and enhancing the social dimension and relationship of distance analysis that appeared as an added value in the direct observation modality. Moreover, to contain the analysis and video viewing times, a maximum time for video recordings will be set (approximate duration between 10 and 15 minutes). As for the direct observation mode, the difficulties of drafting the report can find a solution in the creation of report models that employ dedicated scaffolding and guidance through guided questions, rubrics, observation grids and a prestructuring of the written report. Despite the fact that the study has limitations related to the size of the sample considered and some results may have been induced by the limited possibility for all to choose the preferred option (as a maximum of 10 participants had been set for each subgroup component of the cohorts), the authors consider that it allowed to make a first step in understanding the applicability of the video annotation in the training of in-service teachers in the vocational sector. Further research is anyway needed to better understand the effects of each mode with respect to the development of a reflective attitude, a crucial competence to be supported in teacher education. In this direction, the analysis of the quality of the written composition, as well as of the capacity to both notice (identify and discern specific details) and make the knowledge-based reasoning behind explicit (see van Es, Cashen, Barnhart, & Auger, 2017). For these reasons, the programme presented here and the above-mentioned corrections are worthy of being further applied, thus widening the field of experimentation. References Boldrini, E. & Cattaneo, A. (2013). Written identifications of errors to learn professional procedures in VET. Journal of Vocational Education and Training, 65(4), 525-542. Was it worth the effort? An exploratory study on the usefulness and acceptability of video-annotation within inservice teachers training? Boldrini, Cattaneo, Evi-Colombo Research on Education and Media. Vol. 11, N. 1, Year 2019 - ISSN: 2037-0830 108 Boldrini, E., & Cattaneo, A. (2014). Scaffolding collaborative reflective writing in a VET curriculum. Vocations and learning, 7(2), 145-165. Boldrini, E., Ghisla, G., & Bausch, L. (2014). Progetti di didattica per situazioni. In G. P. Quaglino (Ed.), Formazione. I metodi (pp. 337-360). Milano: Raffaello Cortina. Bonaiuti, G. (2012). La video annotazione per osservare e riflettere. Form@ re-Open Journal per la formazione in rete, 12(79), 71-83. Brinko, K. T. (1993). The practice of giving feedback to improve teaching: What is effective? The Journal of Higher Education, 64(5), 574-593. Calvani, A., Bonaiuti, G., & Andreocci, B. (2012). Il microteaching rinascerà a nuova vita? Video annotazione e sviluppo della riflessività del docente. Giornale Italiano della Ricerca Educativa, 4(6), 29-42. Cattaneo, A., van der Meij, H. & Sauli, F. (2018). An Empirical Test of Three Instructional Scenarios for Hypervideo Use in a Vocational Education Lesson. Computers in the Schools, 35(4), 249-267. Colasante, M. (2011). Using video annotation to reflect on and evaluate physical education pre-service teaching practice. Australian Journal of Educational Technology, 27(1), 66-88. Fukkink, R. G., Trienekens, N., & Kramer, L. J. (2011). Video feedback in education and training: Putting learning in the picture. Educational Psychology Review, 23(1), 45-63. Gaudin, C., & Chaliès, S. (2015). Video viewing in teacher education and professional development: A literature review. Educational Research Review, 16, 41-67. Hattie, J., & Timperley, H. (2007). The Power of feedback. Review of Educational Research, 77(1), 81-112. Kember, D., McKay, J., Sinclair, K., & Wong, F. (2008). A four-category scheme for coding and assessing the level of reflection in written work. Assessment & Evaluation in Higher Education, 33(4), 369–379. Pérez-Torregros, A.B., Díaz-Martín, C., & Ibáñez-Cubillas, P. (2017). The use of video annotation tools in teacher training. Procedia: Social and Behavioral Sciences, 237, 485-464. Picci, P., Calvani, A. Bonaiuti, G. (2012). The use of digital video annotation in teacher training: the teachers’ perspectives. Procedia: Social and Behavioral Sciences, 69, 600-613. Rich, P., & Hannafin, M. (2009). Video Annotation Tools: Technologies to Scaffold, Structure, and Transform Teacher Reflection. Journal of Teacher Education, 60(1), 52-67. Saedon, H., Salleh, S., Balakrishnan, A., Imray, C. H., & Saedon, M. (2012). The role of feedback in improving the effectiveness of workplace based assessments: a systematic review. BMC medical education, 12. Sauli, F., Cattaneo, A. & van der Meij, H. (2018). Hypervideo for educational purposes: a literature review on a multifaceted technological tool. Technology, Pedagogy and Education, 27(1), 115-134. Tripp, T. R, & Rich, P. (2012). The influence of video analysis on the process of teacher change. Teaching and Teacher Education, 28(5), 728–739. Van Es, E. A., M. Cashen, et al. (2017). Learning to Notice Mathematics Instruction: Using Video to Develop Preservice Teachers' Vision of Ambitious Pedagogy. Cognition and Instruction 35(3), 165–187. Voerman, L., Meijer, P. C., Korthagen, F. A., & Simons, R. J. (2012). Types and frequencies of feedback interventions in classroom interaction in secondary education. Teaching and Teacher Education, 28(8), 1107-1115.