Supporting and representing Learning Design with digital tools: in between guidance and flexibility

This is an Author’s Original Manuscript of an article published by Taylor & Francis in Technology, Pedagogy and Education on 29 Jan 2020 available at https://www.tandfonline.com/doi/full/10.1080/1475939X.2020.1714708. Please, cite as: Pozzi, F., Asensio-Perez, J. I., Ceregini, A., Dagnino, F. M., Dimitriadis, Y., & Earp, J. (2020). Supporting and representing Learning Design with digital tools: in between guidance and flexibility. Technology, Pedagogy and Education, 29(1), 109- 128. Supporting and representing Learning Design with digital tools: in between guidance and flexibility The research field of Learning Design (LD) has been active for some time now, but a number of open questions remain for the scientific community to solve. In particular, the paper tackles the issue of understanding: 1) how to support the different phases of the LD process, 2) what representations should be used in the various steps, and 3) ultimately, to what extent should digital LD tools be structured or flexible, either guiding the teacher/designer or - by contrast - leaving her free to pursue her own design path and style. These issues have been core concerns in the LD research community over the years. In the effort to investigate these open questions in depth, the authors have proposed an LD tool called the Pedagogical Planner. This tool has been evaluated in authentic contexts with the ultimate goal of providing input for the ongoing debate. Evaluation has focused on the perceptions and actual usage by teachers, generating significant evaluative data to be used as a spur for further reflection on LD. Keywords: Learning Design; representation; conceptualization; authoring; implementation. Introduction Over recent decades, researchers in the Learning Design (LD) field have been striving to facilitate innovation in teaching and learning processes through effective support for the complex task of conceptualizing and elaborating activity plans that can be enacted, shared and repurposed (Conole, 2012; Mor & Craft, 2012; Persico & Pozzi, 2015). Following different conceptual approaches, researchers have proposed and tested a range of different methods and technological tools, with varying degrees of success (Prieto et al., 2013b; Persico et al., 2013; Celik & Magoulas 2016). In spite of these efforts, teachers’ adoption of LD tools and approaches in their everyday practice is generally reported to be low (Berggren et al., 2005; Neumann et al., 2010; Griffiths et al., 2011; Mor et al., 2013; Mor & Mogilevsky, 2013; Prieto et al., 2014). One of the main reasons behind low adoption lies in the fact that it seems highly challenging to provide adequate support and effective technological solutions for a process that is complex in nature, sometimes systematic, sometimes creative, and in any case not always reducible to a number of predefined steps (Winograd, 1996; Masterman, 2013). Among the issues still on the table is the difficulty in providing support for the various design phases, each one being characterized by specific needs and purposes (Celik and Magoulas, 2016). Another difficulty often highlighted by the research community is to define languages able to support the LD process in its various stages and at the same time effectively represent the complexity of the final output of the design process (Dalziel et al., 2013; Pozzi et al., 2016). These difficulties are at the root of the debate about whether a single tool can adequately cover all LD phases and needs or whether a set of tools would provide a more suitable response to the different needs that might arise (Mor, Craft & Maina, 2015; Masterman & Manton, 2011). Moreover, digital LD tools need to have sufficient flexibility to support creativity and to accommodate teachers’ personal design paths and styles, while at the same time bring structure and guidance to the learning design process, especially for less experienced designers (Celik & Magoulas, 2016; Masterman & Manton, 2011). As part of their contribution to this debate, the authors have proposed an LD tool called the Pedagogical Planner (PP). In this paper, we present the results from the evaluation of the PP with the aim of addressing the following research questions: RQ1: How should the different LD phases (characterized by different needs) be supported? RQ2: What kind of representations best fit teachers’ needs in the various stages of the learning design process? RQ3: To what extent should digital tools provide ‘structured’ or ‘flexible’ support (‘structured’ in terms of being able to provide guidance through the design process; ‘flexible’ in terms of being able to support a multiplicity of routes, styles and purposes through the design process)? This paper begins with an overview of the main research threads currently being explored in the LD field. This is followed by a description of the Pedagogical Planner tool, and then a report is given on a recent round of field-testing. Finally, we discuss the results obtained from this evaluation in an effort to answer the three research questions mentioned above. Background As Winograd (1996) and Masterman (2013) point out, conceiving and planning flows of educational activities is a complex process, which does not necessarily follow predefined steps. In an attempt to reduce this complexity to a more manageable level, some researchers have identified ‘phases’ or ‘stages’ in the LD process. Those proposed by Beetham (2008), for example, are creating a design, instantiating/ setting up the learning environment, realizing/running the design, and reviewing/ reflecting on the design. More recently, Pozzi et al., 2016 have identified the following phases: • conceptualization of the design idea (defining learning objectives, identifying content area/s to be addressed, and choosing the most appropriate pedagogical strategies); • planning and authoring the flow of activities (including association of the educational resources and tools that learners are to use); • delivery of the resulting design (from a single activity to a whole course) and enactment with learners through implementation within some kind of (digital) environment such as a Learning Management System. The definition of these phases derives from a European research project called METIS 1 , funded within the European Community’s LLP programme. METIS brought together a number of researchers exploring different avenues in LD, allowing them to compare approaches, find areas of convergence and, on that basis, devise new solutions. One such outcome was the identification of the above core LD phases, which formed the backbone for the project’s subsequent development of an LD platform (see below for further details on this) (Asensio-Pérez et al., 2017). Accordingly, this paper adopts the categorization and terminology agreed in METIS. As already mentioned, in the last decade researchers working in the LD field have sought to propose digital tools capable of supporting one or more than one of these phases. An interesting review, timeline and categorization of these tools is provided by Celik and Magoulas (2016), encompassing: authoring & sharing tools; assessment planners and learning analytics; reflection tools and pedagogical planners; delivery tools; and, repositories. 1 http://www.metis-project.org/index.php/it/ Without any ambition to be exhaustive, to better contextualise the discussion set out in this paper, it is worth noting here some of the main LD tools specifically designed to support pedagogical reflection and foster teachers’ creativity in conceiving new educational paths. Examples include the Course Map (Conole, 2012), the 4SPPIces approach (Pérez-Sanagustín et al., 2012), the 4Ts model (Pozzi & Persico, 2013), and Persona Cards (Chacón-Perez et al., 2015), which are all examples of tools aimed to support conceptualization of new designs. Other tools, by contrast, address activity planning and delivery to learners (Earp et al., 2013; Muñoz-Cristóbal et al., 2012; Persico & Pozzi, 2015; Pozzi et al., 2016). Examples are Collage (Hernández-Leo et al., 2006), WebCollage (Villasclaras- Fernández et al., 2013), CADMOS (Katsamani, M. & Retalis, 2012), OpenGLM, (De Liddo, et al., 2011), EDIT2 (Sobreira & Tchounikine, 2012), which are mainly intended for authoring designs and packaging them for delivery to learners. Many of such authoring tools draw on the IMS-LD specification (IMS Global Learning Consortium, 2003), which was developed to allow the representation of designs in a machine interpretable way. Other LD applications like LAMS (Dalziel, 2003) and CeLS (Ronen et al., 2006) have been developed not only to facilitate the authoring of designs, but also with particular concern for allowing these to be run online. Indeed, LAMS can be used as a stand-alone e-learning application in its own right, or can be integrated into popular LMS as a sort of plugin providing advanced LD capabilities. CeLS, on the other hand, is dedicated specifically to the design, implementation and management of learning activities inspired by the principles of social constructivism. A final example, LdShake (Hernández-Leo et al., 2011), is devoted in particular to the sharing and repurposing of learning designs (Hernández-Leo et al., 2011). This proliferation of tools that cover only part of the spectrum of LD phases is sometimes seen as a sign of fragmentation in the LD field (Mor et al., 2015). Some researchers have suggested to take advantage of this variety and richness, by using various tools, instead of striving to find one single tool that fits for all purposes (Mor et al., 2015; Masterman & Manton, 2011). In this latter vein, a number of the above mentioned tools have recently been integrated into the ILDE (Integrated Learning Design Environment) (Hernández-Leo et al., 2014; Asensio-Perez et al., 2015), an aggregated LD platform developed as part of the METIS project introduced earlier. The main aim of the project was to tackle dispersion in the LD field by building a one-stop shop offering access to leading LD tools, which designers can pick and choose from depending on their needs. While the project generated positive results in terms of teachers’ acceptance of the ILDE (Asensio-Pérez et al., 2017), additional effort is required to broaden and consolidate acceptance of ILDE among target users, and to ensure it has a significant impact on teaching practice. The above-mentioned examples include ‘structured’ tools i.e. those imposing pre-determined steps in the quest to guide the design process, as well as others that are more ‘open’ and flexible, i.e. intended to accommodate different design routes and styles. In addition, they implement a wide variety of different representation forms (Dalziel et al., 2013), but none of the current notational forms seems to be capable of accommodating the whole range of needs, in particular that of making a design readily understandable to others (colleagues or learners) and machine readable at the same time (Masterman & Manton, 2011). Indeed, Pozzi et al. (2016) argue that, given the different priorities involved, a ‘one size fits all’ representation mode is not really practicable. Rather, they advocate the use of multiple representations to cover the different LD phases, provided that the transition from one to another is smooth for the user. Masterman & Manton (2011) examine representations teachers might need during the design process to support creativity and conceptualization of new design ideas. These authors advocate that, in this particular phase of the process, digital mind maps could be highly supportive, as these resonate with the manual mapping teachers tend to use when they conceptualize without any digital support. Further contribution to the debate has come from the efforts numerous researchers have made to observe and better understand teachers’ actual design practice (Bennett et al., 2015; Celik & Magoulas, 2016; Masterman & Mason, 2011; Oliver, 2006). This has been done on the assumption that the better we understand teachers’ current practice, the more effectively LD tools will be in supporting them. According to these researchers, any LD tool should reflect what teachers traditionally do when they design (Masterman & Mason, 2011) and be grounded on teachers’ actual practice (Masterman et al., 2013). As described in the following sections, all these aspects are still open challenges and have been considered during the development of the Pedagogical Planner, which is illustrated below. This allows us to use the data from the evaluation of this tool as a spur for further reflection, especially on the three research questions already specified in the Introduction, namely: RQ1: How should the different LD phases (characterized by different needs) be supported? RQ2: What kind of representations best fit teachers’ needs in the various stages of the learning design process? RQ3: To what extent should digital tools provide ‘structured’ or ‘flexible’ support (‘structured’ in terms of being able to provide guidance through the design process; ‘flexible’ in terms of being able to support a multiplicity of routes, styles and purposes through the design process)? The Pedagogical Planner The design and development of the Pedagogical Planner represents a long-term undertaking carried out over a decade and spanning various research projects (Bottino et al., 2011; Pozzi et al., 2015). The description of the iterative and user-centered approach adopted to develop the tool is out of the scope of this paper, and is reported in Earp et al. (2013). Here it is important only to stress that the tool is grounded on teachers’ actual practice. Its design and implementation was based on direct observation of teachers’ design practice, leading to the formulation of user requirements and the consequent development of functionalities resonating with teachers’ traditional ways of doing things, as strongly recommended by various authors (Bennett et al., 2015; Celik & Magoulas, 2016; Masterman & Mason, 2011; Oliver, 2006). The overall design and development process was also informed by theory; the research team drew especially on those studies in the literature where teachers’ learning design habits and procedures are investigated (Bennett et al., 2015; Bennett et al., 2008; Norton et al., 2005). In the following, we list the main requirements that have been taken into account during the development of the PP. These represent the authors’ response to the gaps that have emerged in the literature (highlighted in the section above), as well as to needs that emerged from teachers’ practice: • The tool should be able to support the main LD phases (Conceptualization, Authoring and Implementation) and allow a smooth passage and iterative switch (if needed) among the phases (Celik & Magoulas, 2016); • Considering that these phases cover different needs, the tool should be able to support each of them, by providing specific representations able to accommodate the various design needs (Pozzi et al. 2016); • In particular, during Conceptualization, the phase in which the designer’s creativity is most prominent, teachers normally use graphical representations such as mind maps and so the tool should be able to support this feature (Masterman & Manton, 2011); • The tool should allow conceptualization of new designs starting from consideration for the type of cohort of students that will be targeted, the learning outcomes they will need to reach, and the main contextual constraints (Bennett et al., 2015; Bennett et al., 2008; Norton et al., 2005) • Given that the Authoring phase is where the designer brings order out of the (possible) chaos of the Conceptualization phase, and where conceptualization ideas crystalize in the form of an ordered sequence of activities, the tool should be able to represent learning activity flows, possibly by offering multiple types of activities so that a rich learning path can be presented; • The tool should offer both flexibility and structure, by allowing partial usage and non-systematic ways of proceeding, but also by providing guidance to those who need to be scaffolded (Celik & Magoulas, 2016; Masterman, 2013; Masterman & Manton, 2011; Winograd, 1996). These features have recently been integrated into the latest version of the PP, which we shall now illustrate. Context and method of the PP evaluation study After the most recent phase of development of the PP, we carried out a study to evaluate the solutions that had been adopted and to explore teachers’ reactions to the offered functionalities. The context and method used in the evaluation are described below. Context The study was conducted in Autumn 2016 as part of two Continuing Professional Development events held in Italy, one in Genoa and the other in Pescara. A total of 39 primary and secondary teachers took part, 25 in Pescara and 14 in Genoa. Given that the two events were similar as far as objectives and proposed activities, and involved very similar target populations, they are treated here as one and the same experience and the combined results are reported as a whole. The aim of these two-day training events was to introduce teachers to the Learning Design field, to illustrate the variety of available methods and tools, and to allow them to familiarise with the Pedagogical Planner. Informative sessions alternated with practical design activities in which teachers were asked to collaboratively conceptualize, author and implement designs. The main learning strategy adopted for the training path was peer-based discussion, negotiation and production, through which teachers re-examined both their past and current practice. All the participants were quite experienced teachers with some experience using digital technology in class. Despite this, they all defined themselves as ‘novices’ in the field of Learning Design and declared they had never been introduced to any digital LD tool before. Method In order to evaluate the PP functionalities, the authors decided to use the notion of “acceptance”, as defined in the Technology Acceptance Model (TAM). The reasons for adopting this particular model, rather than alternative instruments, were twofold: its wide-scale adoption in the educational technology field for predicting user acceptance of new technologies; and also because it seemed to fit well with the purposes of this study and had already been successfully used by the authors in similar contexts. According to this model, the two main indicators that can predict acceptance of a technology are perceived ease of use and perceived usefulness (Davis, 1989). As further explained in the following, in addition to these indicators, the authors also decided to consider actual usage of the tool functionalities by teachers, as this could help better understand and possibly interpret teachers’ perceptions. Thus, the evaluation questions were formulated as follows: EQ1: To what extent do teachers accept PP support for the LD main phases, namely Conceptualization, Authoring and Implementation? EQ2: To what extent do teachers accept PP support for different LD representations, namely natural language, digital mind maps and activity flow diagrams? EQ3: To what extent might teachers be prepared to adopt the PP? Figure 5 below illustrates how the results obtained for each evaluation question contribute to the research questions mentioned at the beginning of this paper. Specifically, the results obtained under EQ1 will provide inputs to the research question about how to support different LD phases and the corresponding needs (RQ1). EQ2 results will further discussion about the suitability of different representation forms (RQ2). Lastly, these results together with those obtained for EQ3 will spur discussion on the extent to which digital LD tools should be structured or flexible (RQ3). Figure 5. Research questions, evaluation questions and indicators. A questionnaire was administered at the end of each training event to gather data on teachers’ profiles and their perceptions about the ease of use and usefulness of the various functionalities of the Pedagogical Planner. The questionnaire included a set of closed questions to which participants were required to respond with a rating based on a Likert-type scale (from 1=low to 5= high). In each case, respondents were offered the opportunity, if they wished, to write comments in open text fields, explaining and justifying their quantitative answers with more qualitative information about positive or negative aspects of the PP. Moreover, system tracking data and direct observation of teachers provided input on the actual use of system functionalities, as measured by the indicators of ‘actual usage’ in Fig. 5. We also analysed the 25 designs, which were collaboratively produced by the teachers attending the training events reported here. Data analysis and main results In this section, data concerning the three evaluation questions are presented. EQ1: To what extent do teachers accept PP support for the LD main phases, namely Conceptualization, Authoring and Implementation? Table 1 provides an overview of the main results obtained about perceived ease of use and usefulness of the PP’s main functionalities with Quartile 1, median and Quartile 4 given for each one. The median test was applied to establish whether teachers’ evaluations diverged significantly from three in the five-point scale. Q1 Median Q4 Exact Sig. (2-tailed) Conceptualization – ease of use 3 4 4 .000 * Conceptualization – usefulness 3 4 4 .019 * Authoring - ease of use 3 4 4 .000 * Authoring – usefulness 3 3 4 .015 * Implementation - ease of use 2 3 4 1.000 Implementation – usefulness 3 3 4 .093 Table 1. Perceived ease of use and usefulness of the main PP functionalities. As far as Conceptualization is concerned, outcomes for both ease of use (Q1 = 3, Median = 4, Q4 = 4) and usefulness (Q1 = 3, Median = 4, Q4 = 4) are > 3 with a significant p (Exact p < .001 and =.019, respectively). A similar result was obtained for the Authoring functionality, with ease of use (Q1 = 3, Median = 4, Q4 = 4; p<.001) and usefulness (Q1 = 3, Median = 3, Q4 = 4; p=.015), both significantly > 3. While outcomes for Implementation are encouraging, they are not statistically significant either for ease of use or for usefulness. Analysis of the actual usage of the various functionalities for Conceptualization revealed that in all but three of the produced designs teachers made extensive use of the Population and Context fields; they gave detailed descriptions of their prospective target, together with prerequisites for students, as well as the expected context of delivery, including information about the setting and equipment necessary for carrying out the intended activities. This confirms findings in other studies (Bennett et al., 2015; Masterman & Mason, 2011) about the way teachers typically start conceiving a new design. We also observed that 17 designs out of the 25 included a map representing the contents, which was often a focus for discussion within the groups; in a couple of cases, the map was instead used to represent learning objectives. In 13 designs, teachers defined the objectives of the overall design, and all the designs include indications about the prospective metrics to be used for student assessment. Fourteen designs also contained indications about the tools that should be used during the delivery. As far as the Authoring functionalities are concerned, 15 designs included a representation of activity flow; these designs contained an average of 5.6 activities each. Teachers used the group branching option extensively (in 7 designs activities are structured around three groups of students, while in 4 designs students are divided into two sub-groups). The remaining designs (4) are composed of ordered sequences of activities. Within the single activities, usage of the fields was quite scattered; the teachers preferred to focus on the structure of the activity flow, rather than on description of the single nodes. This may have been the result of time constraints imposed by the training events, which probably didn't allow them to go into the single activities in any detail. Implementation was the least used functionality, with only three designs being implemented in Moodle. Once again, time constraints were probably responsible. In any case, observation of the groups revealed that teachers perceived this stage as the most ‘delicate’ and ‘technological’, so most of them preferred to leave this passage for their more technical-minded colleagues in the group to deal with, rather than trying to do it themselves. One last interesting behaviour that was observed regarded teachers’ skipping from one field to another within the same phase, rather than jumping from one phase to another. In particular, in the Conceptualization area, teachers used the mind map to trigger group discussion and then jumped to the text fields to fill them in, then came back to the map and then back to the text fields, and so on and so forth. The same happened within the Authoring area: while discussing the contents of one activity, the teachers jumped to the graphical representation of the activity flow, modified it and then came back to the single activities. EQ2: To what extent do teachers accept PP support for different LD representations, namely natural language, digital mind maps and activity flow diagrams? As explained, we also gauged user perceptions of the different representation modes the PP makes use of. The main results are reported below in Table 2. Q1 Median Q4 Exact Sig. (2-tailed) Natural language - ease of use 3 4 5 .004 * Natural language - usefulness 3 3 5 .004 * Digital mind map - ease of use 3 4 5 .000 * Digital mind map - usefulness 3 4 5 .000 * Activity flow diagrams - ease of use 3 4 4 .000 * Activity flow diagrams – usefulness 3 4 4 .004 * Table 2. Perceived ease of use and usefulness of the main PP representations. As far as natural language descriptions are concerned, both ease of use (Q1 = 3, Median = 4, Q4 = 5) and usefulness (Q1 = 3, Median = 3 , Q4 = 5) were rated significantly higher than 3 (Exact p = .004 in both cases). The same applies to mind maps, with rates for ease of use and usefulness (Q1 = 3, Median = 4, Q4 = 5; p<.001) significantly higher than 3. The response for activity flow diagrams was similarly positive: ease of use was rated as significantly higher than 3 (Q1 = 3, Median = 4, Q4 = 4; p<.001), as was usefulness (Q1 = 3, Median = 4, Q4 = 4; p=.004). Data on usage of the representations have already been reported for EQ1 above and are not repeated here. EQ3: To what extent might teachers be prepared to adopt the PP? In addition to these data, teachers were also asked to give a rating about how useful they considered the PP to be as a whole, and also about its potential usefulness for colleagues. As Table 3 below shows, while the median rates for these questions are encouraging (both Q1 = 3, Median = 3, Q4 = 4), only the Exact p concerning usefulness of the PP as a personal tool is statistically significant (p=.035). Q1 Median Q4 Exact Sig. (2-tailed) PP as personal tool– usefulness 3 3 4 .035 * PP as a tool for others– usefulness 3 3 4 .405 Table 3. Perceived usefulness of the PP as a personal LD tool and as a tool for colleagues. Respondents also had the opportunity to explain their quantitative answers by responding to open questions. These were analysed for occurrences of positive and negative terms, as shown in Table 4 below. Positive aspects Number of occurrences Useful 10 Complete 7 Clear 7 Intuitive 6 Flexible 4 Negative aspects Number of occurrences Graphical aspects to be improved 13 Help in Italian needed 6 Bridge through Glue!-PS to be simplified 5 Menu of the activity flow to be simplified 4 Table 4. Occurrences of positive and negative terms in open responses regarding the PP. Analysis of the open answers confirms “usefulness” as the main perceived advantage brought by the PP. In the following, we report some of the responses regarding PP usefulness: “[The PP] lends homogeneity to the design and constitutes a first check of the validity of the design idea”. “It is useful, because it allows you to proceed in a less rough and episodic way”. “I think the system could also help to make the design clear to students”. “The Authoring phase is represented in a really effective way, as it helps rationalise things and it meets teachers’ needs”. Regarding the aspects identified as needing further improvement, the graphical appearance of the PP is judged to be old-fashioned and not very appealing. The issue of Italian language support arose because during the first of the two events, it was unfortunately not possible to provide the Help in the teachers’ native language; this obviously caused some problems but was fixed in time for the second event. Moreover, according to our respondents, the bridge with Glue!-PS needs to be simplified (Implementation phase), as does the menu to access the activity flow options in the Authoring phase. Discussion This paper illustrates the Pedagogical Planner (PP) in its present version, which is the result of a long-term iterative design process carried out across different contexts (Earp et al., 2013). In the following, we examine the main results obtained from the evaluation round and make suggestions for the research questions outlined at the beginning of the paper. RQ1: How should the different LD phases (characterized by different needs) be supported? Following the indications provided by Bennett et al. (2015), through the evaluation of the PP, we have tried to provide answers regarding the forms of support teachers find most acceptable and best fit their needs in the various phases of the design process. To do so we have sought to ground the development of the tool on teachers’ actual practice, as recommended by Masterman & Manton (2011) and by Oliver (2006); the evaluation round reported here is central to that effort. As a result of this, the functionalities currently provided by the PP resonate with teachers’ established LD practices, as advocated also by Celik & Magoulas (2016). Accordingly, the Conceptualization functionality of the PP allows teachers to focus on students’ characteristics, which is what they usually do. Moreover, the definition of expected learning outcomes is also supported, and these can be mapped thanks to an ad hoc graphical representation, which is what Bennett et al. (2015) suggest on the basis of their own observations of teachers. The definition of the contextual constraints is also allowed, as suggested by Masterman (2013). Thus conceived, the Conceptualization area of the PP has gained the appreciation of teachers, and the extensive use they made of the provided functionalities confirms these are important forms of support in this creative step of the design process. At the same time, we have seen that each group of teachers followed its own particular way of proceeding; for example, some started from definition of the target population, while others preferred to begin from expected learning outcomes. This possibility to choose from where to start the conceptualization process, coupled with partial usage of fields, is thus a desirable affordance for LD tools. As far as the Authoring functionalities are concerned, it seems that graphical representation of activity structure is a valuable feature. By contrast, textual description about each node (activity) in the flow has been less used in our context. This feature might be designated as mandatory only in cases where sharing is the main purpose, remaining an option where the purpose of the design process is more maieutic, i.e. for self-reflection. The Implementation functionality was less extensively explored in our field experiences and the data collected do not allow us to make strong indications about how exactly to support this phase. Certainly, the higher level of technical expertise required at this stage of the process may be somewhat intimidating for teachers, so it would be useful to study more in depth how to simplify this step from a technical point of view. Another inference that we can derive from the present study is the importance of allowing teachers to follow their own way of proceeding, such as skipping from one field to another (especially within a single design phase) without the restriction of mandatory fields or pre-determined steps. Teachers demonstrated that they appreciate collaborative learning design editor based on patterns. Educational Technology & Society, 9, 58–71. IMS Global Learning Consortium (2003). IMS Learning Design v1.0 final specification. Available online at http://www.imsglobal.org/ learningdesign/ Katsamani, Μ., & Retalis, S., (2012). Designing a Moodle course with the CADMOS learning design tool, Educational Media International, 49(4), 317-331. Masterman, L. (2013). The challenge of teachers’ design practice. In H. Beetham & R. Sharpe (eds.), Rethinking Pedagogy for the Digital Age, 2nd edn, Routledge, London, pp. 6477. Masterman, E., & Manton, M. (2011). Teachers’ perspectives on digital tools for pedagogic planning and design. Technology, Pedagogy and Education, 20, 227–246. Masterman, E., Walker, S., & Bower, M. (2013). Computational support for teachers’ design thinking: its feasibility and acceptability to practitioners and institutions. Educational Media International, 50, 12–23. Mor, Y., & Craft, B. (2012). Learning design: reflections upon the current landscape. Research in Learning Technology – Supplement ALT-C 2012 Conference Proceedings, 20:19196, pp. 85-94. Mor, Y., Craft, B., & Hernández-Leo, D. (2013). Editorial: The art and science of learning design. Research in Learning Technology Supplement 2013, 21: 20224. DOI: 10.3402/rlt.v21i0.20224. Mor, Y, Craft, B. & Maina, M. (2015). Introduction – Learning Design. Definitions, Current Issues and Grand Challenges. In M. Maina, B. Craft, & Y. Mor (eds.), The Art & Science of Learning Design (pp. ixxxvi). Rotterdam: Sense Publishers. Mor, Y., & Mogilevsky, O. (2013). The learning design studio: collaborative design inquiry as teachers’ professional development. Research in Learning Technology, 21:22054. Muñoz-Cristóbal, J. A., Prieto, L. P., Asensio-Pérez, J. I., Jorrín-Abellán, I. M., & Dimitriadis, Y. (2012). Lost in translation from abstract learning design to ICT implementation: a study using Moodle for CSCL. In Proceedings of the European Conference on Technology-Enhanced Learning (EC-TEL’12) (pp. 264-277). Springer Berlin Heidelberg. Neumann, S., Klebl, L., Griffiths, D., Hernández-Leo, D., de la Fuente-Valentın, L., Hummel, H., Brouns, F., Derntl, M., & Oberhuemer, P. (2010). Report of the results of an IMS Learning Design expert workshop. International Journal of Emerging Technologies in Learning, 5, 58–72. Available online at http://goo.gl/rN18H3 Norton, L., Richardson, T.E., Hartley, J. Newstead, S. & Mayes, J. (2005). Teachers’ beliefs and intentions concerning teaching in higher education. Higher Education, 50(4), 537-571. DOI: 10.1007/s10734-004-6363-z Oliver, M. (2006). New pedagogies for e-learning?. ALT-J, 14: 133–134. DOI:10.1080/09687760600668453 Pérez-Sanagustín, M., Santos, P., Hernández-Leo, D. & Blat, J. (2012). 4SPPIces: A case study of factors in a scripted collaborative-learning blended course across spatial locations. International Journal of Computer-Supported Collaborative Learning, 7(3), 443-465. DOI: 10.1007/s11412-011-9139-3 Persico, D. & Pozzi, F. (2015), Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46: 230–248. DOI: 10.1111/bjet.12207 Persico, D., Pozzi, F., Anastopoulou, S., Conole, G., Craft, B., Dimitriadis, Y., Hernández-Leo, D., Kali, Y., Mor, Y., Pérez-Sanagustín, M., & Walmsley, H. (2013). Learning design Rashomon I - Supporting the design of one lesson through different approaches. Research in Learning Technology Supplement 2013, 21: 20224. DOI: 10.3402/rlt.v21i0.20224 Pozzi, F., Asensio-Pérez, J. I., & Persico, D. (2016). The case for multiple representations in the learning design lifecycle. In B. Gros, Kinshuk, Maina M. (eds.), The architecture of ubiquitous learning: learning designs for emerging pedagogies, 171- 196, Springer. Pozzi, F., Ceregini, A., Dagnino, F., Ott, M., & Tavella, M. (2015). Supporting the “whole learning design life-cycle” through the Pedagogical Planner. In D4Learning Conference Proceedings, Aalborg, 17-20 November 2015. Pozzi, F., & Persico, D. (2013). Sustaining learning design and pedagogical planning in CSCL. Research in Learning Technology Supplement 2013, 21: 20224. DOI:10.3402/rlt.v21i0.17585. Prieto, L., Asensio-Pérez, J., Dimitriadis, Y., Gómez-Sánchez, E., & Muñoz- Cristóbal, J. (2011). GLUE!-PS: A multi-language architecture and data model to deploy tel designs to multiple learning environments. In K. Carlos, G. Denis, G. Raquel Crespo, W. Fridolin, & W. Martin (eds.), Towards ubiquitous learning (pp. 285–298). Berlin: Springer. Prieto, L. P., Asensio-Pérez, J. I., Muñoz-Cristóbal, J. A., Dimitriadis, Y. A., Jorrín-Abellán, I. M., & Gómez-Sánchez, E. (2013a). Enabling teachers to deploy CSCL designs across distributed learning environments. IEEE Transactions on Learning Technologies, 6(4), 324-336. Prieto, L. P., Dimitriadis, Y., Craft, B., Derntl, M., Émin, V., Katsamani, M., Laurillard, D., Masterman, E., Retalis, S. & Villasclaras, E. (2013b). Learning Design Rashomon II - Exploring one lesson through multiple tools. Research in Learning Technologies Supplement 2013, 21: 20057. DOI: 10.3402/rlt.v21i0.20057 Prieto, L. P., Tchounikine, P., Asensio-Pérez, J., Sobreira, P., & Dimitriadis, Y., (2014). Exploring teachers’ perceptions on different CSCL script editing tools. Computers & Education, 78, 383–396. Ronen, M., Kohen-Vacs, D. & Raz-Fogel, N. (2006). Structuring, Sharing and Reusing Asynchronous Collaborative Pedagogy. In Proceedings of International Conference of the Learning Sciences, ICLS 2006, Indiana University, Bloomington IN. Sobreira, P., & Tchounikine, P. (2012). A model for flexibly editing CSCL scripts. International Journal of Computer-Supported Collaborative Learning, 7, 1–26. Villasclaras-Fernández, E., Hernández-Leo, D., Asensio-Pérez, J. I., & Dimitriadis, Y. (2013). Web Collage: an implementation of support for assessment design in CSCL macro-scripts. Computers & Education, 67, 79-97. DOI: 10.1016/j.compedu.2013.03.002 Winograd, T. (1996). Bringing design to software. New York, NY: ACM Press.