The effects of CLIL on L1 competence development in monolingual contexts

The effects of CLIL on L1 competence development in monolingual contexts Very little research has been conducted on the effects of CLIL on L1 competence development. This article compares CLIL and non-CLIL learners' levels of attainment in Spanish Language and Literature in seven public schools in the South of Spain. Learners' verbal intelligence and motivation were previously matched in order to homogenise the sample. This study focuses on a total of seven public schools (n=271) in the province of Seville, at both primary and secondary level. Students’ performance in L1 (Spanish) was compared to check differences between CLIL and non-CLIL strands. The results suggest that students in CLIL programmes do better in their end-of-year Spanish Language and Literature assessment, compared with students in non-CLIL strands. There is also some evidence to suggest that in schools with early CLIL programmes CLIL students do better in the Spanish Language and Literature assessment than students in schools with late CLIL programmes. Teachers and students surveyed generally believe that CLIL has a positive impact in this area, particularly on students’ understanding of how language works. Keywords: CLIL; communication skills in the L1; language learning; Spanish Introduction CLIL (Content and Language Integrated Learning) is defined as ‘an educational approach in which various language-supportive methodologies are used which lead to a dualfocused form of instruction where attention is given both to the language and the content’ (Coyle, Hood and Marsh 2010: 3). It has its origins in the 1990s, when the term CLIL was coined to refer to a series of teaching methodologies that brought together principles from existing educational approaches such as Content-Based Instruction (CBI), language immersion and bilingual education. There is now a substantial literature seeking to define CLIL and to understand commonalities and differences with earlier approaches (Dalton-Puffer 2011; Ruiz de Zarobe, Sierra, and Gallardo 2011; Fernández- Sanjurjo, Fernández-Costales and Arias 2017). Pérez-Cañado (2012) has suggested that CLIL should be seen as the European counterpart of the American-based CBI. It is certainly true that CLIL has developed mostly in Europe, and most notably in Spain, where it has been seen as particularly appropriate for achieving the European Union’s multilingual policy (European Commission 2004). This is the context of the research reported in this paper. In the years following its appearance, CLIL attracted considerable critical attention. Early research attempted to define the method and outline its main features (Coyle 2008; Coyle, Hood and Marsh 2010; Dalton-Puffer 2011; Pérez-Cañado 2012). In this period, CLIL was conceptualised in very broad terms as an educational approach that could encompass a variety of teaching methods already in use in bilingual, multilingual and immersion programmes in Europe (Ruiz de Zarobe, Sierra and Gallardo 2011). Indeed, CLIL was said to be an ‘umbrella term’ (Mehisto, Marsh and Frigols 2008: 12) covering various teaching methodologies. This allowed different European countries and institutions to adapt it to their particular contexts and educational frameworks, and to tailor it to their own specific linguistic needs. Comentado [E1]: This requires clarification Comentado [H2R1]: Done. As CLIL started to be implemented and developed in several European countries (Coyle, Hood and Marsh 2010), research interest shifted from definition and characterisation to the evaluation of its implementation. Studies began to focus on learning outcomes and to assess students' overall competence development in the foreign language (FL), comparing both CLIL and non-CLIL strands (Admiraal, Westhooff and de Bot 2006; Dalton-Puffer 2011; Lasagabaster 2008). In these studies, research typically showed positive results for CLIL programmes, with CLIL students outperforming non-CLIL students (Dalton-Puffer 2013; Lorenzo, Casal and Moore 2009; Pérez-Cañado 2012). Since then, different lines of research have been developed. One of them focuses on assessing not just the impact on general FL proficiency but on specific aspects of learners' FL competence such as specific communicative language skills or vocabulary or grammar (Lasagabaster 2008; Nieto 2016a, 2016b). Ruiz de Zarobe and Jiménez (2009: 135) provide a complete overview of research on language skills development in CLIL contexts, concluding that ‘under the CLIL approach, some receptive and productive skills seem to benefit more than others: reading, speaking and some writing competencies such as fluency, lexical variety and lexical and syntactic complexity.’ While research generally shows that CLIL learners tend to achieve better results in FL learning than non-CLIL learners, this is not always the case, nor is it always clear why CLIL students outperform non-CLIL students. As Dalton-Puffer (2011: 186) suggests: ‘the question of how much and in what respect CLIL students are better remains of interest, as does the question of why.’ Thus another important line of research considers different variables that may affect the FL learning outcomes of students in CLIL programmes, such as contextual factors, individual learner characteristics and motivation (Lasagabaster 2011; Fernández and Canga 2014; Doiz and Lasagabaster 2014; Sylvén and Thomson 2015; Alejo and Piquer 2016; Arribas 2016; Otwinowska and Foryś 2015; Pfenninger 2016). So far, findings in this area have not been conclusive, indicating that further research is needed to understand the influence of factors such as age, gender, setting (e.g. rural vs. urban), socio-economic status and motivation on the development of CLIL learners’ FL competence. As Nikula et al. (2016: 1) note: ‘CLIL research to date has been mainly characterised by language learning perspectives on learners’ general language skills [in the FL]’ while little is known about the possible effects of CLIL on learners' performance in other areas of the curriculum. CLIL is after all about the integration of both content and language. The impact of CLIL on content subject learning is addressed in another article in the present issue of The Language Learning Journal (see Madrid and Hughes); here, we investigate whether CLIL impacts on the development of learners’ knowledge and skills in the area of the first language (L1). The role of the first and the foreign language in CLIL The impact of CLIL on learners' L1 development has been systematically neglected in the literature (Lasagabaster and Sierra 2010; Dalton-Puffer 2012; Merino and Lasagabaster 2015, Lasagabaster 2017). This is no doubt due to the assumption that CLIL is a methodology that is primarily aimed at improving students' competence in the FL, rather than an educational approach in which the interaction between the languages used stimulates the learning of other curricular content too. Lorenzo, Casal and Moore Comentado [E3]: Are they referring to L2? Comentado [H4R3]: Done. Comentado [E5]: I’m suggesting deleting this as it doesn’t really tell us anything but just repeats the same idea. Comentado [H6R5]: Done. (2009: 435) certainly found that this was the dominant view among L1 teachers surveyed in Andalusia in 2007-2008 and they concluded that: L1 teachers appear reluctant to participate in integration; some even considered CLIL a competitor to L1 learning, in the belief —nourished by a pedestrian view of bilingualism— that different languages represent opposing forces, growing at each other’s expense. However, well-established theories of bilingualism maintainresearches done in CLIL show that the learning of a second language has positive effects on learners’ performance in their L1 due, for example, to the development of language awareness (Cenoz 2015; Dalton-Puffer 2007; Ruiz de Zarobe and Cenoz 2015). Likewise, CLIL theorists agree that CLIL is not only concerned with developing learner’s FL competence, but with the development of general communicative competences (European Council 2001: 13-14), both in the FL and the L1 . As Coyle (2002: 28) explains, drawing on the distinctions made by Cummins (1999), CLIL helps students develop basic interpersonal communication skills (BICS) and cognitive-academic language proficiency (CALP) in both the FL and the L1. Bilingual programmes have a long and established tradition in Spain (Cenoz 2013; Cenoz 2015; Lasagabaster and Ruiz de Zarobe 2010; Pérez-Cañado 2012) so it is perhaps surprising that, relative to other groups of international and European learners, Spanish learners perform comparatively less well both in the first and the foreign language (PIRLS 2011; European Commission 2012; Eurydice 2012; ECCL 2012). Could it be that teaching curricular subjects in a foreign language impacts negatively on L1 skills and knowledge development? It is for this reason that research was undertaken as part of a nationwide research project to look at the impact of CLIL on L1 skills and knowledge. There has been little research on this issue since Marsh (2002: 24) asked the question fifteen years ago: ‘Will the first language develop normally despite a significant amount of instruction time being conducted in another language?’ A number of arguments and considerations underpin the view that L1 and FL competences can develop in parallel, and indeed, should benefit each other; these form the theoretical foundation for CLIL. We review five key points below: Cognitive development According to the literature (Stocco et al. 2014; Olusola et al., 2010 and Le Pichon Vorstman et al. 2009 ), learning more than one language should have cognitive benefits for learners. This idea is central to CLIL; Coyle (2002: 28) for example makes it one of the four key features of CLIL in her ‘4C's model’ along with content, culture and communication: CLIL should cognitively challenge learners - whatever their ability. [ . . . ] Research suggests that these challenges encourage thinking to take place in different languages and at a deeper level of inter-cultural understanding involving both savoir faire and savoir être. Comentado [E7]: Just on speaking performance???? Please review this phrasing and clarify what exactly is being referred to. Comentado [H8R7]: Done. Comentado [E9]: What are these? Please can we have a definition or at the very least a reference. ???? Comentado [H10R9]: Done. Comentado [E11]: The Cummins reference is Cummins, J. 1999. BICS and CALP: clarifying the distinction. ERIC ED438551. https://eric.ed.gov/?id=ED438551 You will need to add this to your references. Comentado [H12R11]: Done. Comentado [E13]: Please state at what level – are we talking secondary learners, primary, adult???? Comentado [MNP14]: PIRLS (fourth grade students: PIRLS 2016 consists of three separate assessments of reading comprehension: PIRLS, PIRLS Literacy, and ePIRLS. PIRLS is a comprehensive assessment of fourth grade students’ reading literacy achievement. Conducted on a regular fiveyear cycle. Eurydice: all education levels. Comentado [MNP15]: No encuentro dónde hacer referencia a que entre los pocos escutudios en algunos se señala que CLIL no afecta al desarrollo de la L1: Admirral et al, 2006 Coyle, Hood and Marsh, 2011 Dalton-Puffer, 2011 Pérez- Cañado, 2012 Evisencias de que mejora: Lorenzo, Casal and Moore 2009 Fernández-Sanjurjo 2017 Comentado [E16]: References please Comentado [MNP17R16]: done Comentado [MNP18]: Coyle 2008 también hace referencia al desarrollo cognitivo: •Develop risk-taking and problem-solving skills in the learners; •Develop study skills, concentration—learning how to learn through the foreign language is fundamental to CLIL; In this view, the cognitive benefits of CLIL should impact both on FL and L1 learning. Thus, teaching content subjects in the FL should not impede, but rather foster, the development of learners’ L1 skills and knowledge. Skills transfer between languages Navés (2009: 28) points to evidence suggesting successful learning of a second language builds on effective L1 development: ‘the lack of continuing first-language development has been found, in some cases, to inhibit the levels of second language proficiency and cognitive academic growth’. Navés refers to Krashen's review of bilingual education (1997: 1) where the latter argues that: …when schools provide children quality education in their primary language, they give them two things: knowledge and literacy. Literacy development in the primary language transfers to the second language. In the same vein, Cummins (2000: 25) contends that ‘the first language must not be abandoned before it is fully developed, whether the second language is introduced simultaneously or successively, early or late, in that process’. In the context of CLIL, Agustín (2009: 112) also acknowledges the influence of the L1 in the process of learning a second or foreign language. Thus, current thinking suggests that rather than competing with FL learning, the development of competence in the L1 potentially strengthens it. The amount, nature and type of input in the L1 and the FL / nature of input Regarding input, most arguments used in CLIL derive from research in the field of second language acquisition (Navés 2009) in particular Krashen's input hypothesis (1985), according to which L2 acquisition can only take place when learners are exposed to sufficient comprehensible input. There are several studies of CLIL in which the amount of FL input is one of the variables considered (Lorenzo, Casal and Moore 2009; Ojeda, 2009; Dalton-Puffer, 2011; Merino and Lasagabester, 2017 and Nieto, 2016) but all of these focus exclusively on the impact on FL, rather than L1, development. Certainly, one of the advantages of CLIL lies precisely in the increased FL input that students receive during instruction compared to regular English as a foreign language (EFL) classes in non- CLIL contexts. At the same time, CLIL may also offer more opportunities for learners to communicate in the FL; in other words, greater ‘output’ and ‘interaction’, both of which are considered important in successful second language acquisition according, respectively, to Swain (1985) and to Long (1996). Indeed, research on CLIL suggests that the success of the methodology is directly related to the number of hours that students are exposed to and, therefore, able to use, work and interact in the FL (Lasagabaster 2010, 2015, 2017; Lorenzo, Casal and Moore 2009). However, a corollary of this is that a balance must be sought in the amount of input that CLIL learners receive in each language, so that exposure to one language is not at the expense of the other. In Andalusia, where the present study takes place, the Plan Estratégico de Desarrollo de las Lenguas en Andalucía (2016) (Strategic Plan for the Development of Languages in Andalucia) states that a minimum of 50% of curriculum time should be delivered in the Comentado [E19]: Please check this revision. Normally, ‘language transfer’ refers to the transfer of linguistic elements from one language to another. I think you’re referring to something rather different here. Comentado [H20R19]: Done. Comentado [E21]: There’s a dissonance between ‘requires’ here and Naves’ rather more nuanced comment ‘in some cases …’ Hence my rewrite. Comentado [H22R21]: OK. Comentado [E23]: This reference does not refer to Krashen’s ‘theory of L2 acquisition’ but rather to Krashen’s review of evidence to support bilingual programmes: hence my rewrite. Comentado [H24R23]: OK. Con formato: Resaltar Comentado [E25]: You need to give the reference. I think it’s 1985 but please check. Comentado [H26R25]: Done. Comentado [E27]: In L1? In L2????? Are you talking just about CLIL studies? Comentado [H28R27]: Done. Comentado [E29]: You talk about ‘several’ but reference only one study. Please check this and provide further references if necessary. Comentado [MNP30R29]: done Comentado [E31]: Which? We really do need the references. Comentado [MNP32R31]: done Comentado [E33]: The references are as follows. Swain, M. 1985. Communicative competence: some roles of comprehensible input and comprehensible output in its development. In Input in Second Language Acquisition, ed. S.M. Gass and C.G. Madden, 235-256. Long, M.1996. The role of the linguistic environment in second language acquisition. In Handbook of Second Language Acquisition, ed. W. Ritchie and T. Bhatia, 413-468. Comentado [H34R33]: Done. Comentado [MNP35R33]: FL if a school is to be accepted within the Bilingual/CLIL Scheme. Beyond this, it is the schools' responsibility to decide the exact number of hours that the FL will be used and how these will be distributed across curricular subjects. Because no maximums are set, exposure to the L1 may be significantly reduced depending on the school and the context, which could potentially have an effect on learners’ L1 development. Thus, monitoring the FL-L1 balance is important. Otro aspecto importante en relación al input que se proporciona en el aula CLIL está relacionado con la naturaleza del input y el consiguiente output que se obtiene (Otwinowska and Forys, 2015). En este sentido, es importante tener en cuenta que el objetivo principal del enfoque CLIL es que se mejore la competencia del alumnado para comunicarse en la foreign language. Así Gallardo et al. (2009) consideran que es necesario que el alumnado sea expuesto a authentic input y que la lengua se use de con communicative purpuses. De ahí que como indica Dalton Puffer (2011) una de la teorías de referencia sea el Krashen’s (1985) monitor model señalando la importancia que se concede en el mismo a la necesidad de que exista un comprehensive input. De la misma forma, Navés (2009:25) hace referencia a la necesidad de que el contexto de aprendizaje de la segunda lengua tenga una condiciones similares al de la primera lengua en tanto en cuanto el foco de la enseñanza debe estar en el significado más que en la forma y el input que se proporciona debe ajustarse al nivel de competencia del alumnado. Siguiendo esta misma idea, Nieto (2016:82) señala la comunicación como objetivo central en la metodología CLIL y la consecuente necesidad de proporcionar un rico y variado input que incluye “different language functions, different genres, academic language, classroom language…” como medio para conseguir foster “student´s involvement and production of comprehensive output”. Tener en cuenta las funciones comunicativas de la lengua y combinarlo con el aprendizaje tanto de interpersonal communication skills (BICS) and cognitive-academic language proficiency (CALP) implica la conveniencia de analizar el input que el profesorado proporciona en el aula no solo en cuanto a la distribución y proporción del mismo, sino también en cuanto a los momentos en los que se utiliza en el aula y los objetivos para los que se emplea. Various studies have focused on the type of content, discursive functions and activities for which the L1 and the FL are used in content subjects in CLIL programmes. For example, Marsh (2002: 97) concluded that ’the hours taught in the L1 will be devoted to knowledge-intensive teaching, whereas the CLIL [. . . ] hours will focus on activities permitting the use of the L2/L3: consolidating knowledge, and using it in various communicative activities.’ In a similar vein, Lorenzo, Casal and Moore (2009: 230) identified different stages in CLIL teaching for which each language is used. Their findings show that FL use varies depending on the discursive function employed. Flexibility Flexibility is a key consideration in CLIL implementation, reflecting the needs of students in their particular learning context and enabling the development both of the L1 and the FL. Baetens (2002: 25) refers to a variety of ways of organising CLIL: CLIL allows for a great flexibility in curricular design and time-table organization, ranging from early total, early partial, late immersion type programmes, to modular Comentado [MNP36]: Language-related variables embedded in the CLIL learning environment, which pertain to the nature of input, interaction and output in the classroom, An important factor that differentiates CLIL classes from language classes is the role of language used in the input, interactions and output” Comentado [MNP37]: (a) by being exposed to authentic input in the second language and (b) by needing to use the second language for real, communicative purposes’ Comentado [MNP38]: Una idea parecida na se si seguir an adiendo aque pero te dejo la cita (Jiménez Catalán and Ruiz de Zarobe (2009: 81): Traditionally, two types of language learning contexts have been distinguished: natural contexts and classroom contexts. Natural learning contexts are characterized by the use of language for communication as well as by exposure to a great deal of varied input provided by native speakers. As Ellis (1995: 17) points out, this kind of research aims at the clarification of theoretical issues but above all aims at the improvement of language teaching by means of developing teachers’ awareness of the nature of the input that learners are exposed to. Comentado [MNP39]: What is at the center for stakeholders is the understanding that CLIL classrooms are an environment for naturalistic language learning, implying that the best kind of language learning proceeds painlessly, without formal instruction. These implicit baseline assumptions are in line with Krashen’s (1985) monitor model, which continues to be the most prominent reception-based theory of language acquisition outside academic research circles. As is well known, the basic idea ... Comentado [MNP40]: Navés (2009:25): We can briefly review the rationale for integrating content and language. Krashen (1982), Lightbown and Spada (2006), Long (1990) and Swain (2000), among others, suggest ... Comentado [MNP41]: Thus, CLIL methodology is based on communication, providing a rich and varied input (different language functions, different genres, academic language, classroom language...) and fostering students’ ... Comentado [MNP42]: Hay que conectar este párrafo con el anterior o fundirlo Comentado [E43]: What is this? Comentado [H44R43]: Done. Comentado [E45]: The significance of this list is not clear: why is it cited? Wouldn’t it be more useful to indicate which functions L, C and Moore 2009 found were typically in the FL or the L1? Comentado [MNP46R45]: Explicación del artículo. Resumimos o citamos los resultados ¿? The results show that content teachers are more likely to employ the L2 in explicitly content-centred teaching: during activities, consolidation and revision and to a ... subject-determined slots in Germany (Mäsch, 1993) and the United Kingdom (Coyle, 2002). The modular approach allows for subjects to be handled in different combinations of languages, thereby responding to the desire to safeguard L1 capacities, improving L2 capacities and introducing L3 or a variety of languages, according to perceived local contingencies. Flexibility also allows for a wide range of abilities to benefit from partial education through a different language. As Marsh (2002: 95) points out, this sort of flexibility is already present in programmes with different and varied forms of CLIL delivery. In countries like Spain and France, state and regional governments allow schools a certain degree of autonomy in deciding which subjects are taught in the FL and how many hours of language exposure to both the L1 and the FL are appropriate, depending on context. Thus, flexibility in CLIL programme design contributes to achieving the right balance between L1 and FL in teaching content subjects to enable knowledge and skills to develop in both the FL and the L1. A study on the impact of CLIL on L1 competence development was carried out in Spain by Merino and Lasagabaster (2015). He concluded that there were no statistically significant differences between CLIL and non-CLIL strands in students’ development of L1 skills and knowledge. However, Lasagabaster's research took place in a bilingual context in Spain where the FL was in fact the learners' L3. To date, there are no few studies (Perez-Cañado, 2018) on the effects of CLIL on learners' L1 competence development in monolingual contexts. The present study addresses that gap, focusing on CLIL schools in Andalusia where the languages of instruction are Spanish (the learners’ only L1) and English (the learners’ FL). Aim and research questions This article presents the results of assessments and a survey carried out in seven public schools in the province of Seville in order to compare outcomes in L1 (Spanish) learning between CLIL and non-CLIL strands. The aim was to see if CLIL had any significant effect on students' L1 results, and to pinpoint any possible reasons for differences, based on variables relating to the particularities of the schools involved, their setting (urban vs. rural) or the balance of L1/FL input and type of input (i.e. discursive functions for which L1/FL was used). The research questions guiding the study were as follows: RQ1. Are there statistically signicant differences in Spanish Language and Literature results between CLIL and non-CLIL students? If so, to what extent are these determined by the setting (rural/urban) and/or the particularities of the school and its CLIL programme? RQ2. Are there variations regarding the proportion of L1 and FL and the type of input in each language that CLIL students receive in their content classes? RQ2. Are the amount, nature and type of input provided in CLIL classrooms playing a role on the learning of both languages? RQ3. What are teachers’ and students’ perceptions on whether participation in a CLIL programme has an impact on students’ L1 skills and knowledge development? The study and research methods Comentado [E47]: And isn’t this the case also in Spain? Your phrasing here seems to suggest France is different from Spain. Comentado [H48R47]: Done. Comentado [E49]: How were these tested? Comentado [MNP50R49]: Two test rounds were conducted in a longitudinal study spanning one year and in which 285 secondary education students took part. In the case of Basque and Spanish, reading comprehension was assessed through standardised tests. These tests were based on those previously used by the ISEI-IVEI (Basque Institute for Research and Evaluation in Education) (ISEI-IVEI 2012a, 2012b). The tests consisted of one or two texts followed by comprehension questions of several kinds with different values according to their complexity, that is, multiple choice, true or false, or those which required a short written answer. Written production in Basque and Spanish was assessed by means of an essay about a given topic that participants had to develop. The ‘profile’ technique (Jacobs et al. 1981) was applied to evaluate the written tests, which consists of five scales referring to the different aspects under consideration: Content, Organization, Vocabulary, Language Use and Mechanics. Within each of these scales there are four bandings (‘excellent to very good’, ‘good to average’, ‘fair to poor’, and ‘very poor’) which give the person marking a series of key words on which to base the specific evaluation criteria. The overall written score is arrived at by adding the scores for each of the scales, which varies from a minimum of 34 points to a maximum of 100. The students’ writings were assessed by two independent evaluators, the correlation coefficients between both being higher than .70 in all the five scales both at T1 and T2. Both evaluators were bilingual in Spanish and Basque, had completed their English Studies degree, and had previously been advised and guided on the procedure to assess the writings. Comentado [E51]: Why should these two contexts be different? You don’t explain specifically, and thus we remain without a clear rationale for your study. Please add a sentence or so reminding us why it should be so significant to examine a monolingual context. Comentado [MNP52R51]: En un contexto monolingue el alumnado no ha tenido la experiencia de aprendizaje bilingüe que se trata de poner en práctica con esta metodología y que implica por ejemplo un desarrollo cognitivo deferente. En contextos que ya son bilingües todos esos procesos ya han ocurrido con la primera y segunda lengua. Comentado [MNP53]: Esto no estaba así pero es que ya no es verdad poruqe Marisa ya ha publicado un artículo sobre esto, es que llevamos ya más de un año y así no se puede tener la bibliografía actualizada!!!Lo ponemos ¿? Con formato: Resaltar The present study belongs to a larger project on CLIL implementation and results in monolingual contexts in Spain. For this specific study, both quantitative and qualitative data have been gathered. The sample of schools was selected in collaboration with the provincial coordinator of bilingual programmes. It is composed of seven state schools in both rural (Schools 1, 3 and 6) and urban (Schools 2, 4, 5 and 7) settings. There were three primary schools (1, 2 and 3) and four secondary schools (4, 5, 6 and 7). Two secondary schools (Schools 4 and 7) took students from early CLIL programmes (i.e. students had been receiving bilingual instruction since the beginning of primary education) while the other two (Schools 5 and 6) had students in late CLIL programmes (i.e. starting bilingual instruction at the beginning of secondary education). At primary level, all CLIL students had received bilingual instruction from the onset of primary education. In all the schools CLIL and non-CLIL strands ran alongside each other. Students in the non-CLIL strands had regular EFL classes. The study involved a total of 271 students: 134 CLIL and 137 non-CLIL, 151 primary school learners (Year 6, age 11) and 120 secondary school students (Year 10, age 15). A group of 38 teachers, including FL teachers, content teachers and language assistants, also took part in this study. In order to create a homogenous sample of students allowing for comparisons, CLIL and non-CLIL groups were matched for verbal intelligence, motivation and out-of- class exposure to English using, respectively, the EFAI (Evaluación Factorial de las Aptitudes Intelectuales) subtest for verbal aptitude was used in primary and secondary versions (Santamaría, Arribas, Pereña and Seisdedos 2014); Pelechano’s (1994) MA motivation questionnaire; and a questionnaire based on the language diary created by Sundqvist and Sylvén (2014) to ascertain the type of activities in which students engaged in English outside school and the amount of time spent on these activities. In order to answer RQ1, CLIL and non-CLIL students’ end-of-year results in Spanish Language and Literature were collected. These comprised a summative assessment marked on a scale of 0 to 10 (with 5 as the pass mark) assessing progress over the school year in the five main curricular areas: oral communication (speaking and listening); written communication (reading); written communication (writing); knowledge of the language as a system; and literary education. In order to answer RQ2, we used responses to questions surveying teachers’ and students’ perceptions concerning the proportion of FL input in CLIL content classes. Regarding the type of input that students received in the L1 and the FL in content classes, semi-structured interviews were carried out with both teachers and students. In this study, the type of input received by the students was analysed in relation to the discursive functions for which each language is used in class. The protocols included a list of these discursive functions, which was used for the categorisation in the analysis. RQ3 used data from another selection of items from the longer questionnaire (Pérez-Cañado 2015). The items (Items 4 and 6 from Section 1 of both the Teacher version and the Student version) used a 4-point Likert scale and focused on gauging students’ and their teachers’ level of agreement on whether their L1 knowledge and skills and their language awareness had improved due to participating in bilingual education. The statistical software SPSS Version 24.0 was used for the analysis and processing of quantitative data. In the case of RQ1, both central tendency (mean) and dispersion measure (standard deviation) were calculated. Student-Fisher t-tests for Comentado [MNP54]: Quizás está más claro si se pone primero los de primaria y secundaria y despue´s los que corresponden a rurales y urbanos y eliminar toda la parte de early y late. Con eso no es necesario hacer la tabla Comentado [E55]: What about the primary schools? Did they all start CLIL from the beginning? Comentado [H56R55]: Done. Comentado [E57]: I think it would be really helpful to have a Table indicating all the different characteristics of the schools. This becomes an important issue later. Comentado [E58]: Can you explain what you mean? Are you saying for a CLIL student with X years’ exposure to English and an intelligence score of X, you were able to find a corresponding student with the same years of exposure and the same intelligence score???? Comentado [MNP59R58]: En todos los artículos aparece así. Creo que el proceso de qué significa matching hay que explicárselo a ella. Se seleccionaron alumnos de todos los niveles para que todos los centros tuvieses una cantidad aproximada de alumnado con todos los niveles tanto de inteligencia, motivación…. Comentado [E60]: We need much greater precision on what was involved: what kind of tests/assessments were involved, how the marking was undertaken (according to what criteria). In particular, the reader has NO idea of what ‘literary education’ might mean. Comentado [MNP61R60]: De esto no tenemos realmente información. Solo se puede decir que la asignatura es de lengua y literatura y cogimos las calificaciones de lengua que tras una evaluación continua había hecho los profesores. Comentado [E62]: None of this is really relevant. Comentado [H63R62]: OK. Comentado [E64]: How on earth did this give you a clear empirical picture of type of input!? Comentado [E65]: I’ve deleted this as it gives no precise information. Please explain specifically how the information about proportion and type of L1/FL input was obtained. I’m afraid it’s not clear from your write-up here. Comentado [MNP66R65]: Para las entrevistas se tenía una guia de preguntas que se les iban haciendo. Las preguntas que se hicieron: ¿En qué porcentaje diría que utiliza usted el inglés en clase? ¿Para qué funciones discursivas utiliza el inglés en clase: proporcionando un listado: Introducir el tema Transmitir contenidos Realizar actividades Aclarar dudas y explicar dificultades Formular preguntas Corregir tareas Consolidar y repasar conocimientos Organizar la clase con distintos tipos de agrupamiento Interactuar con el alumnado/profesorado Suministrar feedback sobre las actuaciones de clase independent groups and analysis of variance (ANOVA) were employed in order to test for statistically significant differences between CLIL and non-CLIL groups within and across schools. Levene’s test was used to check the equality of variances in the CLIL-and non-CLIL groups. Regarding RQ2, students’ and teachers’ reported percentage of usage of the L1 and the FL in class was compared. This quantitative analysis is complemented with qualitative data obtained in the interviews, and analysed following the categories established in the interview protocols. Results Table 1 shows the mean score and standard deviations for the L1 assessment scores for the whole sample (both primary and secondary students), and the results of the Student-Fisher t-test, checking the significance of any differences. As can be seen, a statistically significant difference (p = .001) was found between CLIL and non-CLIL groups in favour of CLIL strands, suggesting that students in CLIL groups obtained significantly higher scores in Spanish Language and Literature. Table 1. Comparison of results in L1 Spanish Language and Literature between CLIL and non-CLIL groups. Group n M SD p-value Non-CLIL 137 6.35 1.793 .001 CLIL 134 7.04 1.692 Furthermore, when primary and secondary education data were disaggregated, similar results were found. There were significant differences between CLIL and non-CLIL groups, again in favour of CLIL strands, at primary (p = .002) and at secondary level (p = .003). This suggests, at the very least, that CLIL education did not impact negatively on students’ L1 attainment and indeed appears to be associated with a positive effect on L1 attainment. Further analyses were then performed in order to explore the influence of other factors. First, using a t-test, significant differences in Spanish Language and Literature scores between schools in urban and schools in rural areas were checked; an ANOVA test was performed to identify any differences between schools on L1 attainment results for CLIL vs. non-CLIL strands. Table 2 shows that no statistically significant differences were found between rural and urban schools (p = .069). This indicates that the setting was not a factor affecting the results obtained when CLIL and non-CLIL groups are contrasted. Table 2. Comparison of results in Spanish Language and Literature between rural vs. urban schools Group n M SD p-value Rural 124 6.90 1.750 .069 Urban 147 6.51 1.780 Comentado [E67]: I’m not sure I’d emphasise the quantitative nature – getting participants to assess % is a fairly loose – and therefore inexact - way of calculating % of use of L1/TL (as opposed to, for example, basing the calculation on analysis of recordings or even class observation notes). Comentado [MNP68R67]: Lo que ella no entiende es que además de las entrevistas se realizaron unos cuestionarios complementaios con una escala Likert que es de dónde salen los datos cuantitativos. Esto está dicho en el párrafo anterior. Comentado [E69]: We have no idea what these were, so it’s a bit pointless to mention them. See above re. need for greater clarity and precision in reporting research instruments. Comentado [MNP70R69]: Información sobre los protocolos para las entrevistas. Ejemplos de las preguntas? Referenciar los instrumentos con el artículo de Marisa: Evaluating CLIL programmes: Instrument Design and Validation, 2016 Con formato: Resaltar Table 3 shows the effect of the independent variable 1 (CLIL/non-CLIL strand) and the independent variable 2 (school) on the dependent variable (L1 results). Both these independent variables were found to have a statistically significant effect with high effect sizes: for CLIL vs. non-CLIL (partial Eta2 = .193) and for school (partial Eta2 = .169). The estimated marginal mean in the CLIL group was higher (3.524) than in the non-CLIL group (2.825). However, when both independent variables were considered together, no statistically significant differences were found (p = .150) and the effect size was lower (partial Eta2 = .036). This suggests that ‘school’ was a crucial variable acting on the results. Table 3. Inter-subject analysis Origin Type III Sum of squares Gl. RMS F. Sig. Eta2 Corrected model 113.837a 13 8.757 6.233 .000 .240 Intercept 2222.622 1 2222.622 1582.095 .000 .860 CLIL/non-CLIL strand 26.944 1 26.944 19,179 .000 .069 School 86.223 6 14.370 10,229 .000 .193 CLIL/non-CLIL x School 13.426 6 2.238 1.593 .150 .036 Further analysis was carried out considering the strands (CLIL- non-CLIL) and the schools. In this analysis, means were contrasted and post-hoc tests performed for both primary and secondary schools. Table 4 shows the descriptive statistics for students in primary education, according to the school attended. En todos los casos las secciones CLIL de cada centro obtienen mejores resultados que las secciones non-CLIL. There are differences between the means of the schools: the highest mean corresponds to School 2 and the lowest one to School 3, both for CLIL and non-CLIL groups. Table 4. Descriptive statistics for individual primary schools Section School M SD Non CLIL School 1 3.23 1.055 School 2 3.68 1.090 School 3 3.03 1.449 Total 3.30 1.231 CLIL School 1 3.88 1.116 School 2 4.33 0.970 School 3 3.63 1.300 Total 3.93 1.153 Table 5 shows the results of the post-hoc analysis with multiple comparisons between schools. There are statistically significant differences between School 2 and School 3 (p = .016). Interestingly, School 2 is the school that has been implementing a CLIL programme longer. implements an early CLIL programme starting at 2005, whereas School 3 has a late one. These data suggest that the lengthyears of implementation of the CLIL programme may be an important factor affecting students' results both in CLIL Con formato: New paragraph, No conservar con el siguiente Con formato: New paragraph Con formato: New paragraph Con formato: New paragraph Con formato: New paragraph Con formato: New paragraph Con formato: Sangría: Primera línea: 0 cm Con formato: Español (España) Con formato: Tachado Con formato: Fuente: Sin Negrita Con formato: Fuente: Sin Negrita Comentado [MNP71]: Para Primaria sí podemos decir que el que tiene mejores resultados lleva más tiempo con el programa, sé que es el que llevaba más tiempo y con la coordinadora con más experiencia. Empezó en el 2005 pero no consigo encontrar los datos de cuándo empezaron los demás. Quizás poner simplemente que el programma es más antiguo y ya. Comentado [E72]: Be careful not to over interpret – it’s only based on one school out of 7 Comentado [H73R72]: Done. References Adesope, Olusola O; Lavin, Tracy; Thompson, Terri; Ungerleider, Charles. 2010. “A Systematic Review and Meta-Analysis of the Cognitive Correlates of Bilingualism.” Review of Educational Research 80 (2): 207, doi: https://doi.org/10.3102/0034654310368803 Admiraal, W., G. Westhoff, and K. de Bot. 2006. “Evaluation of Bilingual Secondary Education in the Netherlands: Students’ Language Proficiency in English.” Educational Research and Evaluation 12 (1): 75–93. Alejo, R., and A. Piquer. 2016. “Urban vs. rural CLIL: an analysis of input-related variables, motivation and language attainment.” Culture and Curriculum 29 (3): 245-262. Agustín Llach, M.P. 2009. “The Role of Spanish L1 in the Vocabulary Use of CLIL and non- CLIL EFL Learners.” In Content and Language Integrated Learning. Evidence from Research in Europe, edited by Y. Ruiz de Zarobe and R. Jiménez, 112-129. Canada: Multilingual Matters. Arribas, M. 2016. “Analysing a whole CLIL school: Students’ attitudes, motivation, and receptive vocabulary outcomes.” Latin American Journal of Content and Language Integrated Learning 9 (2): 267-292. doi:10.5294/la-clil.2016.9.2.2 Baetens, H. 2002. “The Significance of CLIL/EMILE.” In CLIL/EMILE. The European dimension. Actions, trends, and foresight potential, edited by D. Marsh, 20-26. Jyva skyla : University of Jyva skyla . Casal, S., and P. Moore. 2009. “The Andalusian Bilingual Sections Scheme: Evaluation and consultancy.” International CLIL Research Journal 1: 36-46. Cenoz, J. 2013. “Discussion:towards an educational perspective in CLIL language policy and pedagogical practice.” International Journal of Bilingual Education and Bilingualism 16(3):389-394. Doi: 10.1080/13670050.2013.777392 Cenoz, J. 2015. “Content-based instruction and content and language integrated learning: the same or different?” Language, Culture and Curriculum 28(1):8-24. Doi: 10.1080/07908318.2014.1000922 Coyle, D. 2002. “Relevance of CLIL to the European Commission’s Language.” In CLIL/EMILE. The European dimension. Actions, trends, and foresight potential, edited by D. Marsh, 27-29. Jyva skyla : University of Jyva skyla . Coyle, D. 2008. “Content and Language Integrated Learning: towards a connected research agenda for CLIL pedagogies.” International Journal of Bilingual Education and Bilingualism 10(5): 543-562. Coyle, Do, Peter Hood, and David Marsh. 2010. CLIL: Content and Language Integrated Learning. Cambridge: Cambridge University Press. Cummins, J. 1999. BICS and CALP: clarifying the distinction. ERIC ED438551. Accessed 12 September 2016. https://eric.ed.gov/?id=ED438551 Cummnis, J. 2000. Language, Power and Pedagogy: Bilingual Children Caught in the Cross-Fire. Clevedon: Multilingual Matters. Dalton-Puffer, C. 2007. Discourse in Content and Language integrated Learning (CLIL) Classrooms. Language Tearning & Language Teaching 20. Amsterdam: John Benjamins Publiching Company. Dalton-Puffer, C. 2011. “Content-and-Language Integrated Learning: From practice to principles?” Annual Review of Applied Linguistics 31: 182–204. Dalton-Puffer, C., and U. Smit. 2013a. “Content and Language Integrated Learning: A research agenda.” Language Teaching 46: 545-559. doi:10.1017/s0261444813000256 Dalton-Puffer, C., and U. Smit. 2013b. “The power of beliefs: Lay theories and their influence on the implementation of CLIL programmes.” International Journal of Bilingual Education and Bilingualism 16 (3): 267-284, doi: 10.1080/13670050.2013.777385 Doiz, A., D. Lasagabaster, and J. M. Sierra. 2014. “CLIL and motivation: The effect of Individual and contextual variables.” The Language Learning Journal 42: 209-24. European Commision. 2004. Promoting language learning and linguistic diversity: An action plan 2004-06. Luxembourg: European Commision. European Council. 2001. Common European framework of reference for languages: learning, teaching, assessment. Strasbourg: Council of Europe. European Commision. 2012. “Europeans and their Languages. Special Eurobarometre 386.” Accessed 12 September 2016. http://ec.europa.eu/publi_opinion/archives/ebs/ebs_386_en.pdf. Eurydice. 2012. “Key Data on Teaching Languages at School in Europe.” Accessed 12 September 2016. http://eacea.eceuropa.eu/education/eurydice/documents/key_data_series/14 3EN_HI.pdf. Fernández, A., and A. Canga. 2014. “A preliminary study on motivation and gender in CLIL and non-CLIL types of instruction.” International Journal of English Studies 4 (1): 21-36. Fernández-Sanjurjo, J., A. Fernández-Costales and J. Arias. 2017. “Analysing students learning in science in CLIL vs. Non-CLIL programmes: Empirical evidence from Spain.” International Journal of Bilingual Education and Bilingualism, doi: 10.1080/13670050.2017.1294142 Graaff, R., and G. Westhoff. 2007. “An observation tool for effective L2 pedagogy in Content and Language Integrated Learning (CLIL).” The International Journal of Bilingual Education and Bilingualism 10 (5): 603-624. Junta de Andalucía. 2016. “Plan Estratégico de desarrollo de las Lenguas en Andalucía.” Accessed 12 September 2016. http://www.juntadeandalucia.es/educacion/portals/abacoportlet/content/462f16e3-c047-479f-a753-1030bf16f822 Krashen, S. 1985. The Input Hypothesis. London: Longman. Krashen, S. 1997. Why bilingual education. ERIC Digest. Charleston, WV: ERIC/CRESS. Lasagabaster, D. 2008. “Foreign Language Competence in Content and Language Integrated Courses.” The Open Applied Linguistics Journal 1: 30–41. Lasagabaster, D., and J.M. Sierra. 2009. “Language attitudes in CLIL and traditional EFL classes.” International CLIL Research Journal 1 (2): 4-17. Lasagabaster, D., and J. M. Sierra. 2010. “Immersion and CLIL in English: More Differences Than Similarities.” ELT Journal 64: 367–375. Lasagabaster, D. and Y. Ruiz de Zarobe, eds. 2010. CLIL in Spain: Implementation, Results and Teacher Training. Newcastle: Cambridge Scholars. Lasagabaster, D. 2011. “English achievement and student motivation in CLIL and EFL settings.” Innovation in Language Learning and Teaching 5 (1): 3-18. Lasagabaster, D., and A. Doiz. 2015. “A Longitudinal Study on the Impact of CLIL Affective Factors.” Applied Linguistics 24: 1-26. Lasagabaster, D. 2017. “Language learning Motivation and Language Attidutes in Multilingual Spain From an International Perspective.” Modern Language Journal 101:583-596. Long, M.1996. The role of the linguistic environment in second language acquisition. In Handbook of Second Language Acquisition, ed. W. Ritchie and T. Bhatia, 413-468. Lorenzo, F., S. Casal, and P. Moore. 2009. “The Effects of Content and Language Integrated Learning in European Education: Key Findings from the Andalusian Bilingual Sections Evaluation Project.” Applied Linguistics 31 (3): 418–442. Merino, J.A., and D. Lasagabaster. 2015. “CLIL as a way to multilingualism, International Journal of Bilingual Education and Bilingualism.” International Journal of Bilingual Education and Bilingualism, doi: 10.1080/13670050.2015.1128386 Merino, J.A., and D. Lasagabaster. 2017. “The effect of content and language integrated learning programmes´s intensity on English proficiency: A longitudinal study.” International Journal of Applied Linguisitics 28 (1):1-13, https://doi.org/10.1111/ijal.12177 Milla, M.D. and A.V. Casas. 2018. “Teacher Perspectives on CLIL Implementation: A Within-Group Comparison of Key Variables.” Porta Linguarum 29:159-180. Marsh, D., ed. 2002. CLIL/EMILE. The European Dimension. Actions, Trends, and Foresight Potential. Jyva skyla : University of Jyva skyla . Mehisto, P., D. Marsh, M. J. y Frigols. 2008. Uncovering CLIL: Content and Language Integrated Learning in Bilingual and Multilingual Education. Oxford: Macmillan Education. Ministerio de Educación, Cultura y Deporte. 2012. Estudio Europeo de Competencia Lingüística EECL. Informe Español. Vol. I. Madrid: Secretaría General Técnica. Ministerio de Educación, Cultura y Deporte. 2011. Estudio Internacional de Progreso en Comprensión Lectora, Matemáticas y Ciencias. PIRLS-TIMS. Informe Español. Vol. I. Madrid: Instituto Nacional de Evaluación Educativa. Navés, T. 2009. “Effective Content and Language integrated Learning (CLIL) Programmes.” In Content and Language Integrated Learning. Evidence from Research in Europe, edited by Y. Ruiz de Zarobe and R. Jiménez, 22-40. Canada: Multilingual Matters. Nieto, E. 2016a. “The impact of CLIL on the Acquisition of the learning to learn Competence in Secondary School Education in the Bilingual Programmes of Castilla-La Mancha.” Porta Linguarum 2: 21-34. Nieto, E. 2016b. “The impact of CLIL on the Acquisition of L2 Competences Skills in Primary Education.” International Journal of English Studies 16 (2): 81-101. Nikula, T., C. Dalton-Puffer, A. Llinares., and F. Lorenzo. 2016. “More than content and language: the complexity of integration in CLIL and multilingual education.” In Conceptualising integration in CLIL and multilingual education, edited by T. Nikula, E. Dafouz, P. Moore, and U. Smit, 1-28. Bristol: Multilingual Matters. Otwinowska, A., and M. Foryś. 2015. “They learn the CLIL way, but do they like it? Affectivity and cognition in upper-primary CLIL classes.” International Journal of Bilingual Education and Bilingualism, doi:10.1080/13670050.2015.1051944. Pfenninger, S. 2016. “All good things come in threes: early English learning, CLIL and motivation in Switzerland.” Cahiers de l ILSL 48: 119-147. Pavón. V. 2018. “Learning Outcomes in CLIL programmes: a Comparison of Results Between Urban and Rural Enviroments.” Porta Linguarum 29:9-28. Pelechano, V. (1994). “Prueba MA.” Análisis y Modificación de la Conducta, 20: 71-72. Pérez-Cañado, M. 2012. “CLIL research in Europe: past, present and future.” International Journal of Bilingual Education and Bilingualism 15(3): 315-341, doi: 10.1080/13670050.2011.630064 Pérez-Cañado, M. 2014. “Teacher training needs for bilingual education: in-service teacher perceptions.” International Journal of Bilingual Education and Bilingualism 19 (3): 1-30, doi: 10.1080/13670050.2014.980778 Pérez-Cañado, M. 2016. “Are teachers ready for CLIL? Evidence from a European study.” European Journal of Teacher Education 39 (2): 79-112, doi: 10.1080/02619768.2016.1138104 Pérez-Cañado, M. 2018. “The effects od CLIL on L1 and content learning: Update emirical evidence from monolingual contexts.” Learning and Instruction 57: 18-33, https://doi.org/10.1016/j.learninstruc.2017.12.002 Ruiz de Zarobe, Y. and J. Cenoz, 2015. “Way forward in the twenty-first century in content-based instruction: moving towards integration.” Language, Culture and Curriculum 28(1): 90-96, doi: 10.1080/07908318.2014.1000927 Ruiz de Zarobe, Y. and R. Jimenez. 2009. Content and Language Integrated Learning. Evidence from Research in Europe. Canada: Multilingual Matters. Ruiz de Zarobe, Y., J. Sierra and F. Gallardo 2011. Content and Foreign Language Integrated Learning. Contributions to Multilinguals in European Contexts. Bern: Peter Lang. Santamaria, P., Arribas, D., Pereña, J. and Seisdedos, N. (2014). EFAI. Evaluación Factorial de las Aptitudes Intelectuales. Madrid: TEA. Seikkula-Leino, J. 2007. “CLIL Learning: Achievement levels and affective factors.” Language and Education 21(4): 328-341, doi: 10.2167/le635.0 Stocco, A., Yamasaki, B., Natalenko, R. and Prat, C. 2014. “Bilingual brain training: A neurobiological framework of how bilingual experience improves executive function” International Journal of Bilingualism 18 (1):67-92, doi: https://doi.org/10.1177/1367006912456617 Sundqvist, P. and Sylvén, L.K. (2014). “Language-related computer use: Focus on young L2 English learners in Sweden.” ReCALL 26(1):3-20. Swain, M. 1985. Communicative competence: some roles of comprehensible input and comprehensible output in its development. In Input in Second Language Acquisition, ed. S.M. Gass and C.G. Madden, 235-256. Sylvén, L. and A. Thompson. 2015. “Language Learning motivation and CLIL. Is there a connection?” Journal of Immersion and Content-Based Language Education 3 (1): 28-50. Emmanuelle Le Pichon Vorstman; Henriette De Swart, Viktorija Ceginskas & Huub Van Den Bergh. 2009. “Language learning experience in school context and metacognitive awareness of multilingual children.” International Journal of Multilingualism 6(3): 258-280, doi: 10.1080/14790710902878692 Comentado [MNP94]: Así pone para citarlo en la revista pero no tengo muy claro en todos cúal es el nombre y el apellido para adaptarlo al formato de las referencias!!!