What is published in impact journals on school effectiveness? A systematic review of research results and methods

5 Pedagogika / 2020, t. 138, Nr. 2 What is Published in Impact Journals on School Effectiveness? A Systematic Review of Research Results and Methods Jesús García-Jiménez1, Juan Jesús Torres-Gordillo2, Javier Rodríguez-Santero3 1 University of Seville, Department of Educational Research Methods and Diagnostics. C/Pirotecnia, s/n, ES-41013, Seville, Spain, [email protected] 2 University of Seville, Department of Educational Research Methods and Diagnostics. C/Pirotecnia, s/n, ES-41013, Seville, Spain, [email protected] 3 University of Seville, Department of Educational Research Methods and Diagnostics. C/Pirotecnia, s/n, ES-41013, Seville, Spain, [email protected] Abstract. A systematic review of 45 articles on school effectiveness (SE) indexed in SJR and JCR (2014–2018) has been conducted. The results obtained show that the quantitative methodology catalyses the greatest number of researches. The articles are mainly published in Q1-Q2 journals with impact factor. The content of the articles reviewed found that, although exist contextual factors that affect the SE, the variables at the school level also have an impact. Keywords: educational efficiency, educational quality, educational output, school student relationship. Introduction The study of school effectiveness is a popular theme in educational science. In the scientific panorama, forums that analyse school effectiveness have been created, such as the International Congress for School Effectiveness and Improvement (ICSEI) and journals that focus on this subject, such as the School Effectiveness and School Improvement (SESI) (Reynolds et al., 2014). School effectiveness is a concept that has evolved throughout history. The Coleman Report (Coleman et al., 1967) opens the door to the study of school effectiveness and large-scale research. It concludes that the socioeconomic characteristics of the student body explain the greater variance in the results. However, ISSN 1392-0340 (Print) ISSN 2029-0551 (Online) https://doi.org/10.15823/p.2020.138.1 Pedagogika / Pedagogy 2020, t. 138, Nr. 2, p. 5–24 / Vol. 138, No. 2, pp. 5–24, 2020 6 Pedagogika / 2020, t. 138, Nr. 2 the characteristics of schools also affect academic achievement (Azpillaga, Intxausti & Joaristi, 2014; Cohodes, 2016; Fuller & Hollingworth, 2014; Hofman, Hofman & Gray, 2015; Karakolidis, Pitsia & Emvalotis, 2016; Martínez-Abad, Lizasoain, Castro-Morera & Joaristi, 2017; Murillo, 2007; Page, Martín, Orellana & González, 2016; Ranjan, 2014; Schmidt, Burroughs, Zoido & Houang, 2015; Tichnor-Wagner, Harrison & Cohen-Vogel, 2016; Weber, 1971), although the variance explained varies from country to country (Costa & Araújo, 2018; OECD, 2019; Schmidt et al., 2015). For this reason, school effectiveness is currently defined as the impact that the school has on the academic performance and social development of the students (Fuller & Hollingworth, 2014; Murillo, 2007; Reynolds et al., 2014; Scheerens, 2000). On the other hand, effective schools are those that develop students in a comprehensive manner, both in basic instrumental skills and in emotional skills, beyond what would be expected in light of their context, personal characteristics and previous results (Murillo, 2005; Stoll & Fink, 1996). Education systems must guarantee the effectiveness of their schools, because the benefits go beyond training in basic skills (OECD, 2017a). However, not all schools guarantee quality education (OECD, 2017b). Some countries have translated the concern to optimise their country’s education into institutions that seek to improve school efficiency (Childs & Russell, 2017). In the Spanish and Latin-American contexts, large-scale educational performance assessments such as the Programme for International Student Assessment (PISA) have had an impact on educational policies (Kauko, Centeno, Candido, Shiroma & Klutas, 2016; LOMCE, 2013). Standardised tests have not been exempt from criticism. Limitations of assessments, economic focus or loss of humanism are arguments detrimental to standardised assessments (Deng & Gopinathan, 2016; Méndez, 2014; Parra, 2018). Nevertheless, evaluations of education systems allow for their progress (Ferrão, 2014). The availability of standardised test results and questionnaires on the socio-familiar context allow to establish advanced statistical models (Cordero & Manchón, 2014; Feldhoff, Radisch & Bischof, 2016; Gamazo, Martínez-Abad, Olmos-Migueláñez & Rodríguez-Conde, 2017; Jennings, Deming, Jencks, Lopuch & Schueler, 2015; Reynolds et al., 2014). Value-added statistical models make it possible to find schools with a high residual value (Gamazo, et al., 2017; Iyer & Moore, 2017). The residual gain consists of the difference in score expected for its socioeconomic context and that actually obtained. This parameter makes it possible to locate both high or low-efficiency schools, when exceeding the expected score, and low efficiency schools (Martínez-Abad et al., 2017). The OECD (2017b) recommends helping disadvantaged schools by optimizing their personal and financial resources. Schools can be optimised by knowing the variables at the school level that differentiate effective schools (Martínez-Abad et al., 2017; Rutledge, Cohen-Vogel, Osborne-Lampkin & Roberts, 2015). To detect the educational patterns that characterise high and low efficiency schools, some authors start from diagnostic evaluations to identify high or low residue centres (Azpillaga et al., 2014; Intxausti, Etxeberria & Bartau, 2017; Intxausti, 7 Pedagogika / 2020, t. 138, Nr. 2 Joaristi & Lizasoain, 2016; Joaristi, Lizasoain & Azpillaga; 2014; Lizasoain, Bereziartua & Bartau; 2016). In a first phase, by means of multilevel hierarchical models and taking as a reference the residual gain or the growth of scores, a classification of the centres according to the level of school efficiency achieved over the years is carried out. In other words, a longitudinal perspective (Pedroza-Zúñiga, Cetzal, Surema & Lizasoain, 2018) is taken into account. This diagnostic phase is followed by a second phase in which the centres with the highest and lowest residual gain are studied qualitatively. As a result of this study, the educational guidelines that explain the effectiveness (or ineffectiveness) of the schools are established. This type of methodological design is closer to the reality and complexity of education systems (Feldhoff et al., 2016) because they consider both the personal and contextual characteristics of the students and the schools, as well as the processes that take place in the schools (Scheerens, Luyten & van Ravens, 2011) that could explain their added value (Martínez-Abad et al., 2017). Therefore, the following research questions are posed: 1) What research methodologies do journals catalyse for school effectiveness? 2) What are the variables that explain school effectiveness? In order to answer the first question, the aim was to find out the methodological trends of articles published on school effectiveness in journals indexed in JCR (WoS) and SJR (Scopus), in the period between 2014 and 2018. Furthermore, in order to answer the second question, the objective of describing the results that explain school effectiveness in these articles was proposed. This objective is addressed as a response to the demand raised in recent research on school effectiveness (Hajisoteriou, Karousiou & Angelides, 2018; Scheerens, 2014; 2015). Materials and methods The method used in the research was the systematic review, which allows deepening the study of a subject through gathering, systematizing and relating the results of research conducted (Higgins & Wells, 2011; Manterola, Astudillo, Arias, Claros & Mincir, 2013; Martimbianco et al., 2020; Prendes-Espinosa, García-Tudela & Solano-Fernández, 2020). The methodology was adapted to the present investigation using the following techniques: thematic analysis (Braun & Clarke, 2006), systematic conceptual review (Feldhoff et al., 2016; Hallinger, 2014) and narrative synthesis (Kauffman, 2015). The criteria used for the selection of articles that form part of the systematic review were: • The work should focus on school effectiveness. • High-impact articles: high-impact articles were defined as those published in journals indexed in the JCR (WoS) and SJR (Scopus) databases. This requirement was to ensure the scientific quality of the articles. • The period studied was limited to 2014–2018. • The key search words were School Effectiveness, School Improvement, Educational Multilevel Analysis, PISA and School Value-added. 8 Pedagogika / 2020, t. 138, Nr. 2 The systematic review process began with a search of ERIC, WoS and Scopus as the main sources of work collection. High-impact articles were situated in the period 2014– 2018. The initial result was 67 selected articles. After a first exploratory analysis, those that did not meet all the established selection criteria were discarded. An inductive thematic analysis of the studies was used with the selected articles (Braun & Clarke, 2006). Two themes that influence school effectiveness were identified: both contextual and school variables. Again, those that did not respond to the selected topics were discarded. The final result of the articles to be analysed was 45 (dx.doi.org/10.17504/protocols.io.bf34jqqw). After this filter, a systemic conceptual review (Feldhoff et al., 2016; Hallinger, 2014) that allowed us to carry out a descriptive analysis of the research was performed. The variables studied were the approach (qualitative, quantitative or mixed), the impact quartile to which they belong and the data analysis techniques used in this research. In addition, a narrative synthesis (Kauffman, 2015) of the research results was carried out. These were synthesised and pooled in order to draw new conclusions and expose the literature review. In order to guarantee the quality of the systematic review, the indications on the PRISMA checklist were considered (Moher, Liberati, Tetzlaff & Altman, 2009). Results Systematic conceptual review of articles on school effectiveness Figure 1 shows the percentages over the total number of articles, organised by methodological approach according to impact quartile. The impact quartile was divided into Q1-Q2 and Q3-Q4 for this analysis. Figure 1. Results (F) of articles according to methodological approach and impact quartile 9 Pedagogika / 2020, t. 138, Nr. 2 The total results in Figure 1 should be interpreted taking into account that the articles indexed in JCR also belong to SJR. These results revealed that quantitative research was more present in impact journals with a prevalence greater than four times. Easy access to the results of large-scale evaluations may have been one of the reasons (Cordero & Manchón, 2014; Feldhoff et al., 2016; Gamazo et al., 2017; Jennings et al., 2015; Reynolds et al., 2014; Scheerens, 2014). On the other hand, in both JCR (25) and SJR (36), the impact factor of the journals was mainly distributed in Q1-Q2, with a prevalence of three and four times respectively, as opposed to Q3-Q4. From this, it could be inferred that the subject of school effectiveness was of interest and impact in educational research in the period studied. Table 1 distinguishes data analysis techniques in the case of quantitative methodologies and data collection in qualitative designs. Quantitative analysis techniques were divided into multilevel design (when considering the nested structure of the data), value-added designs (without considering the nested structure of the data), descriptive designs and hypothesis contrast, structural equations, regression analysis and others. Table 1 Results (F) of the Data Analysis Techniques Used in Each Methodological Design Method Data analysis Indexed database Total techniques JCR SJR Q1-Q2 Q3-Q4 Q1-Q2 Q3-Q4 Quantitative Multilevel design 12 3 17 7 24 44 VA Design 3 0 3 0 3 Descriptive-contrast of hypotheses 1 0 1 2 3 Structural equations 1 1 2 0 2 Regression 6 1 7 0 7 Other 5 0 5 0 5 Qualitative Content analysis 2 0 2 0 2 14 Category analysis 1 2 3 2 5 Meta-analysis 2 2 5 0 5 Others-not specified 2 0 2 0 2 The most commonly used quantitative data analysis technique was a multilevel design. Multilevel designs respect the nested structure of the data. In education, student results are nested in classrooms, schools and countries. Multilevel designs make it possible to evaluate the added value of schools by taking into account the nested structure of the education system (Gamazo et al., 2017). Depending on the research objective, it was 10 Pedagogika / 2020, t. 138, Nr. 2 used with different nuances. More traditional value-added models underestimate the effect of the centre on school effectiveness (Troncoso, Pampaka & Olsen, 2016). Descriptive-contrast of hypotheses was considered when it was the main tool of the article, not as a complement to other techniques, finding the use of ANOVA (Gaertner, Wurster & Pant, 2014) or the “t” value (Ranjan, 2014). In the regressions, specificities such as quartile regression (Li & Konstantopoulos, 2017) or ordinary squares regression (Hobbs, 2016; Schmidt et al., 2015) could be highlighted as examples. In other quantitative analysis techniques, there were decision trees, quantile value-added case, and the free model according to Page (2017), CLASS-S (an observational instrument), agent-based model and econometric analysis techniques (index of school performance). In qualitative techniques, meta-analysis, and category analysis were the most used. The specific analysis technique used was not always mentioned. Only a few of the authors cited it: a systematic conceptual review (Feldhoff et al., 2016) and realistic synthesis (Ehren, Eddy-Spicer, Bangpan & Reid, 2017). In other techniques, although the process is very detailed by Rutledge et al. (2015), he does not specify the name of the technique although it could be catalogued as content analysis. On the other hand, mapping was used in combination with the critical incident technique (Day, Gu & Sammons, 2016). Narrative synthesis of research results on school effectiveness The results of the narrative synthesis of the 45 papers analysed are presented below. The results were organised into inputs and processes (Scheerens et al., 2011). The inputs describe the results that could explain the effectiveness of the personal and contextual characteristics associated with the students. The processes develop the results that the different authors indicated improved the effectiveness of the schools. Contextual factors associated with school effectiveness The economy, social and cultural status (ESCS) of the family influenced academic results (Karakolidis et al., 2016). Academic performance was unfavorable in families with a low socioeconomic level (Gamazo et al., 2017; Karakolidis et al., 2016; Martínez et al., 2017; Martinez-Abad & Chaparro-Caso, 2017; Murillo, Martínez-Garrido & Hidalgo, 2014; Özdemir, 2016; Perry, 2017; Salgado, Marchione & Gilbert, 2014; Sulis & Porcu, 2014; Troncoso et al., 2016; Tsai, Smith & Hauser, 2017; Van Hek, Kraaykamp & Pelzer, 2018; Valenzuela, Bellei & Allende, 2016). The influence may vary depending on cultural mechanisms given the greater influence of the number of books in some Western countries than in others in the East (Tsai et al., 2017). The location of the school was a predictor of educational performance (Cordero & Manchón, 2014; Kelcey & Shen, 2016; Sulis & Porcu 2014) and of the probabilities of entering university (Jennings et al., 2015). This was probably no more than a spurious correlation determined by socioeconomic level (Gamazo et al., 2017; Karakolidis et al., 2016; Özdemir, 2016; Troncoso et al., 2016). For the same reason, the social networks of 11 Pedagogika / 2020, t. 138, Nr. 2 students and school efficiency were related (Hobbs, 2016; Salgado et al., 2014). Likewise, the greater efficiency of private schools would be explained by the socioeconomic level of the students who attend these schools (Hobbs, 2016; Hofman et al., 2015; Joaristi et al., 2014; Troncoso et al., 2016). Families with a high ESCS will be offered greater learning opportunities to their children (Hobbs, 2016; Schmidt et al., 2015), which would explain that greater book ownership has a positive influence on performance (Evans, Kelley & Sikora, 2014). This could be interpreted as really being the cultural level of the family, rather than the economic level, which would explain a higher percentage of variance with respect to educational performance. Gender was controversial. There were studies that argued that it does not influence performance (Salgado et al., 2014) and others that do. The female gender negatively affected performance in mathematics in adolescence (Gamazo et al., 2017; Hofman et al., 2015; Karakolidis et al., 2016; Özdemir, 2016; Troncoso et al., 2016) and positively influenced, although with less intensity, reading (Gamazo et al., 2017). Female students benefited more from attending schools with a high socioeconomic level (Van Hek et al., 2018). Again, the socioeconomic context could explain the differences between educational opportunities among both genders. The repetition condition of the student body, or if the student has changed schools many times, impaired educational performance (Gamazo et al., 2017). Likewise, others personal characteristics of the student, such as self-esteem, learning strategies or perception of drug use, were another factor to consider (Martinez-Abad & Chaparro-Caso, 2017). Although contextual and personal factors associated with the pupils influenced performance, there are school characteristics that influenced academic outcomes (Azpillaga et al., 2014; Intxausti et al., 2016, 2017; Joaristi et al., 2014; Lizasoain et al., 2016). In addition, learning opportunities would explain 37% of the inequalities associated with ESCS (Schmidt et al., 2015) so those schools that are more effective will be able to reduce the inequality gap in the students (Valenzuela et al., 2016). School factors associated with school effectiveness The variables associated with the school itself and the processes that take place in it significantly affect the achievements of the learners (Hofman et al., 2015; Schmidt etal., 2015). The school influences the performance of its students from the first years of schooling (Ferrão, 2014). There are centres that have a high added value, regardless of the socioeconomic context in which they are inserted (Azpillaga et al., 2014; Cohodes, 2014; Ferrão, 2014; Gamazo et al., 2017; Intxausti et al., 2016, 2017; Jennings et al., 2015; Joaristi et al., 2014; Lizasoain et al., 2016; Martínez et al., 2017; Rutledge et al., 2015). Therefore, educational research should focus its efforts on studying the school variables that affect educational performance (Ferrão, 2014; Joaristi et al., 2014) which cannot always be done in a quantitative way (Fuller & Hollingworth, 2014). The influence of the school could explain 20% of the variance in educational performance (Hofman et al., 2015; Kelcey 12 Pedagogika / 2020, t. 138, Nr. 2 & Shen, 2016); therefore, the control of these variables would make it possible to optimise the teaching-learning processes and move towards more equitable education. The characteristics are presented in 4 categories: educational approach, educational practice, management team and learning culture. Educational practice In terms of educational practice, teachers in effective schools had a greater social and emotional involvement with students and their families. Teachers externalised it as high expectations for students and greater involvement in extracurricular activities (Rutledge et al., 2015). Teachers optimised the use of resources, whether external or internal and systematically help students through extracurricular support or volunteer systems to improve education (Intxausti et al., 2017). There was also greater involvement with student assessments (Ranjan, 2014). These were more frequent and regular in the review of homework and tasks, which was related to better results (Murillo et al., 2014). The highly efficient schools showed a more holistic perspective, attending to the academic and emotional development of the students (Rutledge et al., 2015). In addition, teachers showed more planning and closeness skills (Ranjan, 2014). The involvement of families in the school seemed to be a defining characteristic of centres with high efficiency (Azpillaga et al., 2014; Hajisoteriou et al., 2018), with mothers having a great leading role in education (Cordero & Manchón, 2014). In centres with high efficiency, greater sensitivity was observed towards the families of the students, as well as greater connection with them (Rutledge et al., 2015). There was a discrepancy in the number of students per class, as there was research stating that it did not influence results (Li & Konstantopoulos, 2017). In contrast, Egalite and Kisida (2016) stated that the results worsen as the ratio and educational level increased. However, high-performing schools may be more responsive to challenges because they have better resource planning (Intxausti et al., 2017). Management team The school inspections did not appear to identify processes for improving the effectiveness of schools (Gaertner et al., 2014). However, management, in determining different processes, has an influence on school activity (Prasertcharoensuk & Tang, 2017). The management of high-efficiency schools was sensitive to the needs of the students, flexible to change, open to innovation, and exercises democratic leadership (Day et al., 2016; Intxausti et al., 2016). Principals did not reduce school effectiveness to performance on standardised tests. Management used these tests for school improvement (Intxausti etal., 2016). In addition, the management team was involved with the values of the school community, encouraged the desire of the entire educational community to learn and, when they belonged to schools in disadvantaged contexts, carried out greater involvement and saw difficulties as a challenge to be overcome (Day et al., 2016). 13 Pedagogika / 2020, t. 138, Nr. 2 Learning culture The culture of learning in high-efficiency schools had its particularities: there were more opportunities for participation, formal communication structures, collaborative decision-making processes, shared leadership and high expectations among students and teachers (Intxausti et al., 2016; Maroufkhani, Nourani & Boerhannoeddin, 2015; Tichnor-Wagner et al., 2016). In them, beyond the desire to raise scores on standardised assessment tests, there was an intrinsic motivation for improving education in general. The levels of effort and involvement in achieving this purpose were significantly higher in this type of school (Intxausti et al., 2016). The permanent training of teachers was optimal in high efficiency schools (Lizasoain et al., 2016). On the other hand, low efficiency centres justified low results by pointing to external factors and ineffective processes of training and innovation (Lizasoain et al., 2016). Permanent training in high efficiency schools was characterised by (Lizasoain etal., 2016) systematic implementation, a high degree of involvement located in the school itself (e.g. cascade training) and transfer to practice, which was facilitated by the school and always evaluated. On the other hand, research training of teachers and recurrent reflection on their own practice benefited schools (Mincu, 2015). These skills could make teachers the drivers of change in educational institutions. Educational approach The comprehensive education of the human being was the commitment of the teaching staff and was a key element in the improvement of schools with cultural diversity (Hajisoteriou et al., 2018). There were no discriminatory practices in high-efficiency schools. Awareness among students with special educational needs was higher. Any student who needs support in his or her education was considered to have special educational needs. Primitive definitions of this aspect, which focus more on disability, were not contemplated. Furthermore, the teaching staff had deeply rooted values, based on strong religious or social beliefs (Intxausti et al., 2017). It was clear that teachers in highly effective schools had a proactive attitude towards education, with empathy towards other educational agents and a vision of knowledge that was flexible and under constant review (Hajisoteriou et al., 2018; Intxausti et al., 2017). The composition of the school’s pupils also affected school efficiency. The diversity of genders is positive for effectiveness, perhaps because they had better behaviour (Ning, Van Damme, Van Den Noortgate, Yang & Pant, 2015; Van Hek et al., 2018). The classroom climate was one of the factors affecting performance when perceived positively, along with autonomy in the use of technologies or the adaptation of teaching (Gamazo et al., 2017). Classroom discipline would explain 11% of the variance between schools in reading achievement (Ning et al., 2015). More discipline was associated with better outcomes, although these levels varied from country to country. This could be explained, 20 Pedagogika / 2020, t. 138, Nr. 2 Kelcey, B., & Shen, Z. (2016). Multilevel design of school effectiveness studies in Sub-Saharan Africa.School Effectiveness and School Improvement,27(4), 492–510. http://dx.doi.org/10.10 80/09243453.2016.1168855 Ley orgánica 8/2013, 9 de diciembre, para la mejora de la calidad educativa [Organic Law 8/2013, December 9, for the improvement of educational quality] (LOMCE). Boletín Oficial del Estado, 295, de 10 de diciembre de 2013, 97858 a 97921. Retrieved from: http://www.boe.es/boe/ dias/2013/12/10/pdfs/BOE-A-2013-12886.pdf Li, W., & Konstantopoulos, S. (2017). Does class-size reduction close the achievement gap? Evidence from TIMSS 2011.School Effectiveness and School Improvement,28(2), 292–313. http://dx.doi.org/10.1080/09243453.2017.1280062 Lizasoain, L. & Angulo, A. (2014). Buenas prácticas de escuelas eficaces del País Vasco. Metodología y primeros resultados. [Best practices of highly effective schools in the Basque Country. Methodology and first results]. Participación Educativa, 3(4), 17–27. https://doi. org/10.4438/1886-5097-PE Lizasoain, L. L., Bereziartua, J., & Bartau, I. (2016). La formación permanente del profesorado en centros educativos de alta eficacia [The permanent training of teachers in highly efficient educational centers] Bordón. Revista de pedagogía, 68(2), 199–218. https://doi.org/10.13042/ Bordon.2016.68213 Manterola, C., Astudillo, P., Arias, E., Claros, N., & MINCIR, G. (2013). Systematic reviews of the literature: what should be known about them.Cirugía Española,91(3), 149–155. https:// doi.org/10.1016/j.ciresp.2011.07.009 Maroufkhani, P., Nourani, M., & Boerhannoeddin, A. B. (2015). High-performance work systems and school effectiveness: the case of Malaysian secondary schools.Asia Pacific Education Review,16(3), 461–475. http://dx.doi.org/10.1007/s12564-015-9389-2 Martimbianco, A. L. C., Ferreira, R. E. S., Latorraca, C. de O. C., Bussadori, S. K., Pacheco, R. L., & Riera, R. (2020). Photobiomodulation with low-level laser therapy for treating Achilles tendinopathy: a systematic review and meta-analysis.Clinical Rehabilitation, 34(6), 713– 722.https://doi.org/10.1177/0269215520912820 Martinez-Abad, F., & Chaparro-Caso, A. A. (2017). Data-mining techniques in detecting factors linked to academic achievement.School Effectiveness and School Improvement,28(1), 39–55. http://dx.doi.org/10.1080/09243453.2016.1235591 Martínez-Abad, F., Lizasoain, L., Castro-Morera, M., & Joaristi, L. M. (2017). Selección de escuelas de alta y baja eficacia en Baja California (México). [Selection of High and Low-Effectiveness Schools in Baja California (Mexico)]. Revista electrónica de investigación educativa, 19(2), 38–53. https://doi.org/10.24320/redie.2017.19.2.960 Maxwell, J. A. (2019). The Value of Qualitative Inquiry for Public Policy. Qualitative Inquiry, 26(2), 177–186. https://doi.org/10.1177/1077800419857093 Méndez, J. M. Á. (2014). Institutional assessment: between political pressing needs and school improvement needs.Policy Futures in Education,12(3), 445–454. http://dx.doi.org/10.2304/ pfie.2014.12.3.445 21 Pedagogika / 2020, t. 138, Nr. 2 Mincu, M. E. (2015). Teacher quality and school improvement: what is the role of research?Oxford Review of Education,41(2), 253–269. http://dx.doi.org/10.1080/03054985.2015.1023013 Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine 6(7): e1000097. http:// dx.doi.org/10.1371/journal.pmed.1000097 Murillo, F. J. (2005). La investigación sobre eficacia escolar. [Research on school effectiveness]. Barcelona: Octaedro. Murillo, F. J., Martínez-Garrido, C., & Hidalgo, N. (2014). Incidencia de la forma de evaluar los docentes de Educación Primaria en el rendimiento de los estudiantes en España. [The impact of the evaluation methodology of the Primary Education teachers on the performance of the student in Spain]. Estudios sobre Educación, 27, 91–113. http://dx.doi.org/10.15581/004.27.91-113 Murillo, F. J. (Coord.) (2007). Investigación Iberoamericana sobre Eficacia Escolar. [Ibero-American research on School Effectiveness]. Bogotá: Convenio Andrés Bello. Ning, B., Van Damme, J., Van Den Noortgate, W., Yang, X., & Gielen, S. (2015). The influence of classroom disciplinary climate of schools on reading achievement: a cross-country comparative study. School Effectiveness and School Improvement, 26(4), 586–611. https://doi.org/10.1080/ 09243453.2015.1025796 OECD (2017a). Education at a Glance 2017: OECD Indicators. Paris: OECD Publishing. http:// dx.doi.org/10.1787/eag-2017-en OECD (2017b). Educational Opportunity for All: Overcoming Inequality throughout the Life Course. Paris: OECD publishing. http://dx.doi.org/10.1787/9789264287457-en OECD (2019).Balancing school choice and equity:An international perspective based on Pisa. Paris: OECD Publishing. https://doi.org/10.1787/2592c974-en Özdemir, C. (2016). Equity in the Turkish education system: A multilevel analysis of social background influences on the mathematics performance of 15-year-old students. European Educational Research Journal, 15(2), 193–217. https://doi.org/10.1177/1474904115627159 Page, G. L., Martín, E. S., Orellana, J., & González, J. (2017). Exploring complete school effectiveness via quantile value added. Journal of the Royal Statistical Society: Series A (Statistics in Society), 180(1), 315–340. https://doi.org/10.1111/rssa.12195 Parra, J. D. (2018). Critical realism and school effectiveness research in Colombia: The difference it should make. British Journal of Sociology of Education, 39(1), 107–125. https://doi.org/10.1 080/01425692.2017.1330681 Pedroza-Zúñiga, L. H., Cetzal, P., Surema, R., & Lizasoain, L. (2018). Criterios para la identificación y selección de escuelas eficaces de nivel medio superior. [Criteria for the identification and selection of effective high schools].Revista electrónica de investigación educativa,20(1), 14–25. http://dx.doi.org/10.24320/redie.2018.20.1.2170 Perry, T. (2017). Inter-method reliability of school effectiveness measures: a comparison of value-added and regression discontinuity estimates.School Effectiveness and School Improvement,28(1), 22–38. http://dx.doi.org/10.1080/09243453.2016.1203799 22 Pedagogika / 2020, t. 138, Nr. 2 Prasertcharoensuk, T., & Tang, K. N. (2017). The effect of strategic leadership factors of administrators on school effectiveness under the office of Maha Sarakham primary educational service area 3.Kasetsart Journal of Social Sciences,38(3), 316–323. http://dx.doi.org/10.1016/j. kjss.2016.09.001 Prendes-Espinosa, M., García-Tudela, P., & Solano-Fernández, I. (2020). Gender equality and ICT in the context of formal education: A systematic review. Comunicar, 63, 9–20. https:// doi.org/10.3916/C63-2020-01 Ranjan, M. (2014). School Effectiveness at Primary Level of Education in Relation to Classroom Teaching. International Journal of Instruction, 7(2), 51–64. Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C., & Stringfield, S. (2014). Educational effectiveness research (EER): A state-of-the-art review.School Effectiveness and School Improvement,25(2), 197–230. http://dx.doi.org/10.1080/09243453.2014.885450 Rutledge, S. A., Cohen-Vogel, L., Osborne-Lampkin, L. T., & Roberts, R. L. (2015). Understanding effective high schools: Evidence for personalization for academic and social emotional learning. American Educational Research Journal, 52(6), 1060–1092. https://doi. org/10.3102/0002831215602328 Salgado, M., Marchione, E., & Gilbert, N. (2014). Analysing differential school effectiveness through multilevel and agent-based modelling.Journal of Artificial Societies and Social Simulation,17(4), 3. http://dx.doi.org/10.18564/jasss.2534 Scheerens, J. (2000).Improving school effectiveness. Fundamentals of Educational Planning, 16. Paris: UNESCO, International Institute for educational planning. Scheerens, J. (2014). School, teaching, and system effectiveness: some comments on three state- of-the-art reviews. School Effectiveness and School Improvement, 25(2), 282–290. http://dx.doi. org/10.1080/09243453.2014.885453 Scheerens, J. (2015). Theories on educational effectiveness and ineffectiveness. School Effectiveness and School Improvement, 26(1). 259–289. https://doi.org/10.1080/09243453.2013.858754 Scheerens, J., Luyten, H., & van Ravens, J. (2011). Measuring educational quality by means of indicators. In Scheerens J., Luyten H., & van Ravens J. (Eds.),Perspectives on educational quality (pp. 35–50). Dordrecht: Springer. http://dx.doi.org/10.1007/978-94-007-0926-3_2 Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher, 44(7), 371–386. https://doi.org/10.3102/0013189X15603982 Stoll, L., & Fink, D. (1996).Changing our schools: Linking school effectiveness and school improvement. Chicago: Open University Press. Sulis, I., & Porcu, M. (2015). Assessing divergences in mathematics and reading achievement in Italian primary schools: A proposal of adjusted indicators of school effectiveness. Social Indicators Research, 122(2), 607–634. https://doi.org/10.1007/s11205-014-0701-z Thessin, R. A. (2015). Identify the best evidence for school and student improvement.Phi Delta Kappan,97(4), 69–73. http://dx.doi.org/10.1177/0031721715619923 23 Pedagogika / 2020, t. 138, Nr. 2 Tichnor-Wagner, A., Harrison, C., & Cohen-Vogel, L. (2016). Cultures of learning in effective high schools. Educational Administration Quarterly, 52(4), 602–642. https://doi. org/10.1177/0013161X16644957 Tobin, M., Nugroho, D., & Lietz, P. (2016). Large-scale assessments of students’ learning and education policy: synthesising evidence across world regions. Research Papers in Education, 31(5), 578–594. https://doi.org/10.1080/02671522.2016.1225353 Troncoso, P., Pampaka, M., & Olsen, W. (2016). Beyond traditional school value-added models: a multilevel analysis of complex school effects in Chile. School Effectiveness and School Improvement, 27(3), 293–314. https://doi.org/10.1080/09243453.2015.1084010 Tsai, S. L., Smith, M. L., & Hauser, R. M. (2017). Families, schools, and student achievement inequality: A multilevel MIMIC model approach.Sociology of Education,90(1), 64–88. https:// doi.org/10.1080/09243453.2014.885450 Valenzuela, J. P., Bellei, C., & Allende, C. (2016). Measuring systematic long-term trajectories of school effectiveness improvement.School Effectiveness and School Improvement,27(4), 473–491. https://doi.org/10.1080/09243453.2016.1150861 Van Hek, M., Kraaykamp, G., & Pelzer, B. (2018). Do schools affect girls’ and boys’ reading performance differently? A multilevel study on the gendered effects of school resources and school practices. School Effectiveness and School Improvement, 29(1), 1–21. https://doi.org/ 10.1080/09243453.2017.1382540 Weber, G. (1971). Inner city children can be taught to read: four successful schools. Occasional paper, 18. Washington, DC: Council for Basic Education. Kas publikuojama cituojamo indekso žurnaluose apie mokyklos efektyvumą? Sisteminė tyrimo rezultatų ir metodų apžvalga Jesús García-Jiménez1, Juan Jesús Torres-Gordillo2, Javier Rodríguez-Santero3 1 Sevilijos universitetas, Edukacinių tyrimų metodų ir diagnostikos katedra, C/Pirotecnia, s/n, ES-41013, Sevilija, Ispanija, [email protected] 2 Sevilijos universitetas, Edukacinių tyrimų metodų ir diagnostikos katedra. C/Pirotecnia, s/n, ES-41013, Sevilija, Ispanija, [email protected] 3 Sevilijos universitetas, Edukacinių tyrimų metodų ir diagnostikos katedra, C/Pirotecnia, s/n, ES-41013, Sevilija, Ispanija, [email protected] Santrauka Švietimo efektyvumas mokslo ir socialiniu lygmeniu yra viena iš dominuojančių temų, nagrinėjamų daugelyje mokslinio poveikio straipsnių su aukštu citavimo indeksu (angl. Impact 24 Pedagogika / 2020, t. 138, Nr. 2 Factor). Straipsniai publikuojami referuojamose duomenų bazėse, tokiose kaip WoS ir Scopus. Šiuo tyrimu siekiama atskleisti metodologines švietimo efektyvumo mokykloje tendencijas, aprašomas straipsniuose, išspausdintuose referuojamose JCR (WoS) ir SJR (Scopus) duomenų bazėse (2014– 2018 m.), ir aprašyti jų rezultatus. Buvo išanalizuoti šešiasdešimt septyni straipsniai. Remiantis keturiasdešimt penkiais atrinktais ir nustatytus kriterijus atitikusiais straipsniais, atlikta sisteminė šių straipsnių apžvalga. Tyrimo rezultatai atskleidžia, kad straipsniuose dažniausiai remiamasi kiekybine tyrimo metodologija, o jos pagrindu atliktus tyrimus analizuojantys straipsniai yra publikuojami žurnaluose, reitinguojamuose pirmojo ar antrojo kvartilio lygmeniu pagal citavimo indeksą. Apibendrinti mokyklos, kaip švietimo organizacijos, efektyvumo ypatumai, kurie buvo suskirstyti į keturias kategorijas, apima švietimo požiūrį, švietimo praktiką, švietimo vadybą bei mokymosi kultūrą. Taip pat atsižvelgiama į individualius ir kontekstinius švietimo efektyvumo faktorius bei įtaką akademinių pasiekimų rezultatams. Remiantis tyrime apžvelgtais straipsniais nustatyta, kad, nors ir egzistuoja esminiai veiksniai, darantys įtaką švietimo efektyvumui, kiti kintamieji mokyklos lygmeniu taip pat turi įtakos, pavyzdžiui: mokymosi kultūra, vadovų komanda, edukacinė prieiga ir švietimo praktika. Esminiai žodžiai: švietimo efektyvumas, švietimo kokybė, švietimo rezultatai, mokyklos mokinių santykiai. Gauta 2020 05 21 / Received 21 05 2020 Priimta 2020 07 03 / Accepted 03 07 2020