Quality of Education and Its Evaluation : An Analysis of the Russian Academic Discussion

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=meue20 European Education ISSN: 1056-4934 (Print) 1944-7086 (Online) Journal homepage: http://www.tandfonline.com/loi/meue20 Quality of Education and Its Evaluation: An Analysis of the Russian Academic Discussion Galina Gurova, Nelli Piattoeva & Tuomas Takala To cite this article: Galina Gurova, Nelli Piattoeva & Tuomas Takala (2015) Quality of Education and Its Evaluation: An Analysis of the Russian Academic Discussion, European Education, 47:4, 346-364, DOI: 10.1080/10564934.2015.1107377 To link to this article: https://doi.org/10.1080/10564934.2015.1107377 Published with license by Taylor & Francis Group, LLC© 2015 Galina Gurova, Nelli Piattoeva, and Tuomas Takala Published online: 07 Dec 2015. Submit your article to this journal Article views: 445 View Crossmark data European Education, 47: 346–364, 2015 ISSN: 1056-4934 print/1944-7086 online DOI: 10.1080/10564934.2015.1107377 Quality of Education and Its Evaluation: An Analysis of the Russian Academic Discussion Galina Gurova, Nelli Piattoeva, and Tuomas Takala University of Tampere The article describes and critically analyzes how Russian education researchers approached the topic of quality evaluation in education between 1990 and 2014. Evaluation and quality have grown into major policy issues in education across the world, simultaneously acting as powerful steering mechanisms on national and transnational levels. Russia is no exception to this global phenomenon, but little is known about how Russian education researchers discuss the topic in national academic journals. This article discusses four major periods, each characterized by a shift in the focus of discussion and/or the introduction of a completely new agenda. These periods capture the dominant themes, titled “effective management and customization of education,” “the rise of broad-scale assessments,” “systemic approach to quality evaluation,” and “toward a more nuanced usage of evaluation data.” We interpret the findings within two intertwined conceptual frameworks: governance at a distance and New Public Management. How these frames help us understand the academic discussion on quality evaluation in Russian school education is also discussed. INTRODUCTION Studying education policy necessarily includes exploring the context of its elaboration. Researchers pay attention to social, economic, and political contexts in order to enhance their understanding of the policy process. At the same time, researchers increasingly emphasize the importance of a different kind of context—the ideas, discussions, research, and expert advice surrounding and facilitating the processes of policy making (see, e.g., Grek & Ozga, 2010; Tripney, Kenny, & Gough, 2014; Tseng, 2012). Policies do not simply stem from “objective” circumstances; they are “discursive products” embedded in wider environments of understanding and meaning making (Kauko, Simola, Varjo, & Kalalahti, 2012; Pashby, 2013). Kauko et al. (2012) write about “discursive formations” or “problematiques” that introduce certain political possibilities based on what is defined as a “problem” and consequently politicized. They emphasize that “there is a need to trace genealogies of problems that are specific to a certain discursive field” (p. 222). © 2015 Galina Gurova, Nelli Piattoeva, and Tuomas Takala. Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercialNoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. Address correspondence to Galina Gurova, School of Education, University of Tampere, FI-33014, Finland. E-mail: [email protected] National academic discussion constitutes an important discursive context for policies, influencing and being influenced by them. It demonstrates the genesis of thinking and changes of discourse, and consequently, the history of national education policies in their broader sociopolitical context. It outlines the intellectual space within which research and arguably policy making takes place. Thus, education policy scholars investigate the role of theory and research evidence in politics because the “concepts and arguments developed by researchers play a key role in establishing what is ‘in the true’ and hence ‘real’” (Bacchi, 2012, p. 6). Based on these viewpoints, an analysis of the national academic discussion as a distinct, though far from independent, discursive field can shed light on the national body of knowledge in the area studied, highlight and examine the themes that attract most attention, and indicate issues that have been neglected. Despite the global reach of the quality of education and its measurement agenda (see, e.g., Lingard, Martino, & Rezai-Rashti, 2013; Valverde, 2014), the topic has been discussed primarily in the context of the OECD or EU member countries. Little is known about this phenomenon in Russia (the existing publications take other foci; see, e.g., the juxtaposing of the neoliberal concept of quality with national educational settings in Minina (2013). This article focuses on the academic discussion on quality evaluation in school education in Russian national academic publications. Our interest in evaluation and quality stems from the fact that the two constitute a major policy concern in education across the world, simultaneously acting as powerful steering mechanisms at national and transnational levels (Ozga, Dahler-Larsen, Segerholm, & Simola, 2011; Valverde, 2014). Tracing the discussion on quality and evaluation in school education in Russian academic sources in the period from 1990 to 2014, this article is guided by the following broad questions: (1) How is quality of education and evaluation of quality discussed in Russian academic literature? Which aspects have received most attention and which ones have been neglected? (2) How has the discussion changed over time? Thus, this article analyzes how the discussion on education quality evaluation in general and certain evaluation mechanisms in particular developed over time, and not on what actually happened in the field (although the discussions arguably facilitated certain developments). Our analysis focuses on a context with distinctively different ideological history, administrative practices, and educational research tradition. While rooting our intellectual framework in the (Western) education policy literature that presents quality measurement as a central means of governance at a distance and New Public Management (NPM)-inspired reforms, we remain sensitive to the possibility of other interpretations. Here, our starting point is that (Western) explanations that link the rise of quality control and audit cultures in schools to the ideology of neoliberalism (and by extension, to NPM and governance from a distance) might impoverish the analysis by applying neoliberalism as an encompassing regime of truth (Kipnis, 2008). As Kipnis (2008) writes, “The global rise of audit cultures needs to be understood in a broad, anthropological, comparative framework, not one narrowly concerned with a critique of ideas that diffuse from the West” (p. 286). Concurring with this position, we refrain from making assumptions on the origin of the ideas discussed on the pages of the examined Russian publications, and simultaneously pay attention to the meanings that lie outside the utilized conceptual frame. The article represents the first phase of a research project, which will proceed to the study of national and local level policy documents on quality evaluation in Russia, as well as to interviewing key actors at different levels and conducting ethnographic fieldwork in selected schools, thus enabling us to return to the issue of origin in subsequent work. QUALITY OF EDUCATION AND ITS EVALUATION 347 CONCEPTUALIZATIONS OF EVALUATION AND QUALITY MEASUREMENT AS MECHANISMS OF GOVERNANCE Major works in education policy research have recently approached the topic of quality assurance and evaluation from two intertwined theoretical angles: as a tool of governance at a distance and as a manifestation and a means to introduce New Public Management reforms in education and other social spheres (Ferlie, Musselin, & Andresani, 2008; Hood, 1991; Lawn & Grek, 2012; Miller & Rose, 1990). Governing at a distance is understood here as a specific mode of governance or “governmentality” based on the idea of acting from a center on the habits, desires, and actions of the spatially, culturally or organizationally distant others (Miller & Rose’s concept as described in Rose, O’Malley, & Valverde, 2006, pp. 91–92). This mode of governance implies that the subjects preserve a significant degree of autonomy and “voluntarily” engage in the center-initiated processes that shape their conduct (Miller & Rose, 1990, p.14). Governing at a distance is capable of imposing the logic of the center over scattered and diverse places and producing a common “disciplining and enabling space” (Lawn & Grek, 2012, p. 82). It is this capacity to create cohesion and commonality amidst diversity that makes this mode of governance suitable for most complex political tasks such as “creating Europe,” as powerfully described by Lawn and Grek (2012; also Nóvoa & Yariv-Mashal, 2003). There are several ways in which evaluation and quality assurance procedures contribute to governing at a distance. They set and enforce particular standards against which educational institutions’ work is measured, legitimizing reward and punishment by the authorities. Evaluation procedures produce numerical information that is then used for developing governing strategies and programs, as well as for comparison and benchmarking, which have emerged as cordial steering tools (Nóvoa & Yariv-Mashal, 2003). Governing complex spaces and phenomena is now reinterpreted as a problem of data and its proper collection and analysis (Lawn & Grek, 2012, p. 9; Ozga et al., 2011; Rose & Miller, 1992). The very processes of data production modify behavior of the subjects who produce this data through making them think according to certain norms and pay attention to particular, predetermined aspects of the evaluated phenomena. Data collection redefines practices through the logic of “what is counted, counts,” especially when evaluation processes bear high stakes for the evaluated persons and institutions through redistribution of funds or recruitment decisions. Equally, the processes of data collection as mandated by the authorities contribute to building or strengthening hierarchies and unequal power relations between those who assemble the data and those who are acted upon as data (Piattoeva, 2015; Selwyn, Henderson, & Chao, 2015). Quality assurance procedures increasingly involve self-evaluation practices which, while seemingly permissive and self-initiated, still have to stay in line with the prescribed indicators and central standards, and put emphasis on constant self-regulation and self-improvement (Lawn & Grek, 2012, p. 146). Yet another tool is provided by the public nature of evaluation results, which now appear in diverse rankings, ratings, and reports publicized in and often prepared by commercial media for the purpose of modulating everyday decisions and actions of the consumers of educational services. This development can be viewed as empowering a certain privileged category of citizens as political subjects who are expected to influence educational institutions by means of rational and informed consumer choice (for more discussion on media-assisted, data-driven governing from afar, see Piattoeva, 2015). 348 GUROVA, PIATTOEVA, AND TAKALA Another perspective on evaluation and quality assurance as tools of governance is elaborated in the literature that studies NPM reforms in education. NPM is based on the ideal of transplanting market mechanisms, primarily competition, cost-effectiveness, and empowerment of managers, stakeholders and consumers, to the public sector. NPM’s “business approach” effectively redefines public services as traded commodities, and replaces professional ethics with demand for accountability. The organizations that directly “serve customers” are granted greater autonomy to enable a more “customized” and efficient service, while authorities steer them through setting targets, defining outcomes rather than inputs, and introducing incentives (for more on NPM see e.g., Diefenbach, 2009; Hood, 1991). Performance measurement and management play a key role in the NPM approach (cf. “auditisation variant of NPM” in Ferlie et al., 2008). The evaluation data is utilized for performancebased funding and payment schemes, which evoke competition between and within the assessed institutions, and at the same time disguise the cost-reduction measures undertaken by the authorities in their struggle against the so-called inefficiency. The publication of evaluation results serves competition purposes, as well as the objective to increase accountability of educational institutions to their “stakeholders.” NPM promotes external evaluation procedures and numerical data as tools of increased transparency and efficiency, and seeks to empower “customers” by involving them in the evaluation procedures. We employ the aforementioned conceptualizations as heuristic frames and probe whether Russian academic publications bear the traits of governance at a distance and NPM-related positions in discussing particular forms of quality evaluation through time. QUALITY EVALUATION IN THE SOVIET AND RUSSIAN CONTEXTS International sources often depict the Soviet education system as lacking in any quality assurance and evaluation policy. They infer that since there was no standardized achievement testing at either school graduation or university entry, and no systematic feed of school examination results to the Ministry of Education, the authorities lacked a sufficient basis for decision making1 (Bakker, 1999; West & Crighton, 1999). However, the notion of quality and ideas about its measurement and improvement were not absent from the Soviet education policy discourse, though they appeared in a different guise. Already in the 1940s the government issued decrees “On the improvement of the quality of teaching in schools” that introduced compulsory examinations and a five-tier grading system. Another document issued by the ministerial Collegium recommended to “collect and summarize the best teachers’ experience” (Kukulin, Mayofis, & Safronov, 2015, pp. 643–47). Notably, the examination and grading reform was accompanied by a decree that condemned the practice of socialist competition in school education and prohibited the evaluation of school or teacher quality based on students’ progress in learning (ibid.). By the end of the Soviet period, as both national and international sources suggest, whenever the quality of schools needed to be described, it was usually captured by 1 Some authors of the articles in international sources were involved in international projects on education reforms in Russia, and their arguments often repeat those presented in the reports by the OECD and the World Bank; for example, they criticize the Russian education system for lacking statistics on learning achievements, and for executing examinations that are subjective and noncommensurable in terms of the final results (see, e.g., Bakker, 1999, 2012). QUALITY OF EDUCATION AND ITS EVALUATION 349 the numbers of students with high marks (4 and 5), the percentage of students who entered higher education, the number of gold and silver medalists in subject Olympics and the number of students who continued their tertiary education in prestigious higher education institutions (see, e.g., Bakker, 1999, p. 296). In general, however, Soviet education policies were based on the idea that clearly defined principles (curricular contents, lesson plans, textbooks), sufficient inputs, and well-implemented processes (pedagogies and leadership) would lead to satisfactory education outcomes (West & Crighton, 1999). Public and professional demands for change in Russian education were voiced already in the 1980s in the spirit of the perestroika movement toward a more democratic society. In 1988, Eduard Dneprov, who later became the first minister of education of post-Soviet Russia, was appointed to lead a newly established collective of like-minded reformer pedagogues under the auspices of the VNIK “Bazovaia shkola” (Temporary Scientific Research Collective on the Schools) within the Academy of Pedagogical Sciences. The VNIK participants paved the way for the new thinking, producing a number of central documents that later became the foundation for the post-Soviet educational reforms and new legislation (Kukulin et al., 2015, p. 674; also Long & Long, 1999). After the dissolution of the Soviet Union, the new Russian Law on Education from 1992 was formulated in such terms as “humanization,” “differentiation,” “democratization,” and “pluralization” (Rust, 1992, cited in Polyzoi & Dneprov, 2011, p. 161). The Law enabled decentralization and shifted particular administrative and fiscal responsibilities to regional and local authorities. The ideal of decentralization also unfolded as support for school-level management, increased school choice for families and introduction of nonstate schools, as well as more freedom to generate school-based curricula in all schools regardless of their status (see, e.g., Long & Long, 1999, p. 83). Decentralization is often blamed on the harsh financial reality of the early 1990s that simply left central authorities with no choice other than transfer fiscal responsibility onto the regional and municipal shoulders. However, without denying this reality, it is important to bear in mind the new political and pedagogical thinking that motivated the changes. For instance, the reformers promoted the idea that “educational processes should be governed by those who teach,” meaning that school-based staff should have a greater say in a wide scope of decisions that concern schoolwork (Long & Long, 1999, p. 88). In similar vein, the school administration was envisioned as a communal enterprise that should involve students and parents, as well as teachers and representatives of civil society at large. Regionally and locally tailored education was expected to better serve the local needs in the diverse Russian territories. In practice, coupled with underfunding, decentralization led to increasing inequalities between regions and within them (Polyzoi & Dneprov, 2011). Another major change concerned the overall reinterpretation of education as an education market, leading to partial privatization and commercialization of the school sector (Gounko & Smale, 2007; Silova & Brehm, 2013). The change of political leadership in 1999 and the rapid recovery of the national economy and finances heralded a new phase in the state policy making, including education reforms. The new National Doctrine for Education (2000), the Concept of Modernization of Russian Education (2001), and the Federal Strategic Program for the Development of Education (2005) stressed the role of education in securing the nation’s economic growth, global competitiveness, and human capital development. They also further promoted the introduction of market mechanisms into the education sector and emphasized the importance of efficiency and accountability of education institutions (Gounko & Smale, 2007, pp. 540–541). 350 GUROVA, PIATTOEVA, AND TAKALA It is the modernization reform package of the 2000s that granted the topic of quality of education a dominant position. Several major measures were introduced in the name of quality assurance, including the new state education standards (the part of the curriculum compulsory in all parts of the Russian Federation) and the national tests of educational achievement: the Unified State Exam (USE) executed after the completion of eleven-grade school education, and the State Final Attestation (SFA) undertaken in the last grade of compulsory basic education in grade nine. The State Program for Education Development in 2013–2020 brought a comprehensive system of education evaluation and quality control, composed of state regulation of education activities, assessment of education achievement, procedures of independent quality evaluation, including expert-public accreditation of education programs and self-evaluation of education institutions, and the participation of Russia in international studies (Government of Russia, 2012, p. 218). Of all these, the USE deserves particular mention due to its exceptionally profound influence on the whole education system. Developed as a tool for measuring educational achievement and a means to ensure more equal access to higher education, the USE gradually became a major instrument for measuring overall education quality and ensuring compliance with the official school curricula (Piattoeva, 2015; Tyumeneva, 2013). It shifted attention to measurable outcomes and efficiency, signaling the introduction of an output-based curriculum in Russia, and facilitated the centralization of school education governance. The standardized examination was implemented at the time of growing dissatisfaction with the decentralization process that went too far and endangered national cohesion, highly polarizing debates on the newly developed school standards. The latter largely concerned the applicability of the competenceand outcome-based approach to education in Russia, as promulgated by the international studies of learning achievements, which Russia ‘failed’ (see Lenskaya, 2013; Minina, 2014). Researchers note that the national strategy for the modernization of education adopted by the Russian government since the late 1990s conformed to the recommendations of international organizations (primarily, the World Bank and the OECD), whose “standard package centered around the concepts of cost-effectiveness, market-driven quality control, educational standardization, outcome-based education, decentralization of governance and finance, and privatization of higher education” (Minina, 2013, p. 21; see also Gounko & Smale, 2007; Silova, 2010). The changes that introduced comprehensive evaluation of quality in education also resemble policy advice from international organizations. Throughout the 1990s, reports by the World Bank and the OECD on Russian education argued for a reform of the examination system, the need for statewide educational statistics, and a transition from direct control to indirect means of ensuring compliance through evaluation and accountability (OECD, 1998; World Bank, 1995). These changes took place in the context of the newly introduced NPM-inspired forms of state regulation, including steering by results, and these reforms, too, had strong links to loan conditionalities and expert advice provided by international players (see Gusarova & Ovchinnikova, 2014). SOURCES AND METHODS OF ANALYSIS Several sources were used to identify relevant Russian language academic literature on quality and evaluation in education. First, a search was undertaken within two Russian academic QUALITY OF EDUCATION AND ITS EVALUATION 351 journals in education sciences: (Voprosy obrazovaniia, Educational Studies in its English version) and (Narodnoe obrazovanie, Popular Education). These sources were chosen because they belong to the leading journals in education sciences (along Pedagogika, Obrazovaniye i Nauka and Director Shkoly) and publish regularly on quality and evaluation in education. Our choice of the two aforementioned journals was motivated by an intellectual interest in sources with distinctively different positions and histories. Founded in 1803, Narodnoe obrazovanie is the oldest Russian education journal, and for a long time it served as the official journal for the Ministry of Education. Currently, it seeks to reflect a wide range of views on education, publishing pieces authored by scholars as well as administrators and teachers (Zamost’ianov, 2012). Voprosy obrazovaniia was established in 2004 by the National Research University– Higher School of Economics (HSE), being one of the youngest influential academic journals in education. Although it appears to be more academically oriented, which is arguably the reason for a narrower circle of authors, Voprosy obrazovaniia differs from its “Western” counterparts in publishing Russian translations of reports and recommendations produced, for instance, by the OECD and other influential education policy actors (also such individuals as Sir Michael Barber and Eric Hanushek, whose works are a standard reference in the World Bank’s and Pearson’s policy papers, for instance). In this manner, the journal could be said to channel particular educational ideals without necessarily engaging in their analytical appraisal. By selecting two leading journals with divergent histories, publication policies, and, as it appears, ideological positions, we sought to better capture the breadth of the academic discussion without seeking to compare articles from the two publication venues. As was revealed in the process of data analysis, the circle of authors publishing on quality and evaluation was to a considerable degree similar in the two journals. Apart from the aforementioned journals, additional publications were found through keyword searches in Google Scholar and the four national databases: eLIBRARY.RU, CyberLeninka, web-portal “Economika. Sociologia. Management” and the electronic library of the HSE. Book editions from the 1990s that are by and large not referenced in electronic databases were found by searching the catalogue of the Konstantin Ushinsky Scientific Pedagogical Library in Moscow. A few additional sources were selected from the reference lists of the analyzed publications. Finally, we took into consideration the recommended course literature from two MA-level courses taught at the HSE, “Measurements and evaluation of quality in education” and “National and international programs for evaluating educational achievement.” These courses are unique in Russian higher education, and the programs in which they are taught are among the very few training specialists in education administration and measurement in education. Hence this course literature can be argued to exert particular influence on future Russian education evaluation professionals. The keywords employed in the publication searches were оценка качества образования (otsenka kachestva obrazovaniia; evaluation of educational quality) and управление качеством образования (upravlenie kachestvom obrazovaniia; assurance, or management, of educational quality). From the resulting sources, we selected the texts that concerned school education. The selection process resulted in about 160 pieces published between 1990 and 2014. Of these we read 120 relevant articles (which ranged from three to over 20 pages) and 17 books for which full texts were available. We relied on qualitative content analysis in order to construct the analytical categories from our data (see Bryman, 2004, p. 542; Creswell, 2012). Each source was first examined from the 352 GUROVA, PIATTOEVA, AND TAKALA perspective of its key themes; this analysis was followed by identifying key arguments within each key theme. Based on these, five meta-categories were created, each encompassing diverse themes and arguments: quality and evaluation regarding students’ learning achievement, quality of teachers’ work, quality of school management, quality and its evaluation on the regional and/ or national level, and comparison with other countries, including discussions on international comparative studies. These meta-categories were not mutually exclusive as one article often matched several meta-categories. The sources were then regrouped by publication date in order to see whether there were any thematic similarities within each group and how argumentation changed throughout the years. Due to the scarcity of sources published from 1991 to 1998, these were combined into a single group, and, beginning from 1999, publications were grouped by the year of publication. These initial groups were then reorganized into bigger blocks based on two criteria: proximity in time of publishing and similarity of the discussion foci. This procedure resulted in discerning four periods: from 1990 to 1999, from 2000 to 2004, from 2005 to 2010, and from 2011 onward. Within each period, the academic discussion of quality evaluation concentrated on issues differing from those in the previous or following periods. The thematic differences between the periods are in no way clear-cut. Some strands pervade all periods, while others disappear and reemerge after a while (often connected to the careers of the authors). RESULTS: SHIFTING FOCI OF RESEARCH ON QUALITY OF EDUCATION Overall, the four periods identified in our research capture the dominant themes, which could be titled “effective management and customization of education” (1990–1999), “the rise of broadscale assessments” (2000–2004), “systemic approach to quality evaluation” (2005–2010), and “towards a more nuanced usage of evaluation data” (2011–present). 1990–1999: Effective Management and Customization of Education The publications of the 1990s discussed quality evaluation primarily at the school level in relation to so-called “pedagogical management” (Maiorov, 1998; Potashnik & Moiseev, 1999a, 1999b; Simonov, 1995; Tret’iakov, 1997). The ultimate goal was “good” or “effective” management of schools, rather than “quality education” per se, as the latter was assumed to be essentially different in different schools. Tret’iakov (1997) explains the raising demand for information in the following way: The renewal of the ways in which the schools are managed is connected, first of all, to the development of the system of information-analytical work as the main instrument of management. … The requirements [to the processes of information production and utilization—authors] increase dramatically in the conditions of democratization of school life. The school leaders’ need of information grows, as during transfer to self-regulation the ones who organize the education process seek for new ways to increase the effectiveness of their management activities. (p. 18) The authors suggested that in order to monitor quality school management should develop measurable indicators for the desired outcomes (such as the psychological health of students or the practical relevance of the subjects studied). In fact, “managing by outcomes” appeared QUALITY OF EDUCATION AND ITS EVALUATION 353 management, self-evaluation and customization appeared in the early 1990s, before the NPM’s arrival to the scene as a government technology of the Russian state. Moreover, those seemingly neoliberal, “NPM-istic” practices still lacked in two important characteristics central to the neoliberal audit culture—performance measurement as a standardized, numerical procedure, and the practice of external audits. The latter is important to keep in mind since neoliberalism is an ideology of a small but strong state that governs through market mechanisms at a distance (Harvey, 2005). The authors who wrote in the 1990s appear as antistatist in the sense that they wanted the schools to receive, to a greater or lesser extent, full freedom of operation. However, the ongoing reconsolidation of state power through evaluation (see Piattoeva, 2015) is largely supported in many recent academic discussions that moved from exploring school-level management to evaluation of the national education system as a whole. The controversies apparent in the Russian academic discussion raise the question as to how to understand the political ends and the fundamental values that underlay those practical suggestions. Were the Russian academic community or some of its representatives, exposed to the NPM techniques at a very early stage, converted and began to speak on their behalf even before NPM became an important source of statecraft for the Russian leaders? Alternatively, should the academic debates be interpreted within a different discursive and ideological frame, namely that of democratization and empowerment of the grassroots level regardless of the frequency of such market-oriented terms as customers and consumers, and thus as an intellectual counterreaction to the position of the Soviet state? Overall, what appears as strong elements of the language and tools of NPM cannot necessarily be traced back to that source, underlining, as we already argued in the beginning, the need to avoid a diffusionist mode of thinking that conflates, often erroneously, practical means with the underlying “regimes of truth” (see Kipnis, 2008). There is always a possibility for “[a] relatively independent inventions of governing practices [that] can be masked by the borrowing of labels and terms to give an “international” gloss to what was really being thought locally” (ibid., p. 285). At the same time, there is a chance that local social and political circumstances help to diffuse particular, travelling technologies of governing without absorbing the ideologies that invented them (ibid.). Otherwise, how else to explain the fact of the strong presence of the theme of equality as an objective of national-level evaluation practices in the Russian academic discourse. The materials analyzed in this article point to the importance of exploring this type of questions in subsequent work. Such research tasks lie at the heart comparative of education as an inquiry into the transfer, translation, and transformation of educational processes. AUTHOR BIOS Galina Gurova is a doctoral candidate; her dissertation examines the impact of quality evaluation policies on general education in Russia. School of Education, University of Tampere, Finland. Dr. Nelli Piattoeva is a university lecturer at the School of Education, University of Tampere, Finland; her research deals with the politics performance measurement in Russian general education. Professor Tuomas Takala specializes in comparative education and education policies in developing countries. School of Education, University of Tampere, Finland. 360 GUROVA, PIATTOEVA, AND TAKALA ACKNOWLEDGMENTS We are grateful to the three anonymous reviewers and our partners from the State University, Higher School of Economics (Russia) for their intellectual contribution and comments on the drafts of this article, and for granting access to the university library facilities. FUNDING The research for this article was supported by the Academy of Finland grant 2501273874 for the research project “Transnational Dynamics of Quality Assurance 640 Policies and Evaluation in Brazil, China and Russia” 2014–17. REFERENCES Agranovich, M. (2004). Vozmozhnosti analiza obrazovatel’nykh sistem na osnove rezul’tatov EGE [Opportunities for analysis of educational systems on the basis of USE]. Voprosy obrazovaniia, 2, 272–287. Agranovich, M. (2006). Kompleksnaia otsenka regional’nykh obrazovatel’nykh system [Complex evaluation of regional education systems]. Obrazovatelnaya Politika, 7, 24–27. Agranovich, M., Konstantinovskii, D., & Loginova, O. (2003). Monitoring, otsenka kachestva i statistika obrazovaniia [Monitoring, quality evaluation and educational statistics]. MSSES. Astapov, M., Khlopova, T., & Semenko, E. (2008). Eshche raz o rezul’tatakh EGE kak instrumente upravleniia kachestvom obrazovaniia [Once again on the results of USE as an instrument of managing quality of education]. Narodnoe obrazovanie, 8, 106–110. Bacchi, C. (2012). Why study problematizations? Making politics visible. Open Journal of Political Science, 2(01), 1–8. doi:10.4236/ojps.2012.21001. Bakker, S. A. (1999). Educational assessment in the Russian federation. Assessment in Education: Principles, Policy & Practice, 6(2), 291–303. doi:10.1080/09695949992928. Bakker, S. A. (2012). Introduction to external, independent testing in “New Countries”: Successes and defeats of the introduction of modern educational assessment techniques in former soviet and socialist countries. Educational Measurement: Issues and Practice, 31(82), 38–44. doi:10.1111/j.1745-3992.2012.00234.x. Batrova, O. (2010). Gosudarstvenno-obshchestvennyi kontrol’ i otsenka deiatel’nosti shkoly [State-public control and evaluation of the work of the school]. Narodnoe obrazovanie, 2, 88–95. Bochenkov, S. (2013). Uchitel’, shkola, sistema obrazovaniia v zerkale EGE [Teacher, school, system of education in the mirror of USE]. Problemy sovremennogo obrazovaniia, 3, 27–47. Bochenkov, S., & Valdman, I. (2013). Interpretatsiia i predstavlenie rezul’tatov EGE: Problemy i vozmozhnye resheniia [Interpretation and presentation of the results of USE: Problems and possible solutions]. Voprosy obrazovaniia, 3, 5–24. Bolotov, V. (2004). EGE: Promezhutochnye itogi [USE: Interim results]. Voprosy obrazovaniia, 2, 155–167. Bolotov, V. (2007). Stanovlenie obshcherossiskoi sistemy otsenki kachestva obrazovaniia [Formation of the all-Russia system of evaluation of educational quality]. Vestnik obrazovaniya Rossii, 14, 46–51. Bolotov, V., & Valdman, I. (2012). Usloviia effektivnogo ispolzovaniia rezul’tatov otsenki obrazovatel’nykh dostizhenyi shkol’nikov [Conditions for effective usage of the results of assessment of educational achievement of school students]. Pedagogika, 6, 39–45. Chelyshkova, M. (2002). Teoriia i praktika konstruirovaniia pedagogicheskikh testov [Theory and practice of constructing educational tests]. LOGOS. Creswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches. Sage Publications. Diefenbach, T. (2009). New public management in public sector organizations: The dark sides of managerialistic “enlightenment.” Public Administration, 87(4), 892–909. doi:10.1111/j.1467-9299.2009.01766.x QUALITY OF EDUCATION AND ITS EVALUATION 361 Efendiev, A., & Reshetnikova, K. (2004). Pervye rezul’taty EGE: Analiz sotsial’nykh posledstvii i tendentsii [First results of USE: Analysis of social consequences and tendencies]. Voprosy obrazovaniia, 2, 288–310. Efremova, N. (2003). Sovremennye testovye tekhnologii v obrazovanii [Modern testing technologies in education]. LOGOS. Ferlie, E., Musselin, C., & Andresani, G. (2008). The steering of higher education systems: A public management perspective. Higher Education, 56(3), 325–348. doi:10.1007/s10734-008-9125-5 Firsov, V. (2004). Podvodnye kamni EGE [Pitfalls of USE]. Voprosy obrazovaniia, 2, 187–188. Fomina, N. (2009). Sistemnaia otsenka kachestva pedagogicheskogo truda [A systematic evaluation of quality of pedagogical work]. Narodnoe obrazovanie, 4, 192–198. Froumin, I. (2000). Otsenka kachestva obrazovaniia: Mezhdu kontrolem i podderzhkoi [Educational quality evaluation: In between of control and support]. Pervoye sentiabria, 92. Available at http://ps.1september.ru/article.php? ID=200009203 Froumin, I., Pinskaia, M., & Kosaretskii, S. (2012). Sotsial’no-ekonomicheskoe i territorial’noe neravenstvo uchenikov i shkol [Social-economic and geographical inequality of students and schools]. Narodnoye Obrazovanie, 1, 17–24. Gokhberg, L., & Kovaleva, G. (2004). Aktual’nye napravleniia sovershenstvovaniia statistiki obrazovaniia [Relevant directions for development of educational statistics]. Voprosy obrazovaniia, 1, 274–278. Gounko, T., & Smale, W. (2007). Modernization of Russian higher education: Exploring paths of influence. Compare: A Journal of Comparative and International Education, 37(4), 533–548. doi:10.1080/03057920701366358. Government of Russia. (2012). Gosudarstvennaia programma Rossiiskoi Federatsii “Razvitie obrazovaniia” na 2013– 2020 gody [State program of the Russian Federation “Development of education” for 2013–2020]. http://минобрнау ки.рф/документы/3409/файл/2228/13.05.15-Госпрограмма-РазBитие_образоBания_2013–2020.pdf (accessed 22 April 22 2015). Grek, S., & Ozga, J. (2010). Re-inventing public education: The new role of knowledge in education policy making. Public Policy and Administration, 25(3), 271–288. doi:10.1177/0952076709356870 Gusarova, M., & Ovchinnikova, M. (2014). Introducing performance management in the system of public administration in Russia: Approaches and lessons learn from the last decade of implementation. Voprosy Gosudarstvennogo i Munitsipal’nogo Upravleniya 1 [Public Administration Issues]. www.ecsocman.hse.ru/mags/en/vgu/2014–1.html (accessed 22 May 2015). [In Russian]. Harvey, D. (2005). A brief history of neoliberalism. New York: Oxford University Press. Hood, C. (1991). A public management for all seasons? Public Administration, 69(1), 3–19. doi:10.1111/j.14679299.1991.tb00779.x. Iamburg, E. (2008). My dialektiku uchili ne po Gegeliu … O kachestvennoi i kolichestvennoi otsenke pedagogicheskogo truda [On qualitative and quantitative evaluation of pedagogical work]. Narodnoe obrazovanie, 4, 117–120. Il’ina, L. (2009). Novyi podhod k attestatsii pedagogicheskikh kadrov: Uchet dostizhenii shkol’nikov i masterstva uchitelei [A new approach to the attestation of pedagogical staff: Accounting for the achievements of the students and the skill of the teachers]. Narodnoe obrazovanie, 4, 46–52. Ivenskih, A., & Ivenskih, I. (2011). Formirovanie kul’tury otvetstvennosti kak osnovy sozdaniia sistemy kachestva obrazovaniia v shkole [Formation of the culture of responsibility as a basis for creating the system of educational quality in school]. Voprosy obrazovaniia, 4, 53–57. Karpushin, N., Gadzhieva, L., & Petrogradskih, I. (2006). Munitsipal’naia model’ monitoringa kachestva obrazovaniia [Municipal model of the monitoring of educational quality]. Narodnoe obrazovanie, 1, 93–97. Kasprzhak, A., Polivanova, K., Tsukerman, G., Mitrofanov, K., & Sokolova, O. (2004). Rossiskoe shkol’noe obrazovanie: Vzgliad so storony [Russian school education: A view from outside]. Voprosy obrazovania, 1, 190–231. Kauko, J., Simola, H., Varjo, J., & Kalalahti, M. (2012). What could a dynamics perspective contribute to comparative research? In J. Kivirauma, A. Jauhianen, P. Seppänen, & T. Kaunisto (Eds.), Koulutuksen yhteiskunnallinen ymmärrys. Social perspectives on education, 219–233. Research in Educational Sciences 59. Turku: Finnish Educational Research Association. Kipnis, A. B. (2008). Audit cultures: Neoliberal governmentality, socialist legacy, or technologies of governing? American Ethnologist, 35(2), 275–289. doi:10.1111/j.1548-1425.2008.00034.x Kokareva, Z. (2014). Ostorozhno: Monitoringi! [Caution: State monitoring of schools!]. Narodnoe obrazovanie, 8, 30–33. Kosaretskii, S. (2008). Obshchestvennyi zakaz na soderzhanie i kachestvo obrazovaniia [Public demand in the content and quality and education]. Narodnoe obrazovanie, 2, 107–113. 362 GUROVA, PIATTOEVA, AND TAKALA Kovaleva, G. (2004). O rezul’tatakh edinogo ekzamena [On the results of the unified exam]. Narodnoe obrazovanie, 1, 17–29. Kovaleva, G. (2006). Mozhno li ispol’zovat’ rezul’taty EGE dlia monitoringa kachestva obrazovaniia [How we can use the results of USE for monitoring the quality of education]. Narodnoe obrazovanie, 9, 53–57. Kovaleva, G., Krasnokutskaia, L., Krasnianskaia, K., & Loginova, O. (2004). Itogi uchastiya Rossii v mezhdunarodnoi programme po otsenke obrazovatel’nykh dostizheny (PISA) v 2003 g. [Results of Russia’s participation in the international programme of assessment of educational achievements (PISA) in 2003]. Voprosy obrazovaniia, 1, 181–189. Kuklin, V. (2004). Problemy EGE—deistvitel’nye i mnimye [Problems of USE—actual and fictional]. Voprosy obrazovaniia, 2, 199–220. Kukulin, I., Mayofis, M., & Safronov, P. (2015). Ostrova utopii: Pedagogicheskoe i sotsial’noe proektirovanie poslevoennoi shkoly (1940–1980-e) [The islands of utopia: Pedagogocal and social designing of the postwar school (1940–1980s)]. Moskva: Novoye Literaturnoye Obozreniye. Kurlov, V. (2003). Sotsiologicheskaia informatsiia v upravlenii sistemoi shkol’nogo obrazovaniia [Sociological information in managing the system of school education]. Sankt-peterburgskii gosudarstvennyi universitet pedagogicheskogo masterstva. Kuznetsov, A. (2001). Monitoring kachestva podgotovki uchashchikhsia [Monitoring of the quality of students preparation]. Standarty i monitoring v obrazovanii, 3, 19–21. Lawn, M., & Grek, S. (2012). Europeanizing education: Governing a new policy space. Oxford: Symposium Books. Lebedev, O. (2009a). Otsenka rezul’tatov shkol’nogo obrazovaniia pri perekhode k EGE [Evaluation of school education results after the transition to USE]. Narodnoe obrazovanie, 4, 18–27. Lebedev, O. (2009b). Rezul’taty shkol’nogo obrazovaniia v 2020 g. [Results of school education in 2020]. Voprosy obrazovaniia, 1, 40–59. Lenskaia, E. (2008). Kachestvo obrazovaniia i kachestvo podgotovki uchitelia [Quality of education and quality of teachers’ preparation]. Voprosy obrazovaniia, 4, 81–95. Lenskaya, E. (2013). Russia’s own common core. Phi Delta Kappan, 95(2), 76–77. doi:10.1177/003172171309500219. Lingard, B., Martino, W., & Rezai-Rashti, G. (2013). Testing regimes, accountabilities and education policy: Commensurate global and national developments. Journal of Education Policy, 28(5), 539–556. doi:10.1080/ 02680939.2013.820042. Long, D., & Long, R. (1999). Education of teachers in Russia. Westport: Greenwood Press. Maiorov, A. (1998). Monitoring v obrazovanii [Monitoring in education]. St. Petersburg. Maiorov, A. (2000). Testy shkol’nykh dostizheny: Konstruirovanie, provedenie, ispolzovanie. [Tests of educational achievement: Composing, conducting, using]. St. Petersburg. Miller, P., & Rose, N. (1990). Governing economic life. Economy and Society, 19(1), 1–31. doi:10.1080/ 03085149000000001 Minina, E. (2013). Neoliberalism and education in Russia: Global and local dynamics in post-soviet education reform (Ph.D. dissertation). Oxford: University of Oxford. Minina, E. (2014). Why doesn’t the telephone ring? Reform of educational standards in Russia. InterDisciplines, 5(2), 15–59. Nóvoa, A., & Yariv-Mashal, T. (2003). Comparative research in education: A mode of governance or a historical journey? Comparative Education, 39(4), 423–439. OECD. (1998). Reviews of national policies for education in the Russian federation. Paris: OECD. Ozga, J., Dahler-Larsen, P., Segerholm, C., & Simola, H. (Eds.). (2011). Fabricating quality in education: Data and governance in Europe. New York: Routledge. Pashby, K. (2013). Related and conflated: A theoretical and discursive framing of multiculturalism and global citizenship education in the Canadian context (Ph.D. dissertation). University of Toronto. Piattoeva, N. (2015). Elastic numbers: National examinations data as a technology of government. Journal of Education Policy, 30(3), 316–334. doi:10.1080/02680939.2014.937830. Pinskaia, M., Kosaretskii, S., & Krutii, N. (2012). Uchet kontekstnoi informatsii pri otsenke kachestva raboty shkoly [Accounting for the contextual information in the evaluation of the quality of school work]. Narodnoe obrazovanie, 5, 31–35. Polyzoi, E., & Dneprov, E. (2011). A framework for understanding dramatic change: Educational transformation in post-Soviet Russia. International Perspectives on Education and Society, 14, 155–179. doi:10.1108/s1479-3679 (2010)0000014009. QUALITY OF EDUCATION AND ITS EVALUATION 363 Potashnik, M., & Moiseev, A. (1999a). Upravlenie kachestvom obrazovaniia [Management of the quality of education]. Narodnoe obrazovanie, 7–8, 167–177. Potashnik, M., & Moiseev, A. (1999b). Rezul’taty obrazovaniia [Educational outcomes]. Narodnoe obrazovanie, 9, 137–147. Rose, N., & Miller, P. (1992). Political power beyond the state: Problematics of government. British Journal of Sociology, 43(2), 173–205. doi:10.1111/j.1468-4446.2009.01247.x. Rose, N., O’Malley, P., & Valverde, M. (2006). Governmentality. Annual Review of Law and Social Science, 2, 83–104. Rust, V. D. (1992). An interview with Edward Dneprov: School reform in the Russian Republic. Phi Delta Kappan, 375–377. Selwyn, N., Henderson, M., & Chao, S. H. (2015). Exploring the role of digital data in contemporary schools and schooling-‘200,000 lines in an Excel spreadsheet’. British Educational Research Journal, 41, 767–781. doi:10.1002/berj.3186. Shogenov, A. (2007). Pokazateli i indicatory obrazovaniia. Analiz regional’nykh programm razvitiia obrazovaniia [Indicators in education. Analysis of regional programmes of educational development]. Narodnoe obrazovanie, 7, 47–56. Silova, I. (2010). Rediscovering post-socialism in comparative education. In I. Silova (Ed.), Post-socialism is not dead: (Re)-reading the global in comparative education (pp. 1–24). Bingley: Emerald Group Publishing. Silova, I., & Brehm, W. C. (2013). The shifting boundaries of teacher professionalism: Education privatization(s) in the post-socialist education space. In T. Seddon & J. Levin (Eds.), World yearbook of education 2013, (pp. 55–74). New York: Routledge. Simonov, V. (1995). Pedagogicheskii menedzhment [Pedagogical management]. Rossiskoe pedagogicheskoe agentstvo. Smolin, O. (2004). Ediny gosudarstvennyi ekzamen: Panatsea ili pogibel’? [Unified State Exam: panacea or ruin?]. Narodnoe obrazovanie, 1, 13–16. Tatianchenko, D., & Vorovschikov, S. (1995). Upravlenie kachestvom obrazovaniia: Vkhozhdenie v problemu [Management of the quality of education: Entering the problematics]. Pedagogicheskie innovatsii. Tret’yakov, P. (1997). Upravlenie shkoloi po rezul’tatam. Praktika pedagogicheskogo menedzhmenta. [Managing school by results. The practice of pedagogical management]. Novaia shkola. Tripney, J., Kenny, C., & Gough, D. (2014). Enabling the use of research evidence within educational policymaking in Europe. European Education, 46(1), 55–74. Tseng, V. (2012). The uses of research in policy and practice. Social Policy Report, 26(2), 3–16. Tyumeneva, Y. (2013). Disseminating and using student assessment information in Russia. Washington: World Bank. Valdman, I. (2009). Perspektivnye podhody v upravlenii obrazovaniem i otsenke obrazovatel’nykh rezul’tatov: Uroki mezhdunarodnogo opyta [Promising approaches to the management of education and evaluation of educational results: The lessons from international experience]. Narodnoe obrazovanie, 2, 95–101. Valdman, I. (2012). Podderzhka sil’nykh v usloviiakh zabveniia slabykh: Ne pora li pomeniat’ prioritety? [Supporting outstanding schools while ignoring schools that underachieve: Isn’t it time to change priorities?]. Problemy sovremennogo obrazovaniia, 4, 88–98. Valverde, G. A. (2014). Educational quality: Global politics, comparative inquiry, and opportunities to learn. Comparative Education Review, 58(4), 575–589. West, R., & Crighton, J. (1999). Examination reform in Central and Eastern Europe: Issues and trends. Assessment in Education, 6(2), 271–289. doi:10.1080/09695949992919. World Bank. (1995). Russia: Education in the transition. Washington, DC: World Bank Europe and Central Asia Country Department. Zamost’yanov, A. (2012). Stareishii zhurnal Rossii [Russia’s oldest journal]. Narodnoe obrazovanie. http://narodnoe. org/old/hist/For_210_years.html (accessed 9 September 2015). Zhuravlev, D. (2004). Edinyi gosudarstvennyi ekzamen: Uravnivanie vozmozhnostei vypusknikov ili uravnilovka shkol? [Unified State Exam: Equalizing chances of school-leavers or leveling of schools?]. Narodnoe obrazovanie, 1, 124–127. 364 GUROVA, PIATTOEVA, AND TAKALA