Monitoring SDG 4.7: Assessing Education for Sustainable Development in policies, curricula, training of educators and student assessment (input‐indicator)

Holst, Jorrit; Singer‐Brodowski, Mandy; Brock, Antje; de Haan, Gerhard Article — Published Version Monitoring SDG 4.7: Assessing Education for Sustainable Development in policies, curricula, training of educators and student assessment (input‐indicator) Sustainable Development Provided in Cooperation with: John Wiley & Sons Suggested Citation: Holst, Jorrit; Singer‐Brodowski, Mandy; Brock, Antje; de Haan, Gerhard (2024) : Monitoring SDG 4.7: Assessing Education for Sustainable Development in policies, curricula, training of educators and student assessment (input‐indicator), Sustainable Development, ISSN 1099-1719, John Wiley & Sons, Inc., Chichester, UK, Vol. 32, Iss. 4, pp. 3908-3923, https://doi.org/10.1002/sd.2865 This Version is available at: https://hdl.handle.net/10419/306215 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ RESEARCH ARTICLE Monitoring SDG 4.7: Assessing Education for Sustainable Development in policies, curricula, training of educators and student assessment (input-indicator) Jorrit Holst | Mandy Singer-Brodowski | Antje Brock | Gerhard de Haan Institut Futur, Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany Correspondence Jorrit Holst, Institut Futur, Department of Education and Psychology, Freie Universität Berlin, Fabeckstr. 37, Berlin, 14195, German. Email: [email protected] Funding information Bundesministerium für Bildung und Forschung, Grant/Award Numbers: 01JO2004, 01JO2305 Abstract Education is viewed as a critical keystone in achieving the Sustainable Development Goals (SDGs). Specifically, Education for Sustainable Development (ESD) is meant to enable everyone to contribute to sustainable development (SDG 4.7). This target is monitored using the global indicator 4.7.1 –mainstreaming of ESD in policies, curricula, training of educators and student assessment. Here, we offer a conceptual and methodological framework for assessments of SDG 4.7.1 (input-level) that addresses both quality and depth of implementation and speed of change. The approach combines document analysis with external expert evaluation and is applied to 10-year data (>11,000 documents) from all formal areas of education in Germany (early childhood education, school education, vocational education and training, higher education). Currently, ESD is mostly implemented in Germany as an “add-on”to the educational system, with all sub-indicators ranging from “isolated mentioning”of ESD and related concepts to “partial integration”. Across most areas of education, the sub-indicator training of educators was evaluated as most deficient. With regard to the speed of change, it was found that the implementation of ESD is dynamic, with all sub-indicators having been evaluated as increasing. The proposed framework can increase the validity, reliability, and comparability of both country reporting and scientific assessments of SDG 4.7.1. We argue for independent and integrative monitoring across input, process, output and outcome to complement self-reporting and to support evidence-informed policymaking on sustainability in education. KEYWORDS Sustainable Development Goals (SDGs), SDG 4.7, Education for Sustainable Development (ESD), monitoring & evaluation, document analysis, input-indicator, early childhood education & school education, vocational education & higher education 1|TOWARD SDG 4.7 –RESPONDING TO AN EXISTENTIAL CHALLENGE Sustainable Development (SD) encompasses an ambitious global agenda for the development of resilient, socially just human life within the limits of planet Earth. With 17 globally agreed upon goals, 169 targets and well over 200 indicators, the Sustainable Development Goals (SDGs) are currently the core normative framework for global sustainability efforts (e.g., Biermann et al., 2017; Biermann et al., 2022). Although the SDGs are not without controversy due to their Received: 9 June 2023 Revised: 27 November 2023 Accepted: 11 December 2023 DOI: 10.1002/sd.2865 This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2024 The Authors. Sustainable Development published by ERP Environment and John Wiley & Sons Ltd. 3908 Sustainable Development. 2024;32:3908–3923. wileyonlinelibrary.com/journal/sd inherently conflicting goals (Hickel, 2019; Holden et al., 2017; Spangenberg, 2017), the general necessity and urgency to move toward global sustainability is virtually undisputed in academia and international politics (e.g., Richardson et al., 2023; Rockström et al., 2023; United Nations General Assembly, 2023). As important as the SDGs are, the findings of the mid-term report on their implementation are very sobering (Sachs et al., 2023): Currently, none of the 17 goals are on track, and in some cases progress is reported to be going backwards. Nonetheless, the authors remind us that all of the goals are still achievable (ibid.). For this transition, the mid-term report highlights universal quality education as a critical SD pathway, among others. This is supported by studies on assessments of the SDGs and their interactions, in which quality education (SDG 4) and particularly SDG 4.7 are described as having a strong positive relationship with various other SDGs (Dalampira & Nastis, 2020; Fonseca et al., 2020; Pham-Truffert et al., 2020; Vladimirova & Le Blanc, 2016; Xiao et al., 2023). In this vein, linking education with sustainability has been discussed as fundamental for change toward sustainability both in the fields of education (Agbedahin, 2019; Sterling, 2003,2016; Wals & Benavot, 2017) and sustainability research (Abson et al., 2017; Otto et al., 2020; Sachs et al., 2019; Van Poeck et al., 2020). Correspondingly, target 4.7 of the SDGs calls for education systems worldwide to “ensure [by 2030] that all learners acquire the knowledge and skills needed to promote sustainable development” (United Nations General Assembly, 2015, p. 21). Against this backdrop, Education for Sustainable Development (ESD) is approached by UNESCO and its member states as “an enabler for all 17 SDGs”and a “foundation for the required transformation”(UNESCO, 2021). Consequently, the final declaration at the UNESCO World Conference on ESD in 2021 set a commitment for member states to “[e]nsure that ESD is a foundational element of our education systems at all levels” (UNESCO, 2021, p. 2). To be able to meet this commitment as well as the objective set in SDG 4.7, it is of critical importance to systematically monitor and evaluate the degree to which ESD and sustainability are being integrated within education systems (Brent Edwards et al., 2020; Kioupi & Voulvoulis, 2019; Stepanek Lockhart, 2018). Such data provide the basis for observing trends, identifying progress and gaps as well as deriving necessary policy measures to strengthen the implementation of ESD. As with all SDGs, the development of concrete and facilitative indicators for the integration of sustainability in education is crucial for its governance and practical implementation (Biermann et al., 2017; Hák et al., 2016; Kim, 2023). Currently, the global indicator for target 4.7 of the SDGs is described as the “extent to which (i) global citizenship education and (ii) education for sustainable development are mainstreamed in (a) national education policies; (b) curricula; (c) teacher education; and (d) student assessment” (UNESCO, 2017,2019a,2019b). However, this indicator has not yet been sufficiently operationalized for independent and scientific assessments (Brent Edwards et al., 2020; Giangrande et al., 2019). Monitoring is currently based on countries' self-reporting using a dichotomous (yes/no) assessment scheme as well as qualitative highlighting of best-practice cases (UNESCO, 2019a,2019b). This reliance on countries' self-reporting results in a bias toward positive reporting (e.g., Nazir et al., 2011), which most likely does not reflect the real status of target 4.7. Taking the example of Germany, the current score on the global indicator for SDG 4.7.1 is reported at 1.0 for national education policies (range: 0–1), 0.904 for curricula, 0.95 for teacher education and 0.917 for student assessment (Destatis, 2023). However, the results derived from independent data from the national monitoring of ESD in Germany (e.g., Grund & Brock, 2020; Holst et al., 2020) are significantly different. While there is considerable support for the integration of ESD into the German education system, including through a large participatory multi-stakeholder process (Nationale Plattform BNE c/o BMBF, 2017; Singer-Brodowski et al., 2020), the discrepancies between self-reporting scores and evaluations by independent monitoring provide good reasons for complementing countries' self-reporting on SDG 4.7 with independent evaluations to allow for evidence-informed policy-making (Lingard, 2013). Also, with regard to operationalization, the conceptual openness of the current global indicator 4.7.1. is viewed by some authors as problematic because, among other reasons, “it is not clear how comparisons of this indicator across countries should be interpreted” (Brent Edwards et al., 2020, p. 35) and because the term “mainstreaming”is not sufficiently concrete with regard to depth of integration (Gallwey, 2016). However, a more nuanced approach to measuring SDG 4.7.1 at the input-level of documents has not yet been proposed. Aware of the necessity of monitoring ESD across the different indicator levels (inputs, e.g., in documents; processes, e.g., assessments of Whole Institution Approaches; outputs and outcomes, e.g., assessments of competencies) and the need to develop both international and more context-specific indicators, we focus here on measurement strategies for the assessment of ESD at the input-level of documents. While we are proposing a way to further operationalize SDG 4.7.1 and apply in one illustrative context (Germany), the approach can also be used as a general input-indicator for ESD in other contexts. Various authors have already conducted document analyses on ESD (e.g., Beveridge et al., 2019; Fredriksson et al., 2020; De Haan, 2021; Holst et al., 2020; Krah et al., 2021). However, evaluating the results of these studies has often proved difficult and, because of the lack of clear criteria for “sufficient”or “satisfactory”ESD implementation, it has not been possible to compare evaluations across different contexts and over time. This also accounts for past document analyses as part of the monitoring of ESD in Germany, where the focus has primarily been on whether the goals set by stakeholders themselves were met and how the observed status compared to other contexts described in the international literature (Holst et al., 2020). We thus see a considerable need for a conceptual and methodological framework for evaluating the current status quo and progress toward implementing SDG 4.7. Such a framework must allow for more standardized assessments that provide greater reliability and validity, and thus better comparability across contexts and over time. In this article, we (1) propose an input-level framework for indicatorising and systematically assessing ESD, and specifically SDG 4.7, HOLST ET AL.3909 based on lexical document analysis and external expert evaluation. We (2) report on the results of an assessment of the operationalized indicator 4.7.1 based on a longitudinal analysis of >11,000 documents from early-childhood education, school education, vocational education and training, and higher education in Germany. By combining systematic document analysis with an external expert evaluation, we propose an operationalization that can be used both for independent scientific assessments and as part of countries reporting on SDG 4.7. 2|MONITORING SDG 4.7.1 AT THE INPUT-LEVEL: BASELINE DOCUMENT ANALYSES AND CONCEPTUAL FRAMEWORK FOR OPERATIONALIZATION At the international level, the UNECE expert group on indicators for ESD contributed to the design of ESD indicators during the United Nations Decade on ESD (UNECE, 2005). This provided a basis for countries to report on their progress (UNECE, 2009,2022) and was intended to cover a broad range of educational areas and levels (e.g., input, output/outcome). While international reporting to date has mostly been based on countries' self-assessments, independent national monitoring of ESD has been carried out at Freie Universität Berlin in Germany since 2015. This monitoring includes conducting repeated document analyses to track the status and progress of ESD implementation at the input level of educational governance. Systematic assessments of documents are considered a fundamental part of educational monitoring (Ioannidou, 2010) as documents (e.g., policies, curricula) provide important leverage for integrating emerging concepts and objectives within the structures of education systems. In Germany, data have now been collected and analyzed over a 10-year period (from 2012 to 2021) in all areas of formal education (early childhood education, school education, vocational education and training, higher education). This longitudinal document analysis serves as an exemplary data base for the operationalization of indicator 4.7.1 in this article. In the following sections, we first present a conceptual framework for assessing different degrees of implementation of sustainability and ESD in education system documents, in line with SDG 4.7. We then outline our methodological approach to document analysis and, building on the proposed conceptual framework for indicatorising SDG 4.7.1 at the input-level, and describe the process of expert evaluation. In the results, we first synthesize the core findings from the document analysis on the status of ESD and sustainability in Germany and then present the results of the external expert evaluation on SDG 4.7.1. In the discussion, we reflect upon the core implications of the findings, strengths and limitations of the approach, and ways to further operationalize and assess SDG 4.7. Redesign? Integrate? Add-On? What does it mean to “mainstream ESD”? As the fundamental basis for the development of indicators for SDG 4.7, it is necessary to consider the different degrees to which ESD and sustainability may or may not be “implemented”or “mainstreamed”within education systems. From an implementation theory perspective, the mainstreaming of the social innovation of ESD (Bormann & Nikel, 2017) can be described as the scaling of an educational reform. Coburn (2003) identifies four dimensions of the scaling of such innovations in education: depth, sustainability (in the sense of durability), spread and shift in ownership. A document analysis, by its very nature, can hardly capture a shift in ownership (the fourth dimension of Coburn's approach). However, the proposed framework traces the scaling of ESD in documents in terms of (1) depth, quality and spread (Coburn's first and third dimensions) and (2) stability and speed of scaling (Coburn's second dimension) by analyzing data at different times. With regard to depth and quality, Sterling (2003, 282ff.) describes four different modes of implementation: “denial”(rejection, leading to no change), “bolt-on”(“accommodation”, resulting in surface level reforms where sustainability is added to an existing system without changing the underlying paradigms), “build-in”(“serious greening”, resulting in significant system changes) and “redesign” (whole system change, a “deep reordering of assumptions equivalent to epistemic change”). Since the early 2000s, various authors have referred to this heuristic for evaluating the depth and quality of sustainability implementation (e.g., Kolmos et al., 2016 on engineering education; Wals & Benavot, 2017 generally on education; Weiss & Barth, 2021 on curriculum change in higher education). For the conceptual framework proposed here, we adapt, refine, and further operationalize the heuristic for evaluating SDG 4.7.1. Regarding the second aspect of mainstreaming ESD, speed of change, it is important to consider the usual cycles in which different types of documents are revised. As both aspects are important for the evaluation of the status and progress of ESD implementation, the proposed framework for assessments of SDG 4.7.1 contains two modules for the two separate yet interrelated domains: (1) quality and depth and (2) speed of change. The first (depth) is considered as the foundational module, which can be utilized in one-off or continuous (longitudinal) assessments. The latter (speed of change) requires time series, either –where possible –through the inclusion of past data (e.g., old and new versions of curricula, laws, exams) or, through comparative surveys (ideally, as part of long-term monitoring). Both parts of the framework are introduced in the following. 2.1 |Quality and depth of implementation With regard to depth of implementation, we adapt and operationalize Sterling (2003) heuristic for the evaluation of document analysis within six demarcated assessment categories: (1) No Mentioning, (2) Isolated Mentioning, (3) Add-On, (4) Partial Integration, (5) Substantial Integration and (6) Redesign (for an overview, see Figure 1). In line with Sterling's approach, we suggest focusing primarily on the quality of observed implementation patterns (e.g., Redesign, Integration, AddOn, Isolated Mentioning) rather than the sheer quantity of textual references to concepts such as ESD or sustainability. At the same time, 3910 HOLST ET AL. quantification is to some extent inherent in the categories presented below, considering that “Redesign”and “Substantial Integration”,by definition, require more textual references, while the categories of “No Mentioning”and “Isolated Mentioning”always imply low or nonexistent quantities. Redesign is described by Sterling (2003, p. 284) as transformation in the sense of “a deep, conscious reordering of assumptions equivalent to epistemic change, leading to change of paradigm”. Education is thereby re-thought “through a continuous learning process –to embody and reflect a whole system approach”to sustainability (ibid.:285). This paradigm shift (sustainability as a central objective of quality education) is manifested at all levels of education systems (e.g., policy, organizations and learning networks, specific learning environments and situations). In line with theories on leverage points for system changes, redesign addresses the “mindset or paradigm out of which the goals, rules, feedback structure arise”as well as the “goals of the system”itself (Abson et al., 2017; Meadows, 1997). As the deepest form of implementation, which does not by definition require a fundamental shift in the underlying educational paradigms, Substantial Integration of ESD and sustainability in education system documents is operationalized to include the goal of ensuring “that all learners acquire the knowledge and skills needed to promote sustainable development”(UNESCO, 2020; United Nations General Assembly, 2015, p. 21) as a priority and guiding objective of all learning processes in a given sector of education (UNESCO, 2021). With reference to the sub-areas of the indicator for SDG 4.7, we propose operationalizing this as the following type of integration in the different areas of education: 1. Policy: Ambitious and binding education policy decisions and recommendations of the relevant actors in the respective education sector for the implementation of ESD and sustainability at all levels of the education sector. Key references for this are the roadmap of the UNESCO “ESD for 2030”program and the Berlin Declaration on the implementation of ESD (UNESCO, 2020,2021). 2. Curricula: Cross-cutting integration of discipline-, subjector occupation-specific goals (e.g., relevant competences), content (e.g., aspects of the SDGs, the socio-ecological transformation), didactic principles (e.g., actionand life world orientation, participation), and, if appropriate for the respective type of document, forms of organizational implementation (e.g., inclusion of the socio-physical learning environment as well as regional partners in the sense of a whole institution approach; see Buckler & Creech, 2014; Holst, 2023). 3. Training of educators: Cross-cutting integration of ESDand sustainability-specific competence goals in the relevant documents for the training and further education of educators (e.g., teachers, trainers, early childhood educators), that is, across all subjects, disciplines and age groups. 4. Student performance assessments: Design of examination questions in all subjects, disciplines, and professions in a form that takes into account other objectives of performance assessment and provides an incentive to promote and measure sustainability-related competences, including the critical and systemic examination of current non-sustainable development and the development of future-oriented possibilities for action. By definition, such Substantial Integration implies both high quality and high quantity textual references. In contrast, Partial Integration requires a similarly high level of quality, but a lower level of quantity. This means that several, but not most, documents, groups or contexts apply an ambitious and binding conceptual understanding of ESD. Taking an example from Germany, partial integration might mean that there are high quality passages in texts on sustainability and ESD in the laws or curricula of several, but not most federal states. The quality of implementation of Add-On differs from that of partial and substantial integration in that it refers to a pattern where the integration of textual references to SD, ESD and related concepts is predominantly of low quality and where such references are often simply added to objectives and explanations that are otherwise hardly changed at all. Sterling (2003, p. 282) describes this as “a ‘bolt-on’of sustainability ideas to [an] existing system, which itself remains largely unchanged”. In the case of curricula, for example, this could mean adding the label “ESD”to courses without making substantial changes to their content, adding new electives on topics related to SD without making changes to the main body of compulsory courses, or modifying parts of courses or subjects so that they briefly touch on sustainability. We suggest that isolated textual references to ESD/sustainability learning-related concepts (low quantity) that are not further contextualized and/or picked up in the rest of the document (low quality) should be considered as Isolated Mentioning. The lowest proposed category (No Mentioning) implies that no anchoring can be identified at all. FIGURE 1 Conceptual framework for the depth of implementation of sustainability and Education for Sustainable Development (ESD) in education system documents; refined operationalization based on the underlying heuristic by Sterling (2003). HOLST ET AL.3911 2.2 |Speed of change When assessing documents, speed of change always relates to the usual publication dates and revision cycles (e.g., school curricula are revised less frequently than, for example, examination questions). Moreover, speed of change can only refer to documents in which references to ESD/sustainability can be expected (e.g., no reference to sustainability or ESD could be expected in resolutions by ministries in response to the Covid-19 pandemic). Based on these considerations, we suggest operationalizing speed of change in five categories (Strong Increase, Medium Increase, Small Increase, No Change, Decrease; detailed operationalization: see methods). These can be evaluated either by external experts who have comprehensive knowledge of the fields of education (as in this study), or as part of countries' selfreporting (see discussion). Lastly, an assessment of speed of change requires data for comparisons across time, which may not always be available. In the following section, we provide an introduction to the document analysis that serves as the data basis for external expert evaluation, and operationalize the conceptual framework within a code system for evaluations. 3|METHODS FOR MEASURING SDG 4.7.1: DOCUMENT ANALYSIS AND EXPERT EVALUATION 3.1 |Systematic document analysis (quantitative, qualitative) Since 2017, the national monitoring on ESD in Germany has conducted repeated large-scale document analyses, which serve as the data basis for reviewing the status and progress on SDG 4.7.1 at the input level. This current analysis includes a total of 11,061 documents (2017: 2795 (Brock et al., 2018; Singer-Brodowski, et al., 2019), 2019: 45078 (1713 new) (Holst et al., 2020), 2021/22: 11,061 (6553 new)) from all subdomains of the global indicator for 4.7: education policy, curricula, training of educators and student assessment. Table 1provides an overview of the different types of documents included from each of the four areas of formal education (Early Childhood Education, School Education, Vocational Education and Training, Higher Education) for the four sub-indicators (Education Policy, Curricula, Training of Educators, Student Assessment). The choice of document groups is based on (i) international indicators that were operationalized into groups of documents for assessment (Brock et al., 2018; Singer-Brodowski et al., 2019; UNECE, 2005), (ii) groups of documents specifically addressed in the National Action Plan on ESD (National Platform ESD c/o BMBF, 2017) and (iii) a systematic alignment of the document base with the subdomains of SDG 4.7.1 (education policy, curricula, training of educators, student assessment). A transdisciplinary knowledge exchange with members of the German ESD expert fora took place throughout all process steps. This iterative approach, which began in 2016, gradually led to the inclusion of further groups of documents (e.g., exams in school education and didactic training of educators in higher education were assessed for the first time for this study). Applying the same data collection procedure used for the 2017, 2018, and 2019 benchmark data, the new and updated data for all document groups included in this study were collected between November 2021 and January 2022. A total of 6553 additional documents were downloaded, inventoried and loaded into the data analysis software MAXQDA for this study, bringing the total dataset to 11,061 documents. All data were automatically searched for conceptual keywords relating to (i) ESD, (ii) Sustainability/Sustainable Development, (iii) Perspectives on and from ESD and (iv) Related educational concepts. An overview of all keywords used in the lexical analysis is provided in Figure 2. Keywords on a conceptual level were chosen to address the thematic as well as the didactic level of ESD. All codings of identified text segments were checked manually, for example, to avoid miscoding, such as the use of “nachhaltig”(German for sustainable) only in the sense of “langfristig”(durable/long-term) instead of in the sense of the normative concept of sustainability/SD. As a coding rule, all segments where the coding did not seem entirely clear were strictly coded as “uncertain”and discussed in a peer debriefing (Spall, 1998) with three researchers to reach consensus or, where this was not possible, to vote on the coding. The coded text segments within each group of documents were analyzed both quantitatively and qualitatively. 3.2 |External Expert Evaluation After reports had been finalized in German for each area of education, six external experts on ESD and educational measurement in the German education system were asked to evaluate the descriptive results according to the predefined system of categories. The experts were chosen based on their context of expertise, representing all areas of education, and their professional experience with indicator development as professors in the fields of general education and ESD. Table 2 provides an overview of the experts involved in the external evaluation with their institutional affiliation and context of expertise. The number and selection of experts took into account both the need for reliability and validity of evaluations, and the feasibility of applying the methodological framework in different contexts. Given the extensive and longstanding field knowledge and experience of the experts, in one specific area of education and more generally across all areas of education, it can be assumed that they were able to make robust assessments of the quality and speed categories based on the predefined template (below). A template was developed (supplementary material) to standardize the external expert evaluation, including background information on the document analysis and the scales for evaluation (quality/depth, speed of change). As the descriptive basis for their judgments, the experts received a descriptive raw version of four reports on the most recent document analysis (in German: Brock & Holst, 2022; Holst, 2022; Holst & Singer-Brodowski, 2022; Singer-Brodowski & 3912 HOLST ET AL. TABLE 1 Document groups included in the systematic lexical document analysis sorted by areas of education and sub-indicators of SDG 4.7.1 (education policy, curricula, training of educators, student assessment). Education policy Curricula Training of educators Student assessment Early Childhood Education (ECE) •Laws of all states and federal laws •Educational plans from all states •School curricula of courses in ECE from all federal states •Education reports •Course handbooks and study/examination regulations for all study programs at higher education institutions (HEI) for educators in ECE •Documents from the conferences of Ministers for Youth and Family Affairs (JFMK) / Education (KMK) and the Child and Youth Welfare Association (AGJ) School Education (SE) •Laws of all states •School curricula of all 16 states for 9 selected subjects •Course handbooks and study/examination regulations for teacher training at the 20 higher education institutions with most graduates in Germany •Exams for degrees in secondary education (Abitur, Realschule, Hauptschule) from 2005 until 2021 •Documents from the Standing Conference of the Ministers of Education and Cultural Affairs (KMK) with focus on SE Vocational Education and Training (VET) •Federal and state laws •All new or modified training regulations since 2015 •Course handbooks and study/ examination regulations for teacher training in five selected states •Regulations on exams in training regulations since 2015 •Federal Institute for VET (BIBB), KMK-Committee on VET •All new or modified frame-curricula for VETschools since 2015 •Ordinance on Trainer Aptitude (AEVO) and regulations for further training of educators •Education reports and data reports (BIBB) •Guides for Educational Practice (BIBB) since 2015 Higher Education (HE) •Federal and state laws •Course handbooks and course regulations of 3 subjects (biology, mechanical engineering, business administration) from 20 HEIs •Didactic training of educators at 20 HEIs and respective networks for didactic training in HE •Federal, regional rectors' conferences (HRK/LRK) •Target agreements of all states with HEI •Education reports •Student associations •HEI self-governance (e.g., mission statements, reports, strategies, statutes) from 20 HEIs HOLST ET AL.3913 Holst, 2022). Abstracts, evaluations and recommendations were removed to better ensure an unbiased and independent evaluation by the experts. The descriptive reports were divided into five PDFs (introduction and methods, one PDF for each sub-indicator of SDG 4.7.1) and provided to the experts with the template. The following Tables 3and 4are part of the template and were provided to the experts as the predefined category system for assessment. First, Table 3introduces the category system used to evaluate the quality and depth of ESD implementation. Table 4subsequently introduces the category system used to evaluate speed of change. Importantly, the rating of both categories (depth/quality, speed) refers to the predominant pattern of anchoring, which means that individual references are less important in the evaluation than the larger patterns. During the evaluation, all document groups were evaluated individually and an aggregated assessment was made for each subindicator. After the six expert evaluations had been collected, the mean and modus were calculated for each document group and sub-indicator. 4|RESULTS Section 4.1 provides a descriptive overview of the main results of the document analysis for each area of education. These serve as the data for the evaluation of SDG 4.7.1. The results of the expert evaluation are reported in section 4.2. 4.1 |Implementation of ESD and related concepts in Germany: Descriptive overview Before presenting the results for each area of education, it is important to note that, under the German constitution, education in Germany is the responsibility of the 16 federal states. This means that the federal government only has direct influence on some parts of the education system (e.g., parts of vocational education) and that most data are reported for the 16 states. 4.1.1 | Early childhood education (ECE) Early childhood education in Germany is primarily organized by independent institutions. Municipalities and states support daycare places, and in recent years the federal government has provided funding for overall quality development. At the level of education policy (state laws), there is a slight trend toward the inclusion of ESD compared to the previous study, with 4 state laws (2019: 1) referring to sustainability or ESD. The supporting associations, which are united in the Child and Youth Welfare Association (AGJ), have also taken a position on ESD. 1 Position papers of the Standing Conference of the Ministers of Education and Cultural Affairs (KMK) on the training of early childhood education specialists and support staff have provided a basis for FIGURE 2 Conceptual keywords used for lexical search (translated from German, different forms for singular/ plural as well as different suffixes were included); based on Holst et al. (2020). Keywords which were included in this study for the first time are marked with. TABLE 2 Experts involved in the external expert evaluation with institutional affiliation and context of expertise. Expert Institutional affiliation Context of expertise Prof. Dr. Inka Bormann Freie Universität Berlin Professor of General Education Prof. Dr. Johannes Hartig Leibniz Institute for Research and Information in Education Professor of Educational Measurement Prof. Dr. Werner Kuhlmeier University of Hamburg Professor of Vocational Education Prof. Dr. Armin Lude Ludwigsburg University of Education Professor of Biology and Biology Education Prof. Dr. Heike Molitor Eberswalde University for Sustainable Development Professor of Environmental Education and Education for Sustainable Development Prof. Dr. Marco Rieckmann University of Vechta Professor of Higher Education Development 3914 HOLST ET AL. strengthening ESD in training (see below). ESD has not been included in education reporting. Regarding the curricula, there are no binding documents for young children up to the age of 6. However, by the end of 2021, 9 of the 16 educational plans of the federal states explicitly referred to ESD (2017: 6; 2019: 8). A clear trend can therefore be observed, which applies in particular to the recently revised educational plans. Early childhood educators follow one of two training paths: traditional dual educator training (see section on Vocational Education and Training (VET) below) or an early education/childhood study program at higher education institutions. An increase of references to ESD or related concepts was found in documents related to study programs (3% of documents with references in 2017 to 5% in 2021). Specifically, this means that references to ESD could be found on about one in 31 analyzed pages (2017: 1/139 pages), and references to sustainability on about one in 45 pages (2017: 1/256 pages). There is also an increase in the proportion of ESD in relation to other sustainability-related educational concepts. An even clearer trend is evident in VET for early childhood educators: starting from comparatively few references in the baseline analysis (2017: 1 reference every 280 pages, n=292), ESD is more frequently integrated in more recent documents (2019: 1 reference every 43 pages, n=44; 2021: 1 reference every 24 pages, n=72). No ECE documents exist that could be analyzed in the area of student assessments. 4.1.2 | School education (SE) In Germany, SE laws, curricula and examination questions differ from state to state. The Standing Conference of the Ministers of Education and Cultural Affairs (KMK) therefore wields considerable influence as the joint consulting body of the federal states. A review of the TABLE 3 Category system for quality and depth of ESD implementation with descriptions and further elaborations for the categories “Redesign”,“Substantial Integration”,“Partial Integration”,“Add-On”,“Isolated Mentioning”and “No Mentioning”. Category Description Further elaboration Redesign Comprehensive redesign or reorganization of the fundamental orientation of documents/document groups toward sustainability/ESD (paradigm/lived culture of sustainability). Understanding of ESD as a central, continuous and binding part of the educational mission in the sense of a whole institution and ultimately a whole-system approach, which in turn corresponds to a fundamentally changed educational paradigm/a lived culture of sustainability (cf. also Sterling, 2003). The orientation toward sustainability restructures the goals and paths of the educational sector / institution in question. Substantial Integration Deep and cross-sectional integration of sustainability/ESD in the various documents/document groups; high quantity and quality (substantial integration) of references on ESD and sustainability. An ambitious and binding conceptual understanding of ESD is consistently applied to the content of the respective documents (e.g., curriculum or resolution). Partial Integration Predominantly high content quality of the references to ESD/sustainability in the sense of substantial integration in several documents/document groups (e.g., in several, but not all federal states) with differences between or within the documents/document groups. An ambitious and binding conceptual understanding of ESD is applied in several documents of a document group or in several federal states, subjects, or at several locations with high content quality in the sense of substantial integration (see above). Add-on References to ESD/sustainability are predominantly of medium to low content quality, often as a supplement to otherwise frequently unchanged requirements/objectives/explanations. For example, naming the concept of ESD in curricula, but without describing goals, content, methods/media, and organizational implementation as substantially oriented toward ESD/sustainability. Isolated mentioning Isolated references in individual documents (groups of documents) that are not further contextualized and/or taken up in the rest of the document. Isolated references to for example, sustainability, ESD or related concepts. No mentioning No mentioning of sustainability, ESD or related concepts. TABLE 4 Category system for speed of change of ESDimplementation with descriptions for the categories “Strong Increase”,“Medium Increase”,“Small Increase”,“No Increase”and “Decrease”. Category Description Strong increase Avery strong/rapid increase in references to ESD/sustainability in the respective documents (groups) compared to usual publication/revision times. Medium increase Asignificant increase in references to ESD/sustainability in the respective documents (groups) compared to usual publication/revision times. Small increase Aslow increase in references to ESD/sustainability in the respective documents (groups) compared to usual publication/revision times, recognizable by sporadically increasing references. No change No change is discernible over time. Decrease A reduction of references to sustainability/ESD can be observed in the periods under consideration. HOLST ET AL.3915 Kim, R. E. (2023). Augment the SDG indicator framework. Environmental Science & Policy,142,62–67. https://doi.org/10.1016/j.envsci.2023. 02.004 Kioupi, V., & Voulvoulis, N. (2019). Education for sustainable development: A systemic framework for connecting the SDGs to educational outcomes. Sustainability,11(21), 6104. https://doi.org/10.3390/ su11216104 Klafki, W. (1996). Neue Studien zur Bildungstheorie und Didaktik: Zeitgemäße Allgemeinbildung und kritisch-konstruktive Didaktik. Beltz Verlag. Kolmos, A., Hadgraft, R. G., & Holgaard, J. E. (2016). Response strategies for curriculum change in engineering. International Journal of Technology and Design Education,26(3), 391–411. https://doi.org/10.1007/ s10798-015-9319-y Krah, J. M., Reimann, J., & Molitor, H. (2021). Sustainability in Brandenburg study programs. Perspectives for anchoring sustainability in higher education curricula. Sustainability,13(7), 3958. https://doi.org/10. 3390/su13073958 Kvamme, O. A. (2021). Rethinking Bildung in the Anthropocene: The case of Wolfgang Klafki. HTS Teologiese Studies/Theological Studies,77(3), 1–9. https://doi.org/10.4102/hts.v77i3.6807 Lingard, B. (2013). The impact of research on education policy in an era of evidence-based policy. Critical Studies in Education,54(2), 113–131. https://doi.org/10.1080/17508487.2013.781515 Marron, R. K., & Naughton, D. (2019). Monitoring Progress Towards SDG Target 4.7 in Europe: Proposed Framework and Tools. McCool, S. F., & Stankey, G. H. (2004). Indicators of sustainability: Challenges and opportunities at the interface of science and policy. Environmental Management,33(3), 294–305. https://doi.org/10.1007/ s00267-003-0084-4 McKenzie, M., Bieler, A., & McNeil, R. (2015). Education policy mobility: Reimagining sustainability in neoliberal times. Environmental Education Research,21(3), 319–337. https://doi.org/10.1080/13504622.2014. 993934 Meadows, D. H. (1997). Places to intervene in a system. Whole Earth, 91(1), 78–84. Nationale Plattform BNE c/o BMBF. (2017). Nationaler Aktionsplan Bildung für nachhaltige Entwicklung: Der deutsche Beitrag zum UNESCO-Weltaktionsprogramm. Retrieved from https://www.bmbf. de/bmbf/shareddocs/downloads/files/nationaler_aktionsplan_bildung_ fuer_nachhaltige_entwicklung.pdf?__blob=publicationFile&v=2 Nazir, J., Pedretti, E., Wallace, J., Montemurro, D., & Inwood, H. (2011). Reflections on the Canadian experience with education for climate change and sustainable development. Canadian Journal of Science, Mathematics and Technology Education,11(4), 365–380. https://doi. org/10.1080/14926156.2011.624673 Otto, I. M., Donges, J. F., Cremades, R., Bhowmik, A., Hewitt, R. J., Lucht, W., Rockström, J., Allerberger, F., McCaffrey, M., Doe, S. S. P., Lenferna, A., Morán, N., van Vuuren, D. P., & Schellnhuber, H. J. (2020). Social tipping dynamics for stabilizing Earth's climate by 2050. Proceedings of the National Academy of Sciences of the United States of America,117(5), 2354–2365. https://doi.org/10.1073/pnas. 1900577117 Pham-Truffert, M., Metz, F., Fischer, M., Rueff, H., & Messerli, P. (2020). Interactions among sustainable development goals: Knowledge for identifying multipliers and virtuous cycles. Sustainable Development, 28(5), 1236–1250. https://doi.org/10.1002/sd.2073 Richardson, K., Steffen, W., Lucht, W., Bendtsen, J., Cornell, S. E., Donges, J. F., Drüke, M., Fetzer, I., Bala, G., von Bloh, W., Feulner, G., Fiedler, S., Gerten, D., Gleeson, T., Hofmann, M., Huiskamp, W., Kummu, M., Mohan, C., Nogués-Bravo, D., et al. (2023). Earth beyond six of nine planetary boundaries. Science Advances,9(37), adh2458. https://doi.org/10.1126/sciadv.adh2458 Rieckmann, M., Mindt, L., & Gardiner, S. (2017). Education for sustainable development goals: Learning objectives. UNESCO. Rockström, J., Gupta, J., Qin, D., Lade, S. J., Abrams, J. F., Andersen, L. S., & Zhang, X. (2023). Safe and just earth system boundaries. Nature, 619(7968), 102–111. https://doi.org/10.1038/s41586-023-06083-8 Sachs, J. D., Lafortune, G., Fuller, G., & Drumm, E. (2023). Implementing the SDG stimulus. In Sustainable development report 2023. Dublin University Press. https://doi.org/10.25546/102924 Sachs, J. D., Schmidt-Traub, G., Mazzucato, M., Messner, D., Nakicenovic, N., & Rockström, J. (2019). Six transformations to achieve the sustainable development goals. Nature Sustainability,2(9), 805–814. https://doi.org/10.1038/s41893-019-0352-9 Singer‐Brodowski, M., Brock, A., Etzkorn, N., & Otte, I. (2019). Monitoring of education for sustainable development in Germany –insights from early childhood education, school and higher education. Environmental Education Research,25(4), 492–507. https://doi.org/10.1080/ 13504622.2018.1440380 Singer‐Brodowski, M., von Seggern, J., Duveneck, A., & Etzkorn, N. (2020). Moving (Reflexively within) structures. The Governance of Education for Sustainable Development in Germany. Sustainability,12(7), 2778. https://doi.org/10.3390/su12072778 Singer‐Brodowski, M., & Holst, J. (2022). Nachhaltigkeit & BNE in der Frühen Bildung: Chancen durch Partizipation und ökologische Kinderrechte, Bedarfe bei der Ausbildung von Fachkräften. Kurzbericht des Nationalen Monitorings zu Bildung für nachhaltige Entwicklung. https://doi.org/10.17169/refubium-35735 Sinnes, A. T., & Eriksen, C. C. (2016). Education for sustainable development and international student assessments: Governing education in times of climate change. Global Policy,7(1), 46–55. https://doi.org/10. 1111/1758-5899.12256 Spall, S. (1998). Peer debriefing in qualitative research: Emerging operational models. Qualitative Inquiry,4(2), 280–292. https://doi.org/10. 1177/107780049800400208 Spangenberg, J. H. (2017). Hot air or comprehensive Progress? A critical assessment of the SDGs. Sustainable Development,25(4), 311–321. https://doi.org/10.1002/sd.1657 Steffen, W., Richardson, K., Rockström, J., Cornell, S. E., Ingo, F., Bennett, E. M., Biggs, R., Carpenter, S. R., de Vries, W., de Wit, C. A., Folke, C., Gerten, D., Heinke, J., Mace, G. M., Persson, L. M., Ramanathan, V., Reyers, B., & Sörlin, S. (2015). Panetary boundaries: Guiding human development on a changing planet. Science,348(6240), 1217-c. https://doi.org/10.1126/science.aaa9629 Stepanek Lockhart, A. (2018). Monitoring ESD: Lessons learned and ways forward. In A. Leicht, J. Heiss, & W. J. Byun (Eds.), Issues and trends in education for sustainable development (pp. 215–231). United Nations Educational, Scientific and Cultural Organization. Sterling, S. (2003). Whole systems thinking as a basis for paradigm change in education: Explorations in the context of sustainability (PhD).UniversityofBath. Sterling, S. (2016). A commentary on education and sustainable development goals. Journal of Education for Sustainable Development,10(2), 208–213. https://doi.org/10.1177/0973408216661886 Tilbury, D. (2007). Monitoring and evaluation during the UN decade of education for sustainable development. Journal of Education for Sustainable Development,1(2), 239–254. https://doi.org/10.1177/ 097340820700100214 UNECE. (2005). Indicators for education for sustainable development. Progress report on the work of the Expert Group Retrieved from https:// unece.org/fileadmin/DAM/env/documents/2005/cep/ac.13/cep.ac.13. 2005.9.e.pdf UNECE. (2009). Learning from each other: The UNECE Strategy for Education for Sustainable Development ECE/CEP/159. Retrieved from https://sdgs.un.org/sites/default/files/publications/798ece5.pdf UNECE. (2022). Implementation of the UNECE strategy for ESD across the ECE region (2015–2018). Retrieved from https://unece.org/sites/ default/files/2022-09/Implementation%20of%20the%20UNECE% 20Strategy_web_final_05.09.2022.pdf 3922 HOLST ET AL. UNESCO. (2017). Measurement Strategy for SDG Target 4.7: Proposal by GAML Task Force 4.7. Retrieved from GAML4/17 website: http://uis. unesco.org/sites/default/files/documents/gaml4-measurement-strategysdg-target4.7.pdf UNESCO. (2019a). Proposal for monitoring of SDG indicators 4.7.1, 12.8.1 and 13.3.1. TCG6/REF/14. Retrieved from United Nations Educational, Scientific and Cultural Organization website: https://tcg.uis. unesco.org/wp-content/uploads/sites/4/2019/08/TCG6-REF-14Proposal-for-monitoring-of-SDG-indicators-4.7.1-12.8.1-and-13.3.1.pdf UNESCO. (2019b). TCG6/REF/4. SDG indicator 4.7.1: Proposal for a measurement strategy. Retrieved from United Nations Educational, Scientific and Cultural Organization website: https://tcg.uis.unesco. org/wp-content/uploads/sites/4/2019/08/TCG6-REF-4-4.7.1-Proposalfor-measurement-strategy.pdf UNESCO. (2020). Education for sustainable development. A roadmap. UNESCO. UNESCO. (2021). Berlin declaration on education for sustainable development. UNESCO. United Nations General Assembly. (2015). 70/1. Transforming Our World: The 2030 Agenda for Sustainable Development. Retrieved from http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1& Lang=E United Nations General Assembly. (2017). Education for sustainable development in the framework of the 2030 Agenda for Sustainable Development: Resolution 72/222. United Nations General Assembly. (2023). Political declaration of the highlevel political forum on sustainable development convened under the auspices of the General Assembly A/HLPF/2023/L.1. Retrieved from https://hlpf.un.org/sites/default/files/2023-09/A%20HLPF%202023% 20L1.pdf?_gl=1*krm5r*_ga*Mjc5NDQ5NDA4LjE2MDMxNDUzNDU.*_ga_ TK9BQL5X7Z*MTY5NjM0NTIzNi4xMjEuMS4xNjk2MzQ3Nzk1LjAu MC4w Unterhalter, E. (2019). The many meanings of quality education: Politics of targets and indicators in SDG 4. Global Policy,10(S1), 39–51. https:// doi.org/10.1111/1758-5899.12591 Van Poeck, K., Östman, L., & Block, T. (2020). Opening up the black box of learning-by-doing in sustainability transitions. Environmental Innovation and Societal Transitions,34, 298–310. https://doi.org/10.1016/j.eist. 2018.12.006 Vladimirova, K., & Le Blanc, D. (2016). Exploring links between education and sustainable development goals through the lens of UN flagship reports. Sustainable Development,24(4), 254–271. https://doi.org/10. 1002/sd.1626 Wals, A. E. J., & Benavot, A. (2017). Can we meet the sustainability challenges? The role of education and lifelong learning. European Journal of Education,52(4), 1–10. https://doi.org/10.1111/ejed.12250 Weiss, M., Barth, M., & von Wehrden, H. (2021). The patterns of curriculum change processes that embed sustainability in higher education institutions. Sustainability Science,16(5), 1579–1593. https://doi.org/ 10.1007/s11625-021-00984-1 Xiao, H., Liu, Y., & Ren, J. (2023). Synergies and trade-offs across sustainable development goals: A novel method incorporating indirect interactions analysis. Sustainable Development,31(2), 1135–1148. https:// doi.org/10.1002/sd.2446 SUPPORTING INFORMATION Additional supporting information can be found online in the Supporting Information section at the end of this article. How to cite this article: Holst, J., Singer-Brodowski, M., Brock, A., & de Haan, G. (2024). Monitoring SDG 4.7: Assessing Education for Sustainable Development in policies, curricula, training of educators and student assessment (input-indicator). Sustainable Development,32(4), 3908–3923. https://doi.org/ 10.1002/sd.2865 HOLST ET AL.3923