Do remote learning tools reduce learning loss during school closure? Experience from Central Asia and the Caucasus during the COVID-19 pandemic

Kodama, Wataru; Morgan, Peter J.; Azhgaliyeva, Dina Working Paper Do remote learning tools reduce learning loss during school closure? Experience from Central Asia and the Caucasus during the COVID-19 pandemic ADBI Working Paper, No. 1450 Provided in Cooperation with: Asian Development Bank Institute (ADBI), Tokyo Suggested Citation: Kodama, Wataru; Morgan, Peter J.; Azhgaliyeva, Dina (2024) : Do remote learning tools reduce learning loss during school closure? Experience from Central Asia and the Caucasus during the COVID-19 pandemic, ADBI Working Paper, No. 1450, Asian Development Bank Institute (ADBI), Tokyo, https://doi.org/10.56506/ROAZ2346 This Version is available at: https://hdl.handle.net/10419/301955 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc-nd/3.0/igo/ ADBI Working Paper Series DO REMOTE LEARNING TOOLS REDUCE LEARNING LOSS DURING SCHOOL CLOSURE? EXPERIENCE FROM CENTRAL ASIA AND THE CAUCASUS DURING THE COVID-19 PANDEMIC Wataru Kodama, Peter Morgan, and Dina Azhgaliyeva No. 1450 May 2024 Asian Development Bank Institute The Working Paper series is a continuation of the formerly named Discussion Paper series; the numbering of the papers continued without interruption or change. ADBI’s working papers reflect initial ideas on a topic and are posted online for discussion. Some working papers may develop into other forms of publication. Suggested citation: Kodama, W., P. Morgan, and D. Azhgaliyeva. 2024. Do Remote Learning Tools Reduce Learning Loss during School Closure? Experience from Central Asia and the Caucasus During the COVID-19 Pandemic. ADBI Working Paper 1450. Tokyo: Asian Development Bank Institute. Available: https://doi.org/10.56506/ROAZ2346 Please contact the authors for information about this paper. Email: [email protected] Wataru Kodama is a research associate, Peter Morgan is senior consulting economist and advisor to the dean, and Dina Azhgaliyeva is a senior research fellow, all at the Asian Development Bank Institute, Tokyo, Japan. The views expressed in this paper are the views of the author and do not necessarily reflect the views or policies of ADBI, ADB, its Board of Directors, or the governments they represent. ADBI does not guarantee the accuracy of the data included in this paper and accepts no responsibility for any consequences of their use. Terminology used may not necessarily be consistent with ADB official terms. Discussion papers are subject to formal revision and correction before they are finalized and considered published. The authors are grateful to Daniel C. Suryadarma and participants at the ADBI Conference on Increasing the Resilience of Education Systems in Asia and the Pacific for their valuable comments. Asian Development Bank Institute Kasumigaseki Building, 8th Floor 3-2-5 Kasumigaseki, Chiyoda-ku Tokyo 100-6008, Japan Tel: +81-3-3593-5500 Fax: +81-3-3593-5571 URL: www.adbi.org E-mail: [email protected] © 2024 Asian Development Bank Institute ADBI Working Paper 1450 W. Kodama et al. Abstract School closures have a negative impact on children’s educational outcomes. During the COVID-19 pandemic, Central Asia and the Caucasus introduced online classes and TV programs with a view to preventing learning loss. Based on a household survey in nine countries (Afghanistan, Azerbaijan, Georgia, Kazakhstan, the Kyrgyz Republic, Mongolia, Pakistan, Tajikistan, and Uzbekistan), this is one of the first regional studies to document learning loss and the efficacy of remote learning tools in Central Asia and the Caucasus. Some 79% of households felt that their children’s learning progress was slower than in in-person schooling, including 39% who perceived either “very little progress” or “no progress.” Our econometric analysis finds that online classes reduced perceived learning loss during school closures, but this does not hold for TV programs. In addition, traditional paper lessons were also effective in reducing perceived learning loss but only for shorter school closure durations. The usage of online classes is strongly correlated with household income and location (urban vs. rural) as well as internet connectivity. Combined usage of TV programs and paper lessons may be an effective alternative to online lessons during longer closure periods. Keywords: remote learning, learning loss, online class, COVID-19 pandemic, Central Asia JEL Classification: I20, I21, I25 ADBI Working Paper 1450 W. Kodama et al. Contents 1. INTRODUCTION .......................................................................................................... 1 2. DATA AND CONTEXT ................................................................................................. 2 2.1 Data .................................................................................................................. 2 2.2 Context ............................................................................................................. 3 3. STYLIZED FACTS ....................................................................................................... 5 4. ECONOMETRIC ANALYSIS ........................................................................................ 7 4.1 Determinants of the Usage of Distance Learning Tools ................................... 7 4.2 Remote Learning Tools and Perceived Learning Loss .................................... 9 4.3 Effectiveness of Multiple Distance Learning Modes ....................................... 11 4.4 Heterogeneity ................................................................................................. 13 5. CONCLUSION AND POLICY RECOMMENDATIONS .............................................. 14 REFERENCES ...................................................................................................................... 15 APPENDIX ............................................................................................................................. 17 ADBI Working Paper 1450 W. Kodama et al. 1 1. INTRODUCTION School closures have a negative impact on children's educational outcomes, which could eventually lead to a reduction in the future productivity and income of both individuals and countries (Hanushek and Woessmann 2020). During the COVID-19 pandemic, large and unequal effects of school closure on learning progress were found (Agostinelli et al. 2022; Engzell, Frey, and Verhagen 2021; Haelermans et al. 2022; Lichand et al. 2022; Moscoviz and Evans 2022). Patrinos (2023) find that one additional week of school closure increases the degree of learning loss by 1% of a standard deviation (SD). Alasino et al. (2024) find that student performance is 0.2–0.3 SD lower after school closures in Mexico, equivalent to 0.66–1.05 years of schooling. During school closures, the transition to online classes/learning was observed worldwide because in-person classes were not available. In addition to online classes, many Asian countries introduced TV and/or radio programs as an additional remote learning tool because internet connections were not necessarily available in some areas (UNESCO, UNICEF, and World Bank 2021). Due to efforts to ensure continuity in learning with remote learning tools in Uzbekistan, one study suggests that there were no learning losses during school closures (Iqbal and Patrinos 2023). However, evidence from other Asian countries suggests there were delays in learning progress during the school closure period (ADB 2022; Maddawin et al. 2024). Experiences of school closures in Central Asia and the Caucasus provide important lessons for building resilient education systems. The region experienced perhaps the largest effect of school closures on students’ learning progress due to relatively longer periods of closure (UNESCO 2021). At the same time, to prevent learning losses, the governments and international partners provided not only online classes but also other remote learning tools such as TV, paper lessons, and radio programs. Assessing the learning losses during the school closure periods and the efficacy of these remote learning tools in reducing such losses may help prevent them during future school closures, which could be due not only to pandemics but also to other shocks, including air pollution and natural disasters. It is important to learn from the experience of the COVID-19 pandemic to provide evidence-based policy recommendations. Thus far, however, the evidence is very limited in Central Asia and the Caucasus, and this is one of the first regional studies to document learning losses and the efficacy of remote learning tools during the pandemic. We collected household survey data from 10 member countries of the Central Asia Regional Economic Cooperation Program (CAREC), excluding the People’s Republic of China (PRC). Since there were no school closures in Turkmenistan, for this study we use data from nine countries, namely Afghanistan, Azerbaijan, Georgia, Kazakhstan, the Kyrgyz Republic, Mongolia, Pakistan, Tajikistan, and Uzbekistan. Our survey data include parents’ perceived progress of their children’s studies (“perceived learning progress or loss”) and children’s actual usage of remote learning tools. The results suggest that 79% of the households felt that their children’s learning progress was slower than in in-person schooling, including 39% who perceived either “very little progress” or “no progress,” despite the governments’ efforts to minimize learning loss with remote learning tools. Our econometric analysis suggests that online classes, whose usage strongly correlates with household income and location (urban vs. rural), were effective in reducing perceived learning loss during school closures, but this does not hold for TV programs. Traditional paper lessons were also commonly used, but they were effective in reducing learning losses only for shorter school closure periods. We also find that a combined usage of TV programs and paper lessons seemed effective under longer closure periods. ADBI Working Paper 1450 W. Kodama et al. 2 This study contributes to the literature in two ways. First, it investigates the effectiveness of remote learning, including tools like online classes, paper lessons, and TV programs. Previous studies suggest that, if used properly, online classes may be as effective as traditional in-person classes in terms of learning outcomes (Swan 2003). Some recent studies suggest that online classes were to some extent effective for learning progress during school closures (Awal 2023; Roman and Plopeanu 2021). However online classes in Central Asia and the Caucasus (particularly in rural areas) were challenged by the lack of access to a good-quality internet connection, as well as access to devices (e.g., computers, tablets, smartphones). Instead, offline tools such as TV programs and paper lessons were provided for children who could not access online classes, which could help to mitigate the negative impact. Our results suggest that, when combined with paper lessons, TV programs may mitigate the negative effects of school closure on learning progress, although the effects are smaller than those of online classes. Second, this is one of the first regional studies to document learning losses in Central Asia and the Caucasus during the pandemic. Previous studies on learning losses mainly focus on high-income countries such as the United States, the Netherlands, and Germany, while some focus on middleand low-income countries like Bangladesh, Brazil, Indonesia, and Kenya (cf., Patrinos, Vegas, and Carter-Rau 2023). Maddawin \et al. (2024) recently reports learning losses in Southeast Asian countries, yet evidence from Central Asia and the Caucasus is thus far limited despite the longer duration of school closures in the region. Although we were not able to implement objective measures of learning losses like test scores, our survey data reflect respondents’ actual responses to governments’ policies on school closures and remote learning. We add unique evidence on parents’ perceived learning progress in Central Asia and the Caucasus to the literature. The remainder of this paper is organized as follows. Section 2 describes the data and context of this study. Section 3 presents descriptive evidence. Section 4 investigates the effectiveness of remote learning tools and presents the results of econometric analysis. Section 5 concludes and provides policy recommendations. 2. DATA AND CONTEXT 2.1 Data Following the COVID-19 pandemic and school closures in 2020 and 2021, we conducted household surveys in nine CAREC member countries (excluding the PRC and Turkmenistan) from September to October 2022 with a view to understanding the impacts of the pandemic on households and children’s educational progress.1 Computer-assisted telephone interviews (CATIs) were carried out on randomly selected representative samples of more than 1,000 households from each country, as face-to-face interviews were not practical under the pandemic conditions. For each country, we confirm that the regional and household income (quantile) distributions are in line with the national statistics. The questionnaire covered socioeconomic status, employment, gender and education of the household head, household income and expenditure, and children’s education (if any) during the time of the pandemic. Azhgaliyeva et al. (2022) and Kodama et al. (forthcoming) for further details. 1 Turkmenistan was excluded from this study because it reported no school closures during the COVID-19 pandemic. ADBI Working Paper 1450 W. Kodama et al. 3 For those respondents who have any school-age children (aged six to 18), we asked questions related to school closures and the perceived learning progress of the school-age child who most recently had a birthday. In addition to basic information on this child such as grade level as of now and before the school closures (December 2019), we asked about the duration of school closures since February 2020, modes of remote learning (class instruction) they used during the closures, the parents’ perceptions of the child’s learning progress during the closures, and whether the child was enrolled in school after reopening. In this study, we restrict the samples to households who had any school-age children and experienced school closures. This leaves a subsample of 3,600 households, equivalent to around 40% of the original dataset. Table 1 shows the distribution of our sample. The sample mainly consists of households with elementary school and middle school children: 36% in Grades 1 to 3, 25% in Grades 4 to 6, and 24% in middle school. Nearly half of the households reside in urban areas and the remaining half live in rural areas. Table 1: Sample Distribution: Grade Level, Area (Rural vs. Urban), and Country Share, % (restricted sample) Share, % (original sample) Grade level (December 2019) Preschool/kindergarten 5.20 Elementary P1-P3 36.25 Elementary P4-P6 25.14 Middle school 23.83 High/vocational school 9.58 Location (rural vs. urban) Rural 53.05 53.06 Urban 46.95 46.94 Country Afghanistan 6.75 11.21 Azerbaijan 10.33 10.86 Georgia 8.25 10.86 Kazakhstan 10.33 11.00 Kyrgyz Republic 13.12 11.44 Mongolia 12.44 10.97 Pakistan 18.07 11.12 Tajikistan 10.95 10.96 Uzbekistan 9.77 11.58 Observations 3,600 10,207 Source: Authors’ estimates from the survey. 2.2 Context During the time of the COVID-19 pandemic, CAREC countries implemented various measures, including lockdown and school closures. Table 2 shows the duration of school closures in CAREC countries based on UNESCO’s database. In Afghanistan and Pakistan, on average, schools were fully closed for more than 35 weeks and partially closed for around 25 weeks. These closures may have led to learning losses of 0.35 to 0.60 SDs (based on the estimates by Patrinos (2023)). Azerbaijan and ADBI Working Paper 1450 W. Kodama et al. 4 Mongolia also had long closure periods with 29 and 24 weeks of full closure and 20 and 34 weeks of partial closure, respectively. In Kazakhstan, schools were partially closed for 43 weeks. In contrast, Tajikistan had no full school closure but had four weeks of partial closure. Table 2: Duration of School Closures and Types of Distance Learning (Feb 2020 – March 2022) Duration (week) Modes of Remote Learning Fully Closed Partially Closed Online Classes TV Programs Radio Programs Take-home Packages (paper) Afghanistan 35 26 ✔ ✔ ✔ Azerbaijan 29 20 ✔ ✔ Georgia 19 16 ✔ ✔ Kazakhstan 9 43 ✔ ✔ ✔ Kyrgyz Rep. 14 13 ✔ ✔ ✔ ✔ Mongolia 24 34 ✔ ✔ Pakistan 37 24 ✔ ✔ ✔ ✔ PRC 9 21 ✔ ✔ ✔ Tajikistan 0 4 Uzbekistan 13 1 ✔ ✔ ✔ Sources: UNESCO, UNICEF, and World Bank (2021); UNESCO’s COVID-19 Global Monitoring Database, COVID-19 Education Response. https://covid19.uis.unesco.org/global-monitoring-school-closures-covid19/country-dashboard/. Figure 1: Reported Duration of School Closures, % Households Notes: The abbreviations (iso3) of the countries are as follows: Afghanistan (AFG), Azerbaijan (AZE), Georgia (GEO), Kazakhstan (KAZ), Kyrgyz Republic (KGZ), Mongolia (MNG), Pakistan (PAK), Tajikistan (TAJ), and Uzbekistan (UZB). Source: Authors’ estimates from the survey. ADBI Working Paper 1450 W. Kodama et al. 11 Figure 4: Probability of “Severe Learning Loss” by Remote Learning Modes Notes: This figure reports the predicted probability of very little or no progress during school closure. The estimation is based on Column 1 in Table 4. 4.3 Effectiveness of Multiple Distance Learning Modes Even though we found that only online classes were effective in mitigating severe learning loss due to longer school closures, the role of using multiple remote learning modes is not considered in our previous regression. Specifically, paper lessons may enhance the effectiveness of online classes or TV programs. To elicit the effectiveness of using multiple remote learning modes, we estimate the same equation as in Table 4 but with (i) a binary variable that represents longer school closure (= 1 if longer than six months and = 0 if otherwise) to simplify the econometric model, and (ii) interaction terms between multiple learning modes (online classes and paper lessons; TV programs and paper lessons) and the duration of school closure. Table 5 reports the results. As in Table 4, a longer school closure duration increases the probability of severe learning loss (Column 1) or higher severity of learning loss (Column 2). The use of online classes relates to a lower probability of severe learning loss, but its combination with paper lessons does not bring about any further benefits. In contrast, even though the use of TV programs alone is not related to a lower probability, its combined usage with paper lessons mitigates the negative effects of longer school closures. To demonstrate the role of single and multiple remote learning modes, Table 6 shows the estimated probability of severe learning loss by duration of school closure (less than vs. more than six months) and types of learning modes, based on the results from the regression reported in Column 1 of Table 5. Without taking any remote learning mode, on average the probability increases by 33% when the school closure duration is longer than six months. This probability increase is still 26% and 42% with online classes alone and TV programs alone, respectively. The usage of only paper lessons relates to the lowest probability of severe learning loss under shorter school closures ADBI Working Paper 1450 W. Kodama et al. 12 (< 6 months), but the probability increases sharply by 90% when the duration is longer (> 6 months). Under longer school closures, online classes, online and paper lessons, and TV and paper lessons relate to the lowest probabilities of severe learning loss (0.430, 0.429, and 0.426, respectively). The benefits of using multiple learning modes are seen for TV programs under longer school closure periods. Compared to online classes, TV programs are not interactive, and students tend to be less active. Paper lessons (e.g., homework) may supplement this limitation of TV programs by enhancing students’ efforts with learning. Table 5: Single and Multiple Remote Learning Modes and Perceived Learning Progress (1) Logit Model (2) Ordered Logit Model (= 1 if Very Little or No Progress; = 0 Otherwise) (= 1 if Same, = 2 if Slower, = 3 if Very Little, = 4 if No Progress) Coefficient Marginal Effect Coefficient Marginal Effect Duration (> 6 months) 0.686*** (0.200) 0.160*** (0.047) 0.582*** (0.173) 0.064*** (0.019) Mode of remote learning Online classes –0.355** (0.180) –0.083** (0.042) –0.317** (0.156) –0.035** (0.017) TV programs –0.099 (0.196) –0.023 (0.046) 0.017 (0.147) 0.002 (0.016) Paper lessons –0.927*** (0.167) –0.217*** (0.039) –1.023*** (0.157) –0.112*** (0.018) Online # Paper 0.646** (0.292) 0.151** (0.068) 0.799*** (0.250) 0.088*** (0.028) TV # Paper 1.210*** (0.321) 0.283*** (0.075) 1.109*** (0.267) 0.121*** (0.029) Duration # Mode Duration # Online –0.355 (0.230) –0.083 (0.054) –0.349* (0.194) –0.038* (0.021) Duration # TV 0.156 (0.242) 0.037 (0.057) –0.053 (0.185) –0.006 (0.020) Duration # Paper 0.430 (0.271) 0.100 (0.063) 0.418* (0.235) 0.046* (0.026) Duration # Online # Paper 0.028 (0.399) 0.007 (0.093) –0.223 (0.340) –0.024 (0.037) Duration # TV # Paper –1.533*** (0.422) –0.358*** (0.098) –1.273*** (0.354) –0.139*** (0.039) Country fixed effects ✔ ✔ Other control ✔ ✔ Observations 3,553 3,553 Notes: Asterisks (*, **, and ***) denote significance at 10, 5, and 1 % levels, respectively. Standard errors are reported in parenthesis. The marginal effect of the ordered logit model shows the effect on the probability of “no progress.” Source: Authors’ estimates from the survey. Table 6: Probability of Severe Learning Loss by Single and Multiple Remote Learning Modes Duration of School Closure Differences Less than 6 Months More than 6 Months % Change Chi-squared None 0.424 (0.027) 0.564 (0.033) 33.02 11.97*** Online classes only 0.341 (0.021) 0.430 (0.022) 26.10 9.62*** TV programs only 0.394 (0.042) 0.561 (0.039) 42.39 8.76*** Paper lessons only 0.239 (0.217) 0.455 (0.037) 90.38 29.41*** Online and Paper 0.324 (0.043) 0.429 (0.040) 32.41 3.37* TV and Paper 0.434 (0.057) 0.426 (0.053) –1.84 0.01 Observations 3,553 Notes: The estimates are based on the regression reported in Column 1 of Table 5. Asterisks (*, **, and ***) denote significance at 10, 5, and 1 % levels, respectively. Standard errors are reported in parenthesis. Source: Authors’ estimates from the survey. ADBI Working Paper 1450 W. Kodama et al. 13 4.4 Heterogeneity We also investigate the heterogeneity across grade levels in 2019 (before the school closures) using the same specification as in Table 5 to distinguish the role of single and multiple remote learning modes. We consider the grade levels of: (i) Grades 1 to 3 in elementary school; (ii) Grades 4 to 6 in elementary school; (iii) middle school (typically Grades 6 to 9); and (iv) high and vocational school (typically Grades 10 to 12). Table 7 shows the results. For all grade groups except for high school, a longer school closure duration is related to a higher likelihood of “severe learning loss.” When the school closure duration is shorter (less than six months), paper lessons are related to a lower probability of learning losses for all grade groups except for middle school. Online classes also relate to lower probabilities for middle and high school students. However, TV programs have no positive effects on learning progress. Table 7: Remote Learning Modes and Perceived Learning Loss by Grade Level in 2019, Overall Marginal Effects of Logit Model (1) (2) (3) (4) Elementary 1–3 Elementary 4–6 Middle School High School Duration (> 6 months) 0.220*** (0.072) 0.179* (0.093) 0.195** (0.090) –0.119 (0.142) Mode of remote learning Online classes 0.058 (0.066) –0.055 (0.096) –0.142* (0.082) –0.382*** (0.131) TV programs –0.019 (0.068) 0.163* (0.088) –0.152 (0.096) –0.119 (0.233) Paper lessons –0.259*** (0.059) –0.053 (0.085) –0.297*** (0.095) –0.269** (0.118) Online # Paper 0.099 (0.110) 0.263** (0.130) 0.295** (0.147) 0.560*** (0.209) TV # Paper 0.408*** (0.113) –0.183 (0.149) 0.156 (0.188) 0.128 (0.350) Duration # Mode Duration # Online –0.161** (0.082) –0.110 (0.111) –0.008 (0.109) 0.278* (0.166) Duration # TV 0.083 (0.087) –0.171 (0.105) 0.069 (0.112) 0.095 (0.266) Duration # Paper 0.046 (0.096) 0.001 (0.128) 0.225 (0.151) 0.702** (0.304) Duration # Online # Paper –0.038 (0.148) –0.080 (0.175) –0.193 (0.202) –1.031*** (0.373) Duration # TV # Paper –0.330** (0.152) 0.084 (0.185) –0.244 (0.230) –0.117 (0.457) Country fixed effects ✔ ✔ ✔ ✔ Other control ✔ ✔ ✔ ✔ Observations 1,263 876 830 324 Notes: Asterisks (*, **, and ***) denote significance at 10, 5, and 1 % levels, respectively. Standard errors are reported in parenthesis. Source: Authors’ estimates from the survey. When the school closure duration is longer (longer than six months), for elementary Grade 1–3 students, online classes appear to be effective in mitigating severe learning loss. Combining online classes and paper lessons is also effective for high school students. Even though combined TV programs and paper lessons relate to a lower probability for elementary Grade 1–3 students, the effects are canceled out by their effect without the interaction term. Overall, these heterogeneities confirm the effectiveness of online classes and suggest the importance of providing a mode of remote learning that is appropriate for a child’s grade level. ADBI Working Paper 1450 W. Kodama et al. 14 5. CONCLUSION AND POLICY RECOMMENDATIONS School closures negatively affect children’s education outcomes. During school closures caused by the COVID-19 pandemic in Central Asia and the Caucasus, online classes and TV programs were introduced as remote learning tools to reduce learning loss, along with traditional paper lessons. Based on a household survey covering nine countries, this study documents parents’ perceived learning losses of children and the effectiveness of remote learning modes during school closures. Our survey data suggest that 65% and 30% of children used online classes and TV programs, respectively, and 36% relied on conventional paper lessons. Some used multiple modes, while as many as 24% of children did not use any remote learning tools. Despite the governments’ efforts to minimize learning losses with remote learning tools, 79% of households with children enrolled in school felt that their children’s learning progress was slower than in in-person schooling, while 39% believed that their children had shown “very little progress” or “no progress.” Our econometric analysis revealed some important findings. First, the usage of online classes is strongly correlated with advantageous household characteristics such as a good internet connection, urban residence, and a higher household income. TV programs were more commonly used by rural households. Second, online classes are found to reduce learning loss during school closures, but this does not hold for TV programs. Online classes also reduce the negative effects of a longer school closure duration on perceived learning progress. Third, on average, paper lessons are also found to be an effective mode of remote learning, but their effectiveness is limited to shorter school closure durations. Finally, combined usage of TV programs and paper lessons seems to be effective in mitigating the negative effects of longer school closure durations. The above findings have the following policy implications. During school closures, which may also be due to natural disasters, air pollution, extreme weather, conflicts, etc., or more generally when a child is unable to go to school, online classes appear to be the best option for minimizing learning loss (as this tool provides interaction). However, in places with limited access to the internet or smart devices they are unlikely to be used by all children. In such places, paper lessons could be used as a way to reduce learning loss for a short school closure duration. Overall, building the capacity to provide children with disadvantaged backgrounds with effective access to online classes is critically important for reducing learning losses and for preventing widening gaps in terms of learning losses during future school closures. In addition, in the absence of access to online lessons during long-term school closures, the usage of TV programs together with paper lessons could be an effective alternative tool, which requires further investigation into its effectiveness. ADBI Working Paper 1450 W. Kodama et al. 15 REFERENCES ADB. 2022. Learning and Earning Losses from COVID-19 School Closures in Developing Asia. In Asian Development Outlook 2022: Mobilizing Taxes for Development. Manila: Asian Development Bank. Agostinelli, F., M. Doepke, G. Sorrenti, and F. Zilibotti. 2022. When the Great Equalizer Shuts Down: Schools, Peers, and Parents in Pandemic Times. Journal of Public Economics 206: 104574. https://doi.org/10.1016/j.jpubeco.2021.104574. Alasino, E., M. J. Ramírez, M. Romero, N. Schady, and D. Uribe. 2024. Learning Losses During the COVID-19 Pandemic: Evidence from Mexico. Economics of Education Review 98: 102492. https://doi.org/10.1016/ j.econedurev.2023.102492 Awal, R. 2023. Measuring the Effectiveness of Online Classes During the COVID-19 Pandemic: Case Study of a Government College in Bangladesh. Journal of Management and Business Education 6(1): 43–57. https://doi.org/10.35564/ jmbe.2023.0003. Azhgaliyeva, D., R. Mishra, T. Long, P. Morgan, and W. Kodama. 2022. Impacts of COVID-19 on Households’ Business, Employment and School Education: Evidence from Household Survey in CAREC Countries. ADBI Working Paper, 1335. https://doi.org/10.56506/spwh1535. Engzell, P., A. Frey, and M. D. Verhagen. 2021. Learning Loss Due to School Closures During the COVID-19 Pandemic. Proceedings of the National Academy of Sciences 118(17). https://doi.org/10.1073/pnas.2022376118. Haelermans, C. et al. 2022. Sharp Increase in Inequality in Education in Times of the COVID-19 Pandemic. PLoS ONE 17: e0261114. https://doi.org/10.1080/ 00313831.2021.2006307. Hanushek, E., and L. Woessmann. 2020. The Economic Impacts of Learning Losses. OECD Education Working Papers, No. 225. OECD Publishing, Paris, https://doi.org/10.1787/21908d74-en. Iqbal, S. A., and H. A. Patrinos. 2023. Learning During the Pandemic: Evidence from Uzbekistan. Policy Research Working Papers. https://doi.org/10.1596/18139450–10474. Kodama, W., P. Morgan, D. Azhgaliyeva, T. Long, and K. Kim. Forthcoming. Family Business during the COVID-19 Pandemic in Asia: Role of Government Financial Aid and Coping Strategies. World Development. Lichand, G., C. A. Doria, O. Leal-Neto, and J. P. Fernandes. 2022. The Impacts of Remote Learning in Secondary Education During the Pandemic in Brazil. Nature Human Behaviour 6(8): 1180–1180. https://doi.org/10.1038/s41562-02201420-9. Maddawin, A., P. Morgan, A. Park, D. Suryadarma, T. Q. Long, and P. Vandenberg. 2024. Learning Disruptions during the COVID-19 Pandemic: Evidence from Household Surveys in Southeast Asia. International Journal of Educational Development, 107: 103053. https://doi.org/10.1016/j.ijedudev.2024.103053. ADBI Working Paper 1450 W. Kodama et al. 16 Marteau, J. F. 2020. Post-COVID Education in Kazakhstan: Heavy Losses and Deepening Inequality. World Bank Blogs. https://blogs.worldbank.org/ europeandcentralasia/post-covid-education-kazakhstan-heavy-losses-anddeepening-inequality. Meliboeva, N., H. Patrinos, and J. Teixeira. 2020. Uzbekistan: A Timely Response on Learning During School Closures. https://blogs.worldbank.org/ europeandcentralasia/uzbekistan-timely-response-learning-during-schoolclosures (accessed 22 December 2023). Ministry of Education, Science, Culture, and Sports of Georgia. 2020. Pandemic and General Education in Georgia. https://mes.gov.ge/mesgifs/1609844890_2020Annual-report-ENG.pdf (accessed 22 December 2023). Ministry of Science and Education of Azerbaijan. 2020. Azerbaijan: How One Ministry Found the Right Strategy, Resources, and Technology to Quickly Create Online Classrooms. https://www.edu.gov.az/en/news-and-updates/18629 (assessed 22 December 2023). Moscoviz, L., and D. Evans. 2022. Learning Loss and Student Dropouts during the COVID-19 Pandemic: A Review of the Evidence Two Years after Schools Shut Down. Washington, DC: Center for Global Development. https://www.cgdev.org/ sites/default/files/learning-loss-and-student-dropouts-during-covid-19-pandemicreview-evidence-two-years.pdf. Patrinos, H. A. 2023. The Longer Students Were Out of School, the Less They Learned. Journal of School Choice 17(2): 161–175. https://doi.org/10.1080/ 15582159.2023.2210941. Patrinos, H., Vegas, E., and Carter-Rau, R. 2023. An Analysis of COVID-19 Student Learning Loss. In The Oxford Research Encyclopedia of Economics and Finance, edited by Jonathan Hamilton. Oxford: Oxford University Press. Roman, M., and A. Plopeanu. 2021. The Effectiveness of the Emergency eLearning during COVID-19 Pandemic. The Case of Higher Education in Economics in Romania. International Review of Economics Education 37: 100218. https://doi.org/10.1016/j.iree.2021.100218. Swan, K. 2003 . Learning Effectiveness: What the Research Tells Us. In Elements of Quality Online Education, Practice and Direction, edited by J. Bourne and J. C. Moore. Needham, MA: Sloan Center for Online Education, pp. 13–45. United Nations. 2020. Distance Learning in Kazakhstan under Quarantine. https://www.astanacivilservicehub.org/uploads/covid/files/Distance%20learning_ Kazakhstan%20case.pdf (accessed 22 December 2023). UNESCO. 2021. School Closures and Regional Policies to Mitigate Learning Loss Due to COVID-19: A Focus on the Asia-Pacific. Paris: UNESCO Publishing. UNESCO, UNICEF, and World Bank. 2021. The State of the Global Education Crisis: A Pathway to Recovery. Paris: UNESCO Publishing. UNICEF. 2020, May. UNICEF Azerbaijan Country Office COVID-19 Situation Report No. 5. https://reliefweb.int/report/azerbaijan/unicef-azerbaijan-country-officecovid-19-situation-report-no-5-1-may-2020. ADBI Working Paper 1450 W. Kodama et al. 17 APPENDIX Table A1: Income Levels in Local Currency for Each Country Uzbekistan, sum ≤ 1,200,000 1,200,001– 2,000,000 2,000,001– 3,200,000 < 3,200,000 Note: Since different countries have different numbers of income classes, in our empirical analyses we recategorized the income classes into four groups to make them consistent across countries. See Azhgaliyeva et al. (2022) for more details. Tajikistan, somoni ≤ 800 801– 1,400 1,401– 2,400 < 2,400 Pakistan, rupee ≤ 5,000 5,001– 15,000 15,001– 30,000 30,001– 60,000 > 60,000 Mongolia, tugrug ≤ 500,000 500,001– 900,000 900,001– 1,100,000 1,100,001– 2,100,000 > 2,100,001 Kyrgyz Republic, som ≤ 6,000 6,001– 12,000 12,001– 20,000 > 20,000 Kazakhstan, tenge ≤ 60,000 60,001– 100,000 100,001– 150,000 150,001– 250,000 250,001– 300,000 > 300,000 Georgia, gel ≤ 299 300–599 600–999 1,000– 1,499 > 1,500 Azerbaijan, manat ≤ 820 820,1– 1, 035,0 1, 035,1 – 1, 240,0 1, 240,1 – 1, 650,0 > 1, 650,1 Afghanistan, afn ≤ 5, 000 5,001– 15,000 15,001– 30,000 30,001– 60,000 > 60,000 Revised Income Class 1 2 3 4 Original Income Class 1 2 3 4 5 6