(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Effective remediation programs for vulnerable students to overcome learning loss [1]

['Madelon Jacobs', 'Research Centre For Education', 'The Labour Market', 'Roa', 'School Of Business', 'Economics', 'Maastricht University', 'The Netherlands', 'Carla Haelermans', 'Netherlands Initiative For Education Research']

Date: 2025-06 

Abstract In March 2020, the world was forced to cope with the COVID-19 pandemic, and as a result, schools in many countries closed to minimize the spread of the coronavirus. Students were required to shift to online learning from home. Not long after that, research showed that students in the first half-year of the COVID-19 pandemic experienced extensive learning losses compared to peers from previous cohorts. An important question is whether these losses can be remediated or lead to permanently lower attainment for a generation of students. Here, we study the effectiveness of a nationwide attempt at remediation in the Netherlands, where the government provided funds for schools to proactively minimize the detrimental effects of the COVID-19 pandemic. Unique standardized test score data of over 66,000 Dutch primary school students were combined with information about the various remediation programs employed uniquely in each school to investigate to what extent remediation occurred. Applying a Difference-in-Differences design, we find an overall statistically significant and meaningful increase in the achievement of vulnerable students after participation in remediation programs (+ 0.05 SD). Participating students had a higher increase in test scores compared to non-participants within the same school, and the inequality between the latter and former has slightly been reduced (~10%). Remediation programs offering individual support to students or in small groups, and with a sole emphasis on cognitive skill development, were most successful.

Citation: Jacobs M, Haelermans C, Meeter M (2025) Effective remediation programs for vulnerable students to overcome learning loss. PLoS One 20(5): e0323352. https://doi.org/10.1371/journal.pone.0323352 Editor: Jake Anders, University College London, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND Received: April 30, 2024; Accepted: April 7, 2025; Published: May 14, 2025 Copyright: © 2025 Jacobs et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: This section contains information on the data collection for the study "The Effects of Remediation Programs during the COVID-19 Pandemic on the Achievements of Vulnerable Students". The study is based on data from the Netherlands Cohort Study on Education (NCO in Dutch) and information about remediation programs sourced from schools, including participation lists and content details. Access to the NCO data is only possible via the Remote Access (RA) infrastructure of Statistics Netherlands. The data cannot be shared publicly because of privacy restrictions and cannot be exported from the secure virtual environment at Statistics Netherlands. However, all data are available at Statistics Netherlands, and researchers can request data access provided they work at an institution with which Statistics Netherlands has agreements. Requests for data access can be sent to info@nationaalcohortonderzoek.nl. A detailed description of the data and the access procedure can be found in Haelermans, C., T. Huijgen, M. Jacobs, M. Levels, R. van der Velden, L. van Vugt and S. van Wetten (2020). Using Data to Advance Educational Research, Policy and Practice: Design, Content and Research Potential of the Netherlands Cohort Study on Education. European Sociological Review, 36(4), 643-662, https://doi.org/10.1093/esr/jcaa027. However, data on remediation programs are not openly requestable for researchers. Data obtained from funding recipients and participation lists obtained from schools are confidential and cannot be shared due to legal and ethical restrictions. Researchers seeking access for replication purposes may contact the authors and can create a project at Statistics Netherlands (provided they work at an institution with which Statistics Netherlands has agreements). The authors are allowed to provide the data on the remediation programs under strict conditions within other projects at Statistics Netherlands. Funding: This research was supported by the Netherlands Initiative for Education Research (NRO), project number 40.5.20937.001. The funders had no role in study design, data collection and analyses, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist.

Introduction The start of the COVID-19 pandemic in March 2020 forced countries around the globe to take rigorous measures to decrease the spread of the coronavirus. One of these measures was closing schools, and, in some countries, education shifted to online distance learning, whereas in other countries, students received no education during the COVID-19 pandemic at all [1]. Consequently, it has created a severe crisis in education and learning [2], where students spent between one week to more than one school year at home [3]. Research showed significant learning losses after the school closures compared to students from previous cohorts in countries worldwide [4–11]. Vulnerable students faced disproportionate learning losses after the school closures, causing inequalities to rise between students [12,13]. The literature uses many terms to describe the detrimental effects of school closures on students’ achievements, including learning delays, learning growth, learning loss, student achievements, and educational outcomes. This paper uses the term learning loss. With the apparent learning losses and the affected students in countries worldwide, reopening schools and offering remediation was urgent. Half of all countries worldwide increased their education budget in 2020 and 2021 compared to pre-COVID-19 years [14]. Around 40% of all nations made various efforts to counteract the learning loss, from extending the academic year to prioritizing specific skills over other learning areas usually taught in school. Furthermore, many schools implemented programs specifically for the most vulnerable and disadvantaged students. These programs aimed to assist students who lagged behind in school, enabling them to catch up, counteract learning losses, and avoid grade retention. In a quick attempt to counteract the learning losses, the Dutch Ministry of Education, Culture and Science provided funds to schools for remediation programs after the first half-year of the COVID-19 pandemic. In this paper, we focus on the introduction of these well-funded remediation programs in primary education in the Netherlands. Using the exceptional nationwide database of standardized test scores in the Netherlands to assess learning losses, these data can provide insight into the remediation efforts provided by these programs. We refer to these programs, set up to tackle the COVID-19 learning loss, as “remediation programs”, as these are not clean individual interventions set up in experimental settings but are more fuzzy and often consist of many elements. This research explores the characteristics of students who participated in remediation programs, identifying the criteria for their selection (research aim 1). Furthermore, it examines the relationship between participation in these programs and student achievements (research aim 2). Additionally, the study aims to explore the diversity in characteristics of remediation programs to identify the most effective characteristics in enhancing student achievements (research aim 3). The remainder of this paper shows that vulnerable students are rightfully selected for remediation programs and that, on average, these programs increase students’ achievements.

Relevant literature After the first school closures due to the COVID-19 pandemic, many studies looked at students’ learning losses in school, and several review studies are now available [4,8,11]. These review studies show that in almost all countries around the world, students learned less during COVID-19. For example, Betthäuser, Bach-Mortensen [11] show that learning losses occurred across 15 countries, based on 42 studies, with an average learning loss of 0.14 SD. Furthermore, they show that learning losses are more prominent among young students and students from lower socio-economic backgrounds [8,15]. Besides the learning losses in mathematics and reading, students also struggled in other areas related to education, such as their non-cognitive skills or their well-being [16–18]. We refer to the cited review studies for more discussion of learning loss; below, we focus on studies that looked at potential solutions for remediating cognitive and/or non-cognitive skills. With the knowledge of the learning losses occurring after the school closures, initiatives have been taken to make up for them. Even before the COVID-19 pandemic, the effectiveness of educational interventions was an important research topic. Numerous studies have examined effective interventions to improve skills such as reading, mathematics, and other cognitive and non-cognitive skills [15,19–35]. Many types of interventions have been developed to assist struggling students with additional support, time, and instruction. These type of interventions often have several important aspects in common. The first common aspect is that additional instruction is more tailored to the specific needs of struggling students. If classroom instructions are too challenging, students will not develop effectively. Hence, the additional instructional time should be better adjusted to the needs of the students. This aligns, for example, with Vygotsky’s theory on the zone of proximal development [36], where students require a specific level of difficulty to perform optimally. The second common aspect is that additional instruction or support is often provided in smaller groups. This allows students better opportunities to receive (more) individual feedback and improves their engagement. This corresponds with Slavin’s findings on small-group instruction [37]. Thirdly, additional instruction could also be defined as more guidance, where struggling students receive additional guidance compared to peers in the classroom. This is in harmony with Sweller’s cognitive load theory [38], which states that students learn more effectively when their cognitive load is minimised through structured, step-by-step guidance. These three common aspects occur in many, if not most, of the interventions focused on remediation. Below, we will summarize the interventions that are most applicable to the case of the Netherlands. One of the most effective intervention, is one-to-one tutoring. In this intervention, a student receives individual support outside the classroom. The student receives extensive individual support specifically targeting particular areas, such as mathematics or reading. The average effect size is estimated to be around 0.40 SD increase in attainment or roughly six months of additional progress [22]. On average, one-to-one tutoring seems more effective for reading (+0.49 SD) than mathematics (+0.14 SD) [22]. Besides one-to-one tutoring, small-scale tutoring is an often used intervention. This intervention may be preferred as one-to-one tutoring can be expensive. Small-scale tutoring teaches several students simultaneously, often between two and five students per group. Groups larger than six or seven students see a reduction in effectiveness [24]. The group setting allows students to learn from each other and the teacher. In this type of intervention, the average effect size is around 0.31 SD increase in test scores [24,27]. Similar to one-to-one tutoring, the intervention seems to be slightly more effective for reading (+0.31 SD) compared to mathematics (+0.23 SD) [24]. Smaller classrooms are also an option to provide more customized education. However, it is argued to be a costly intervention for relatively smaller effect sizes [25]. Average effect sizes are considered low, with an increase of around 0.14 SD, or around 2 months of additional progress. Smaller classrooms also seem more effective for reading (+0.14 SD) than mathematics (+0.08 SD). Teaching students effective learning strategies could further improve educational performance for struggling students. Research indicates that metacognition improves performance, often defined as learning to learn [39]. Teaching students how to plan, monitor, and evaluate their own learning increases their ability to retain information and apply knowledge. These interventions equip students with tools to learn more effectively on their own. Research indicates that the average effect sizes range from 0.62 to 0.69 SD, corresponding to about 8 months of progress. Metacognitive strategies have been particularly effective for mathematics [40]. Individualized instruction, or differentiation, involves providing students personalized tasks and support, often while using educational technology. It is based on the idea that all students have unique needs, and thus, a customized approach, often regarding the pace of learning, will increase educational performances. Various strategies of individualized instruction have been studied, often for mathematics, where students can work through individual sets of tasks, often independently. Average effect sizes range from approximately 0.19 to 0.26 SD, corresponding to about three months of progress [41]. Extended school days, where students stay at school for longer, increase students’ educational achievements. If used effectively, this strategy provides more opportunities for direct instruction and practice after school [42]. Longer school days allow teachers more time to assess student progress and provide additional instruction. Without structured guidance, however, extending the school day could have no effect [43], where it may result in student fatigue and diminishing returns [42]. The average effect sizes of this intervention range from approximately 0.19 to 0.26 SD for both mathematics and reading in primary education [44]. Summer schools are another type of intervention that extends the official school year. They select a small group of students to receive additional teaching and instructions over the summer break [23,28,29,35]. The average effect size is around 0.23 SD, or 3 months of additional progress [23]. The effects appear larger for reading (0.23 SD) than for mathematic progress (0.14 SD). Some types of interventions are targeted at all students within the classroom and are meant to increase the learning and support for all students, such as improving instructional quality in the regular classroom and the professional development of teachers. Successful programs of this type are characterized by their emphasis on student engagement and motivation strategies [34], which are expected to enhance student learning outcomes. These interventions are potential solutions for providing additional support to low-achieving students, without organizing additional programs for these students, with average effect sizes ranging around 0.33 SD for mathematics [33] or 0.56 SD for reading [27]. Improving resources and study material is another type of intervention that can be used for struggling students [30–32]. Offering additional resources to schools can include increasing the number of teachers [30], but often refers to implementing adaptive online programs for specific subjects [19]. Research by Ma, Adesope [32] shows adaptive online programs improve achievement in math and reading, with effect sizes ranging from 0.36 to 0.57 SD compared to traditional instruction. However, Steenbergen-Hu and Cooper [31] suggest these programs are more effective for students with higher prior knowledge, which may limit their effectiveness in the COVID-19 setting. It is important to note that the success of the interventions discussed above is contingent upon the quality of their implementation and the expertise of the teacher/tutor(s) involved. Therefore, it is more accurate to consider these interventions as “promising” rather than guaranteed solutions. Their effectiveness may not be as pronounced as demonstrated in prior studies, and the potential for variation in outcomes should be taken into account. We expect to find lower effects compared to experimental interventions.

Setting in the Netherlands In the Netherlands, students experienced eight weeks of distance learning due to school closures related to COVID-19, with teaching functioning at a 30% efficiency compared to traditional in-school teaching in the spring of 2020 [10]. The first official school closure lasted for eight weeks, including two weeks of holidays, so students missed six weeks of school. After the schools reopened, students had four weeks of half-time learning at school and half-time distance learning at home, so they missed an additional two weeks of learning at school. In total, therefore, students missed eight weeks of school. Previous research in the Netherlands showed significant learning losses and rising inequalities among students [5,7] after the school closure from March 16th, 2020, to May 10th, 2020 [45]. The results showed that students had, on average, a learning loss around 0.08 SD, while students from lower-educated parents faced 60% larger losses [5]. Other research in the Netherlands showed that after a full year of the COVID-19 pandemic, including two school closures, average learning losses were 0.17 SD for reading and 0.12 SD for mathematics [7]. Furthermore, based on parental education and parental income, students from disadvantaged backgrounds faced larger losses on top of already existing inequalities. To quickly counteract the learning losses, the Dutch Ministry of Education, Culture and Science provided funds (around 3.4 billion euros, which is around 8% of the yearly education budget, or 0.3% of GDP) to schools for remediation programs after the first half-year of the COVID-19 pandemic. When schools fulfilled the specified criteria for the program, they were eligible to seek financial assistance from the Ministry. These conditions included a minimum of 25 hours of remediation per student, the involvement of the lowest-performing students, and the ability to request funds for up to 10% of the total student population in the school. Most schools received funds for 10% of their students. However, disadvantaged schools received funds for 20% of their students. This was determined based on the school-disadvantage score, which expresses expected educational disadvantages at the school level. Furthermore, the programs had to be carried out in the school year 2020/2021. Besides these criteria, schools were free to design the remediation program as they deemed necessary to counteract the learning losses among their students [19]. Schools could apply for funding through several calls, the first call was between the 2nd and 21st of June 2020, and the second was between the 18th of August and the 18th of September 2020. The Netherlands is an ideal case to assess the impact of remediation programs on students’ achievement in primary education. The data infrastructure allows looking at educational trajectories and achievements using longitudinal administrative data from the Netherlands Cohort Study on Education [46]. National standardized test scores in comprehensive reading and mathematics measure students’ achievements twice a year. Moreover, detailed information regarding the content and participants of the remediation programs, as well as students’ background information, is available. Combined with register data from Statistics Netherlands, unique, high-quality data on 66,439 students is available.

Conclusion and discussion COVID-19 has created an enormous crisis in education and learning [2], with vulnerable and disadvantaged students in particular facing large learning losses due to school closures [4–7,10,12,13]. Around 40% of all countries made various efforts at remediation. Here, we study the effectiveness of one remarkably swift and ambitious attempt at remediation. Upon recognizing the learning losses experienced by students during the COVID-19 pandemic, the Dutch government allocated funds to schools, allowing them to establish remediation programs based on their own preferences. We show that vulnerable students with disadvantaged backgrounds were selected for the remediation programs (research aim 1), and that there was a significant positive increase in test scores after participation of 0.05 SD (research aim 2). Especially individual or small-scale remediation programs, and programs with a sole emphasis on cognitive skill development, result in an increase in achievements after participation (research aim 3). Our study shows that remediation programs that address content gaps due to the COVID-19 pandemic can be effective. This approach is not new; however, the rapid implementation of these programs and the diversity of schools have resulted in varying levels of success. Although not universally effective, many programs have shown enough promise to justify allocating funds for their development. Furthermore, the results show that schools can manage this autonomy in the Dutch context, where schools enjoy considerable autonomy. Our study shows that the students who most needed support were correctly identified and enrolled in these remediation programs. Furthermore, schools were able to develop and implement programs tailored to the needs of these students. This suggests a robust capacity within schools to use their autonomy in ways that benefit students, particularly in challenging circumstances such as those posed by the pandemic. Our Difference-in-Differences (DiD) analyses estimate the effect of remediation programs at approximately 0.05 SD, with a 95% confidence interval ranging from 0.019 to 0.085 SD. While this effect size may appear modest [50], it signals a meaningful and positive impact. Given the relatively short duration of these programs, typically between 25 and 40 hours, this gain is notable. To put this into perspective, the average yearly learning progress in primary education is about 0.4 SD [51] under normal circumstances, a benchmark that was likely unattainable during the pandemic. Even within these constraints, remediation programs contributed to students’ progress. Considering the time investment, the observed effect suggests that remediation programs provided a positive return in terms of learning gains relative to the limited instructional hours. Therefore, these remediation programs are a solution to the COVID-19 related learning losses. Relative to the 0.08 SD learning loss reported by Engzell et al. [5] or the 0.15 SD learning loss reported by Haelermans et al. [7], the remediation programs have successfully mitigated between half to one-third of the learning losses obtained in the first half year of the pandemic. Given the divergence in the pre-trends regarding test score progress between the midyear test of 2019/2020 and end-of-year test in 2019/2020 among participating and non-participating students, it is important to consider that the overall effects of the program might be biased in either direction. The estimate may be biased upward if the participating students would have recovered without the remediation programs, such as due to the reopening of schools in general. Conversely, it may be biased downward if disadvantaged students, who were more likely to participate, would have continued to experience lower learning growth without the support of remediation programs. Interpreting the results in this paper as a causal effect of the remediation programs requires caution. However, this approach aims to approximate causality as best as possible, considering that these programs have a non-random treatment group. While it remains possible that our estimates are subject to both upward and downward bias, we have taken the necessary steps to strengthen their robustness. Overall, the evidence indicates that remediation programs were effective, though the precise magnitude of the effect may vary depending on the chosen specification. The same caution is needed when interpreting the estimates regarding the different characteristics of remediation programs as these estimates may be subject to bias or may be confounded by unobserved heterogeneity. Our robustness checks, including PSM models and analyses focusing on the lower-performing half of students, show somewhat more comparable pre-trends. The results of these robustness checks are largely consistent with our main outcomes. It is important to note that while the effect sizes of interventions mentioned in previous studies in the main text were often more substantial, these were frequently measured in controlled experimental settings. Proven effective interventions in the past do not necessarily lead to effective interventions in the future [52], as interventions are seldom carried out as designed, partly explaining differences in expected outcomes. Furthermore, these controlled interventions were often conducted in small-scale settings with close involvement from developers, leading to high-quality implementations that may not be replicable in larger, real-world settings [33]. The fact that our research identifies significant effects in a natural context underscores the relevance of these interventions, even in less-than-ideal conditions. Additionally, we must consider that the remediation programs have been designed and implemented in a turbulent school year. Besides the chaotic time during the COVID-19 pandemic, a nationwide teacher shortage made designing and implementing the programs difficult. Lastly, a second school closure might have altered the planning and timing of the remediation efforts. This second school closure in the Netherlands took place from December 16th, to February 8th, leaving schools closed for another 7.5 weeks (including two weeks of Christmas holidays). This research is among the first to assess remediation programs after the COVID-19 pandemic at the national level. The diversity of remediation programs offered in the Netherlands allows us to study the effectiveness in a comparative setting. Whereas much previously mentioned research focuses on reading or mathematics, our research adds to existing knowledge by analyzing all programs on reading and mathematics. Besides, previous research into remediation programs has been (mostly) conducted outside of the Netherlands in different educational settings, mainly focused in the US and UK, for example, research by Slavin [27] and the Education Endowment Foundation [20–26]. Previous research showed that remediation efforts faded quickly [53]. Future research could track students over a more extended period and look at the long-term outcomes of these programs on vulnerable students, specifically the likelihood of grade retention in the upcoming years. The hope is that with remediation, students will be able to keep up better in their regular classes and not have to repeat a year. The Netherlands has since funded a National Program for Education, called the NPO, with which remediation initiatives are extended up to 2023. Future research could also look at the effects of a lengthier funding initiative over the past years since the COVID-19 pandemic. These results were found in one country that was unusually fast in setting up funds for remediation programs. It is an open question to what extent results would replicate elsewhere. One factor that would support generalization is that individual schools developed programs, which resulted in a wide array of program goals, designs, and implementations, unlike where one actor was chosen for design and implementation. The results found in this study may thus represent an average of what different centrally designed, uniformly implemented programs (such as that developed by EEF for schools in England and Wales [15]) would reach.

Acknowledgments The authors thank two anonymous reviewers, Bart Golsteyn, Babs Jacobs, Kars van Oosterhout, Nico Pestel, and Sanne van Wetten for their helpful comments on previous versions of this paper.

[END]
---
[1] Url: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0323352

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/