Context matters: critical evidence gaps in non-state education
This Blog Post is contributed by the Education Finance Network which convenes diverse education stakeholders with a focus on directing non-state resources toward creating inclusive, high-quality education in low- and middle-income countries globally. It is part of the NORRAG Blog Series on Missing Education Data which aims to further explore six themes that emerged from the inaugural summit and its accompanying papers and expert consultations. In this post, the author provides a review of the evidence gaps regarding non-state actors in education and implications for research and data governance.
Missing data and evidence in the education sector is a challenge for both public and non-state provision. In the non-state sector, which provides schooling to more than 350 million children globally[1], low-cost private schools often struggle to allocate resources to data collection, financial monitoring, and the alignment of their data with standardized public Education Management Information Systems. More broadly, there are also large-scale gaps in evidence in specific areas, such as gender, that are consistently missing across the non-state and public sectors. However, it is often difficult to identify what those specific gaps are amidst the vast amount of research that does exist.
The new Non-State Actors in Education: Evidence Gap Map (EGM), developed by the Education Finance Network, illuminates the breadth and scale of the involvement of non-state actors in education in low- and middle-income countries (LMICs). Built on a scoping review which included peer reviewed papers and grey literature, the EGM consolidates nearly 200 papers published since 2015. Research on 23 intervention areas is mapped into the overarching categories of financing mechanisms, education delivery model, and ancillary services. For each intervention, the map presents the outcomes measured by the studies, such as enrollment, affordability, and learning outcomes.
The studies are presented in an interactive visual map so that users can easily identify which interventions and outcomes have been subject to the most research, and each study is summarized to facilitate its interpretation. The tool can be used by decision-makers and practitioners to learn what interventions have shown evidence of effectiveness, and by researchers to understand what areas should be prioritized to address current critical gaps in the literature.
Where are the evidence gaps?
The following describes areas where an analysis of the EGM has identified a need for further research.
Impact of different education delivery models on improving education access
The impact of core education delivery models on access is concentrated around low-cost private schools and Public-Private Partnerships (PPPs). Many studies included in the EGM evaluate the aggregate impact on access of a variety of low-cost private school models, including for-profit school chains, single-school sole proprietors, faith-based, and philanthropic schools. Evidence on the affordability of these schools for low-income households is contrasting and often highly dependent on context. While some studies[2] conclude that low-cost private schools are out of reach for low-income families, other studies have indicated substantial access among low-income groups. For example, 16.5% of children in the poorest quintile in India[3] attend private schools, and that low-cost private schools in Uganda[4] successfully reach low-income and disadvantaged students.
Studies on PPPs offer promising evidence that this delivery model may increase access by expanding provision of primary and secondary education in a way that is more affordable to governments.[5] However, the aggregate impact of PPP education provision is inconclusive and highly dependent on contextual factors. [6][7]
Given the contrasting evidence around the impact of core delivery models, additional analysis should be conducted on the conditions under which low-cost private schools and PPPs do increase equity and access, and why they succeed in expanding access in certain contexts and not in others.
Impact of financing modalities on improving education access
The impact of financing modalities on student attendance, including female attendance, is mostly concentrated around vouchers schemes. Most studies and systematic reviews agree that voucher schemes do succeed in increasing school access and enrollment, particularly for girls and secondary-aged children who would otherwise be out of school. [8]
The evidence around other modalities, such as school fee loans and results-based financing, is more limited, although a rigorous evaluation of the Educate Girls Development Impact Bond in India found that the program was successful in expanding access for girls’ education. [9] Further research into innovative financing for education is required to investigate the impact of distinct modalities and to understand the conditions under which they are successful.
Impact on learning outcomes for low-income students
Many studies included in the EGM evaluate the aggregate impact of a variety of low-cost private school models on learning outcomes. There is moderate evidence from studies in India[10], Pakistan[11], Uganda,[12] and Kenya[13] that low-cost private schools succeed in improving learning outcomes. However, evidence remains unclear on whether this increase in learning outcomes holds true for the most disadvantaged and lowest-income students in these schools. Other studies in Columbia[14], Peru[15], Uganda[16], India[17] and Kenya[18] found no impact of private schooling on learning outcomes, especially when controlling for socio-economic status. The diverging findings in many country-specific studies indicate the need for global reviews and meta-analyses.
Equity outcomes in education beyond income levels
The EGM found significant gaps in evidence around outcomes on equity in education outside of income levels, particularly on gender outcomes and equity for marginalized groups. There is a limited focus on gender outcomes across all studies included in the EGM. While most studies disaggregate findings on learning outcomes and attendance by gender, they do not add additional analysis around the reasons for any discrepancy in attendance or outcomes between girls and boys. Very few studies disaggregate findings by, or include an analysis of, marginalized groups, such as children with disabilities, minority ethnic groups, and refugees.
School quality beyond test scores
Evidence on learning outcomes relies primarily on tests to assess student performance in literacy and numeracy, which is only one indicator of school quality and does not capture the complete picture. Additional research that assesses school quality outside of test scores is needed, as evidence is limited around whether various models of core delivery have a lower pupil-teacher ratio, offer a greater range of extra-curricular activities, or have access to better materials or resources. Likewise, very little evidence exists on school quality related to diversity and inclusion in language and curriculum, and ensuring curriculum is relevant to students’ background and contexts.
Further research to understand what works in each context
While the EGM shows that there is a vast body of evidence around the impact of non-state actors in education, it also identifies areas where more research is needed to understand the nuances of what works in each context. The evidence around impact of different delivery models is contrasting and highly dependent on context, calling for further research to understand the conditions under which specific models succeed in improving education access. The EGM also highlights a lack of evidence around the impact of financing modalities such as school fee loans and results-based financing. Echoing findings explored in another post in the Missing Data blog series[19], the EGM presents significant gaps in evidence around gender outcomes, which calls for improved data collection, disaggregation, and analysis. The expansion of the research base, comparison with government funded public schooling, and data-informed decision making could be further enhanced by improving the alignment and integration of private data systems with government Education Management and Information Systems.
By making these gaps visible, the Education Finance Network hopes that the EGM will encourage further research to improve our understanding of what interventions, in which contexts, are effective in moving us closer to achieving access to quality education for all.
Author
Launched in April 2022, the Education Finance Network convenes diverse education stakeholders, including foundations, donors, impact investors, and practitioner networks, with a focus on directing non-state resources toward creating inclusive, high-quality education in LMICs globally. With over 80 members across 29 countries, the Network has hosted events on various topics, including results-based finance for youth workforce development, public-private partnerships in education, and how impact investors can drive access to education for all children. The Network is member-driven, focusing on the interests and needs of its members to shape programming content, professional development activities, and topics for its research products. To learn more, including how to apply to become a member, visit our webpage.
Image credit: Opportunity International Edufinance
References
[1] UNESCO. Global Education Monitoring (GEM) Report 2021/2. Retrieved from: https://www.unesco.org/gem-report/en
[2] Aslam, M. (2017). Non-state education provision; access and quality for the marginalised.
[3] Gruijters, R. J., Alcott, B. and Rose, P. (2020) The effect of private schooling on learning outcomes in South Asia and East Africa: A within-family approach. Working Paper No. 20/7., REAL Centre, University of Cambridge. 10.5281/zenodo.3686733
[4] Economic Policy Research Centre. (2016). Evaluation of the PEAS network under the Uganda Universal Secondary Education Programme, Baseline Survey Report
[5] Barrera-Osorio, F, David S. Blakeslee, Matthew Hoover, Leigh Linden, Dhushyanth Raju, and Stephen P. Ryan. (2017). Delivering education to the underserved through a Public-Private Partnership program in Pakistan. NBER Working Paper 23870.
[6] Aslam, Monazza, Shenila Rawal, and Sahar Saeed. (2017). Public-Private Partnerships in education in developing countries: A rigorous review of the evidence. London: Ark Education Partnerships Group.
[7] Crawfurd, L. and Hares, S. (2021). The impact of private schools, school chains and PPPs in developing countries. Paper commissioned for the 2021/2 Global Education Monitoring Report, Non-state actors in education. Retrieved from:: https://unesdoc.unesco.org/ark:/48223/pf0000380060
[8] Day Ashley, L. Skinner, R., Meyer, A, and Perry, T. (2020). Private education and disadvantaged children in India: A literature review of three models of private school provision. Save The Children International. Retrieved from:: https://resourcecentre.savethechildren.net/pdf/private_education_ and_disadvantaged_children_in_india_literature_review_of_three_models_of_private_school_ provision_final.pdf/
[9] IDinsight. (2018). Educate Girls Development Impact Bond. Final evaluation report. Retrieved from: https://static1.squarespace.com/static/5b7cc54eec4eb7d25f7af2be/t/5bce543ee4966befb6 6c13e5/1540248811132/EG+DIB+Project+Report_Final_High+Res_Web.pdf
[10] Gruijters, R. J., Alcott, B. and Rose, P. (2020). The effect of private schooling on learning outcomes in South Asia and East Africa: A within-family approach. Working Paper No. 20/7., REAL Centre, University of Cambridge. 10.5281/zenodo.3686733
[11] Hafeez, F., Haider, A., and uz Zafar, N. (2016). Impact of Public-Private-Partnership programmes on students’ learning outcomes: Evidence from a quasi-experiment. The Pakistan Development Review, 55(4), 955–1017. Retrieved from: http://www.jstor.org/stable/44986026
[12] Gruijters, R. J., Alcott, B. and Rose, P. (2020). The effect of private schooling on learning outcomes in South Asia and East Africa: A within-family approach. Working Paper No. 20/7., REAL Centre, University of Cambridge. 10.5281/zenodo.3686733
[13] Baum, D. and Isaac Riley. (2019). The relative effectiveness of private and public schools: evidence from Kenya, School Effectiveness and School Improvement, 30:2, 104-130, DOI: 10.1080/09243453.2018.1520132
[14] Balsera, M., Delphine Dorsi, Andreu Termes, Xavier Bonal, Antoni Verger and Javier Gonzalez Diaz. (2016). Private actors and the right to education, Compare: A Journal of Comparative and International Education, 46:6, 976-1000, DOI: 10.1080/03057925.2016.1207939
[15] Eigbiremolen, G. O., Ogbuabor, J.,E., and Nwambe, C.S. (2019). Estimating Private School Effects for School Children in peru: Evidence from Individual-level Panel Data. Journal of International Development 32(2): 131-149. https://doi.org/10.1002/jid.3438
[16] Masuda, K. and C. Yamauchi. (2018) .The effects of universal secondary education program accompanying Public-Private Partnership on students’ access, sorting and achievement: Evidence from Uganda. CEI Working Paper Series 2018-4, Center for Economic Institutions, Institute of Economic Research, Hitotsubashi University.
[17] Muralidharan, K. and Sundararaman, V. 2015. The aggregate effect of school choice: evidence from a two-stage experiment in India. The Quarterly Journal of Economics. Vol. 130, Issue 3.
[18] Zuilkowski, S. S., Piper, B., and Ong’ele, S. (2020). Are low-cost private schools worth the investment? Evidence on literacy and mathematics gains in Nairobi primary schools. Teachers College Record, 122(1), 1–30.
[19] Unterhalter, E., Longlands, H. and Vaughan, R.P. (2022). Missing Education Data: Gender, Data and Measurement. Retrieved from: https://www.norrag.eight-id.com/missing-education-data-gender-data-thematic-session/