Aligning Key Initiatives in Monitoring Learning by Luis Crouch and Silvia Montoya

By Luis Crouch, RTI, and Silvia Montoya, UNESCO Institute for Statistics.

In a recent blog on this site, we noted that there is currently a global multiplicity of strong initiatives in generating better data on learning outcomes, oriented at Sustainable Development Goal 4 (SDG 4). And, that’s only in the area of learning outcomes. If one adds other key aspects of SDG 4 and the global and thematic indicators, the number of initiatives is almost too large to catalogue. Even within the limited area of learning outcomes, some of the elements in the various initiatives create synergies, but others might create duplication or at least different proposed ways of doing things, if there is not a forum to clear lessons-learned and the results of different approaches. In our recent blog, we promised to present a systematic analysis that, using a matrix format, compares elements of the various initiatives. This, we hope, would be of use to the institutions involved and the funders asked to underwrite the various proposals.

In this blog we review and compare four initiatives, the Global Alliance to Monitor Learning (GAML also available in detail here), the Assessment for Learning (A4L) initiative, the learning assessment recommendations of the Commission on Financing of Global Education (chaired by Gordon Brown, the UN Special Envoy for Global Education). There are also interesting discussion documents from the background work done by the Center for Global Development for the Commission.

The following simplified matrix lays out some of the key actions or tasks that can be deemed important for producing robust, comparable indicators on learning, with specific reference to not just tracking, but moving towards actually achieving SDG4. It is important to note that the content of the matrix, for each initiative, is not a set of policy recommendations from the authors of this blog, but simply a listing of the ideas found in key initiatives. The matrix makes it clear where there is agreement but perhaps also some duplication, and where various initiatives have a unique contribution to make.

Our recommendations could be summarized as follows: a) make it clear what the areas of specialization of the various initiatives are; and, b) where the various initiatives coincide on recommending a certain point, make sure there is clarity as to the approach, or at least a mechanism for reaching clarity.

Specifically, it seems to us that the Commission takes a fundraising approach and proposes that it could raise the funding needed to support work towards measuring SDG 4 in general, and learning outcomes specifically. The Commission coincides with many of the policy recommendations of the other initiatives and cross-references GAML. It seems, also, that A4L has some detailed discussion for how one could make grants to countries and/or regional bodies, while also backing key ideas such as common metrics. It, or the approaches recommended, contain a discussion of financing mechanisms and grants that could be used to disburse funds gathered under the aegis of what the Commission recommends. And, finally, it seems to us that GAML has the most complete proposals for coordinating the overall effort and lining up the technical work needed to underpin the measurement the “brokerage,” convening, and commissioning of technical work on assessment, for instance. In that sense, a fairly clear division of labor could emerge. But making sure that the division of labor emerges will take some conscious work.

One way to delineate amongst these various initiatives would be for UIS to commission a one-off team of experts to validate the various approaches and suggest ways to specialize and develop each initiative’s comparative advantage, so as to maximize synergies and avoid duplication and confusion.

As a last point: sometimes global initiatives take a long time to jell, especially if coordination among various initiatives is necessary. But there are some urgent tasks to be done. We would suggest advancing on those urgent tasks under the aegis of UIS, until bigger issues can be resolved. This would require funding so as to put a “down payment” on the various tools that have to emerge.

Key actions for producing robust, comparable indicators on learning

Define comparable learning metrics or scales, foster their usage – Without some form of common scale or metrics, it is difficult to judge progress objectively. However, there are many possible approaches to comparability, requiring different degrees of consensus. On this score, the various initiatives agree on the importance of the issue, but only GAML (and associated documents) makes a case for a particular mechanism to resolve it. The language in some initiatives suggests some possible duplication. 

Set quality standards for assessment data. Standards for assessment systems capacity (like the World Bank’s SABER) – Even if the goal were to simply report on learning to the global community, strong national systems are key. But having strong national systems that can assist in actually fostering learning, rather than just reporting on it, is itself a strong priority. Strong systems for assuring the quality of data, and the quality of data-producing systems. Guidance on these issues is a global public good with large economies of scale.

Capacity building of national systems – This would be key, even under the minimalist assumption that indicators are only for reporting. But such minimalist assumptions are not the case. Indicators are to be used not just for tracking, but for policymaking to improve national performance and the SDGs.

Actual development of technical tools, most likely contracted out or in partnership with the assessment industry – Linking items or a new actual global assessment, data-analytical packages, etc. 

Allocate funding to countries for improving systems or to participate in regional/international assessments, indirect support to regional and international assessments – Some poorer countries may require an incentive to participate in regional/international assessments or to participate in linking their national assessments to a global metric or scale.

Direct support to regional assessment programmes and existing international assessments – Regional assessment programmes receive widespread support from countries and are therefore a logical way to think about globally-linkable assessment. Supporting such regional programmes could be made conditional on their being linkable to a global scale. No initiative discusses direct support to international assessments.

Direct (grant) support to development of national learning assessment systems – May be needed in poorer countries to stimulate progress towards assessment systems. Could be made partially results-based.

Coordination of the above and other comments – Clearly some degree of brokering and coordination of the above is needed, as these tasks are all somewhat dependent on each other. 

 Luis Crouch, Chief Technical Officer, International Development Group, RTI.  

 Silvia Montoya (@montoya_sil) is the Director of the UNESCO Institute for Statistics.

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,500 registered members worldwide and is free to join. Not a member? Join free here.