The Busan Discourse on Results, Effectiveness, Impact, and Value for Money: what do they Mean for Emerging Donors?
By Soyeun Kim, University of Leeds & Re-shaping Development Institute, S. Korea.
“Over 2,000 delegates are reviewing global progress in improving the impact and Value for Money of development aid” (emphasis added, official HLF4 website).
The preoccupation with results and Value for Money (VfM) has been increasingly visible among the traditional aid circle in recent years. With the economic/financial difficulties at home and the media frenzy on emerging powers (e.g. China’s resource-backed development projects in Africa), the combined considerations of economy, efficiency and effectiveness seem to have gained greater kudos. Indeed, as the quote above highlights, VfM’s firm presence in Busan HLF4 was highlighted by its frequent appearance in various documents/statements from building block sessions to side events. Yet, in the outcome document, VfM is only mentioned once while results 24 times, effectiveness 15 times, and impact 8 times. Indeed, this observation, although cursory, throws up various questions – particularly relating to those emerging donors including China, India and Brazil who have signed up for the Busan Partnership document. What would the result-based management (RBM) and the VfM mean for them, and how would they (or would they not) implement it? For example, Premier Wen Jiabao’s speech at the opening ceremony of the 4th FOCAC was titled “Deepen Result-Oriented Cooperation and Promote Common Development”. But curiously, despite its grand appearance in the title, the result agenda was not really present in Wen’s speech. We cannot help but wonder what Premier Wen meant by ‘result-oriented cooperation’ – does it differ from VfM and RBM debates among the traditional donors?
In an attempt to answer these questions, I critically reflect upon the recent debates (and development) in S. Korea on the RBM and the VfM. Despite its DAC membership, Korea still shares many similarities with other emerging donors. And simultaneously, the donor has begun putting effort towards a result-based ODA system. Thus, the case of Korea would provide some insight into how others may approach the agenda.
The first question concerns the ambiguity of the RBM and VfM concepts themselves – e.g. who defines results, what the intended results are for. The rhetoric of a ‘win-win’ and ‘common’ development outcome between the partners (therefore mutual benefits) can be easily found in the Korean aid policy discourse. Yet, the reality seems otherwise. It is often the result that the donor intends. For example, one of my colleagues in Korea confided in me with frustration about the way in which project evaluation criteria were dictated by their utility for advancing Korean companies’ overseas expansion. Often such objectives are ‘hidden’ away from public eyes – and are often structurally incorporated into the aid system itself. Such donor-driven VfM is also seen in the recent CNK (a Korean diamond mining company operating in Cameroon) scandal. In order to secure a mining concession (an intended result), some Korean government officials allegedly prioritised grant aid to Cameroon over other projects. Thus, structural inclusion of private sector initiatives and tying aid are often practised as one way to ensure the result and VfM.
The second point relates to too much (shifting) emphasis on results. Therefore more resource has been allocated to showcasing ‘effective’ results while the important process of learning seems rather overlooked in project implementation. This is not to say increased efforts on monitoring and evaluation are a bad thing. But the efforts have to be based on a more integrated approach. There are two cases I have come across showing some worrying signs. For example, one of the KOICA-run training sessions on the RBM used a flow chart that separated the project process from project result. This flow chart was then followed by a comment stating that “the focus of M[onitoring] & E[valuation] has been shifted from the process to result”. The recent emphasis on the RBM does not necessarily mean shifting away from the project process. The learning (via trial and error) during the project process forms important information/knowledge to share with the partners – which ultimately should serve as evidence to improve development impact.
Further, Korea’s fragmented aid structure hinders systematic knowledge sharing with the partner countries for better development result. For example, the Bangladesh government wanted to see successful KOICA funded grant aid (ICT) projects being further developed into EDCF ODA loan projects. However, KOICA’s post evaluation often only took place after two-three years from the project completion. Such fragmentation and ‘time gap’ hamper not only knowledge sharing between donor and partner countries – but also more effective cooperation between the donor agencies (KOICA and EDCF). In addition to this, there is an added complication. The project management company (the contractor) does not fully share the acquired knowledge during the project as the information is regarded as the company’s intellectual property.
The third question concerns the highly technical and expert-led process of management and evaluation. RBM is necessarily about measuring, monitoring and evaluation. Yet, these activities are not fully integrated into the project cycle. Moreover, technical capacity for RBM is not yet established nor institutionalised. Korean aid agencies have relatively recently begun to recruit and to ‘nurture’ in-house experts. Therefore, often commissioning evaluation projects means that contractors have to set up a set of evaluation guidelines and to simultaneously evaluate projects. Without a solid basis for measuring, monitoring and evaluating results, it is indeed difficult to expect effective implementation of the RBM and VfM.
But some Korean aid officials are also cautious of blindly following the international aid norms while emphasising the importance of realising the spirit of Paris Declaration. Indeed, as my KoFID colleagues emphasised “[a]fter all, measure[ing] development results should be seen as a tool and not as a goal”.
Soyeun Kim is a Lecturer/ Director of Online MA Programme in Asia Pacific Studies, Department of East Asian Studies, University of Leeds; and, affiliated to the Reshaping Development Institute, South Korea. Email: s.s.kim@leeds.ac.uk
An earlier version of this piece appeared in NORRAG NEWS Value for Money in International Education: A New World of Results, Impacts and Outcomes, No.47, April 2012, pp. 45-47.
Reblogged this on mande.weitzenegger.de.
I have been unable to access Mande Weitzenegger’s response.
Most interesting. Valuable, even. But scarcely surprising. Value for Money being mentioned only once in a document presenting the outcomes of an event ostensibly focussed on VfM is a fairly standard case study. Participating people have their status-driven agendas, donors and various stakeholders their conflicting institutional goals.
I have been examining some eighty international development interventions that commenced in and around 2006 in relation to their Logical Frameworks. Annexed to financing agreements, they constitute elements of the contracts between funding agencies and those who would represent the beneficiaries. Undoubtedly the indicators at the results level should depict the planned outcomes and offer a means of assessing ‘outputs’ against ‘inputs’: all should be there in the respective LogFrame cells.
Needless to say, this is seldom the case. The indicators are very seldom ‘objectively verifiable’, often merely suggesting the dimension along which achievement might be measured (e.g. ‘more graduates’ or ‘better examination results’). There is little evidence that costs have been based on anything beyond level of the grant or loan available and the estimated prices of the various ingredients: no-one seems to be considering likely intended outcomes in relation to the levels of financing. Consequently, only a handful of the LogFrames studied offer anything meaningful to those seriously interested in Results Based Management or Monitoring. The opportunity to specify and then determine the consequences of particular development aid investments has long been there but has seldom been taken.
Adding the notion of ‘value’ plunges the analyses into dangerous confusion. All too frequently, the temptation by non-educators has been to use crude estimates of (along with even cruder assumptions regarding) incremental remuneration (‘X additional years of primary schooling brings about Y additional duly discounted lifetime earnings’). Assuredly there is much to be gained by comparing the costs of training with the benefits of better-skilled workers. The objective is reasonably clear, the desired outcome more or less measurable, and the economic benefits and monetary costs may with care be compared, all to the enlightenment of donors and the reassurance of electorates.
But education isn’t like that. Engendering a love of learning and underwriting enthusiasms and generally having fun in understanding and responding to our world is self-evidently good in a moral sense but in no way constitutes an economic good. Bringing the marketplace into the schoolroom devalues the invaluable, transforming a wonderful universal right into a mundane tradable commodity. Perhaps subconsciously the scribes of Busan recognised the underlying contradiction in attempting VfM analyses where the outcome facilitated (‘education’) is beyond valuation.
Pingback : Chalkboards and Cheeseboards – Resisting the Workplace’s Colonisation of the Schoolroom | NORRAG NEWSBite
Reblogged this on mande.weitzenegger.de.
I have been unable to access Mande Weitzenegger’s response.
Most interesting. Valuable, even. But scarcely surprising. Value for Money being mentioned only once in a document presenting the outcomes of an event ostensibly focussed on VfM is a fairly standard case study. Participating people have their status-driven agendas, donors and various stakeholders their conflicting institutional goals.
I have been examining some eighty international development interventions that commenced in and around 2006 in relation to their Logical Frameworks. Annexed to financing agreements, they constitute elements of the contracts between funding agencies and those who would represent the beneficiaries. Undoubtedly the indicators at the results level should depict the planned outcomes and offer a means of assessing ‘outputs’ against ‘inputs’: all should be there in the respective LogFrame cells.
Needless to say, this is seldom the case. The indicators are very seldom ‘objectively verifiable’, often merely suggesting the dimension along which achievement might be measured (e.g. ‘more graduates’ or ‘better examination results’). There is little evidence that costs have been based on anything beyond level of the grant or loan available and the estimated prices of the various ingredients: no-one seems to be considering likely intended outcomes in relation to the levels of financing. Consequently, only a handful of the LogFrames studied offer anything meaningful to those seriously interested in Results Based Management or Monitoring. The opportunity to specify and then determine the consequences of particular development aid investments has long been there but has seldom been taken.
Adding the notion of ‘value’ plunges the analyses into dangerous confusion. All too frequently, the temptation by non-educators has been to use crude estimates of (along with even cruder assumptions regarding) incremental remuneration (‘X additional years of primary schooling brings about Y additional duly discounted lifetime earnings’). Assuredly there is much to be gained by comparing the costs of training with the benefits of better-skilled workers. The objective is reasonably clear, the desired outcome more or less measurable, and the economic benefits and monetary costs may with care be compared, all to the enlightenment of donors and the reassurance of electorates.
But education isn’t like that. Engendering a love of learning and underwriting enthusiasms and generally having fun in understanding and responding to our world is self-evidently good in a moral sense but in no way constitutes an economic good. Bringing the marketplace into the schoolroom devalues the invaluable, transforming a wonderful universal right into a mundane tradable commodity. Perhaps subconsciously the scribes of Busan recognised the underlying contradiction in attempting VfM analyses where the outcome facilitated (‘education’) is beyond valuation.