A new perspective on aid delivery

Richard Curtain

It is rare to get a real insight into the differences within and between agencies in how aid is delivered. A new book, Navigation by Judgment: Why and When Top-down Management of Foreign Aid Doesn’t Work by Dan Honig, does this.

For most of us who have worked in aid-funded programs, it is a mystifying and frustrating business, especially when you have had the experience of working for different aid agencies as I have had (UNICEF, AusAid, USAID and the German aid agency (GIZ), as well as multilaterals such as the ADB).

After reading a highly favourable review by Lant Pritchett on the Building State Capability Blog, I purchased the book, not least because I noted that Dan had worked on a youth-related project in Timor-Leste, as I had done.

The book’s insight is not a new one, but what is new is that it is built on two solid evidential bases: an econometric analysis of a huge database of delivery outcomes of aid projects and eight case studies in two countries of the differences between USAID and DFID (UK aid) modes of aid delivery.

Honig’s main finding, with some qualifications, is that program outcomes are better where aid agencies allow and encourage their direct staff or contracted staff to have greater latitude to exercise ‘judgement’. I prefer to use the Australian spelling because the word ‘judgment’ in an Australian context can refer to legal decisions, the opposite of what Honig is referring to.

The value of navigation by judgement applies especially when the operating context is unpredictable and where performance is difficult to measure. Judgement refers to the scope that staff have to make their own decisions within the broad parameters set by the program design. This is in contrast to top-down management, which in the words of the book’s fly leaf is problematic: ‘Tight controls and a focus on reaching pre-set measurable targets often prevent front-line workers from using skill, local knowledge and creativity to solve problems in ways that maximise the impact of foreign aid’.

The econometric analysis is of evaluations of over 14,000 projects from nine international development organisations. Spoiler alert: no Australian aid projects are included. The analysis uses self-evaluated project outcomes as a measure of success. If you want to see Dan present the econometric results and more else besides, this video is well worth viewing.

In short, Honig finds that navigation by judgement is most helpful where environment predictability is lowest. The other main econometric finding is that where project performance is more difficult to verify externally, navigation by judgement has a strong relationship with project success.

A related finding is that projects based on top-down management are not as effective as those which allow more navigation by judgement even where project performance can be better measured.

However, the most interesting part of the book for me were the case studies of the eight USAID and DFID projects in Liberia and South Africa. A constant theme for the USAID case studies is the use of quantified measures of outputs, which is often at the expense of any assessment of project effectiveness. One example is USAID funding of training workshops on managing budgets for local government in South Africa. This mode of delivery was adopted because their deliverables could be measured, despite strong indications that the training was having little or no effect. In contrast, DFID put long-term advisers into the same municipalities with a general brief to improve their budgeting performance, which had much more impact.

However, the most interesting part of the book for me were the case studies of the eight USAID and DFID projects in Liberia and South Africa. A constant theme for the USAID case studies is the use of quantified measures of outputs, which is often at the expense of any assessment of project effectiveness. One example is USAID funding of training workshops on managing budgets for local government in South Africa. This mode of delivery was adopted because their deliverables could be measured, despite strong indications that the training was having little or no effect. In contrast, DFID put long-term advisers into the same municipalities with a general brief to improve their budgeting performance, which had much more impact.

In only one case, a USAID approach focusing on output measurement was highly successful. This approach worked well for reducing mother-child HIV transmission rates through antiretroviral injections. By contrast, in the same setting, the more open-ended approach of DFID was unsuccessful due to a lack of focus on delivering a simple output.

The conclusion drawn from the case studies is that navigation by judgement works better where the operating environment is more unpredictable.

Why the different approaches of the two aid agencies? Honig identifies the authorising environment and the different type of reporting required as the major factors explaining the choice of navigation strategy. USAID is an agency within the State Department and does not have its own political leader. USAID funding is uncertain from year to year and tightly controlled by the US Congress, which in many cases pre-specifies what the funding is for. DFID, in contrast, is a separate ministry led by a cabinet-level minister. DFID’s budget is more stable with a high level of funding (0.70% of GNI) guaranteed by legislation and met in 2016 and 2017.

The limitations of the analysis is what it does not cover. Its origin as a PhD thesis explains its rigorous but narrow focus. Despite a large part of the database of projects being from the World Bank open data source, there is no analysis of World Bank projects and their operating context. The World Bank does not have the same accountability to one political master, so how do its projects perform according to the type of navigation strategies used? Is there evidence that US political demands for narrow accountability also shape how the World Bank’s projects perform?

The large project performance database compiled and coded by Honig for this study is open access. The challenge is to assemble a similar database of Australian aid-funded projects. Further analysis using the analytical framework proposed in this book could enable Australian program designers to work out where a more flexible and more open-ended design is likely to be more successful, based on information on the unpredictability of the setting and the difficulty of measuring the outcomes to be achieved in simple ways.

Richard Curtain is a Research Fellow with the Development Policy Centre. His research focuses on international labour mobility in the Pacific and Timor-Leste.