Improving the monitoring and evaluation of facilities in the Australian aid program

Scott Bayley

In Australia, monitoring and evaluation guidance provides a reasonably comprehensive approach for evaluating individual aid investments, however, providing detailed insights into the facilities is still difficult.

International development is a highly challenging sector with a range of different approaches used to deliver sustainable impacts for those most in need. Australia has been making use of the ‘facility’ approach to deliver aid since the 1990s. This is particularly the case when the operational environment is highly complex, and a degree of flexibility is required.

Flexible in nature, facilities may work in single or multiple sectors, and can have various purposes, such as achieving a specific development outcome, undertaking administrative functions, or supporting wider diplomatic work. Facilities are normally implemented by a managing contractor or multilateral partner. The increasing use of facilities is a logical response to the need for flexibility in the Australian aid program which is operating within significant staffing constraints, but this trend is not without risk.

Exploring challenges and opportunities

In the right context facilities provide a useful approach for meeting aid policy objectives and achieving efficiencies. Their overwhelming advantage is their unique flexibility, which allows shifts in program resources in line with evolving priorities, context, and experience – without the need for contractual amendments.

However, facilities also pose their own special set of challenges. As the flexibility of an investment increases, so does the complexity of its management, and the risk of failure. Facility managers need to juggle contractual considerations, stakeholder engagement, relationship building, and technical policy challenges while simultaneously implementing multiple streams of program activities.

Furthermore, unless there is a clear definition of the facility’s desired outcomes, coupled with criteria to identify which activities should be funded, there is a risk of strategic drift – which can result in creating a fragmented set of activities that are then difficult to monitor, evaluate, and report against.

Raising monitoring and evaluation ambitions

The effectiveness of aid facilities and the adequacy of their monitoring and evaluation practices has been the subject of some debate, controversy, and criticism.

In Australia, the Department of Foreign Affairs and Trade’s 2017 monitoring and evaluation (M&E) guidance provides a reasonably comprehensive approach for evaluating individual aid investments, however, providing detailed insights into the M&E of facilities is still difficult. It’s important to consider why that is, and what can be done about it.

The answer to this might lie in the overarching way the term ‘facility’ is used in practice. It covers at least four different types of interventions, each with unique requirements for monitoring and evaluation. To understand where challenges occur, it’s crucial to differentiate between different types of facilities.

Type 1: Support facilities

These facilities focus on the provision of administrative support to the Department of Foreign Affairs and Trade rather than on the achievement of development outcomes per se. These support functions include procurement, recruitment, and placing of advisors in the field. By providing this support, Australian government officials are freed up to focus on strategic matters and managing relationships with government partners.

With this type of facility, monitoring and evaluation requirements are very basic, including service delivery milestones and quality standards for deliverables.

Type 2: Grants facilities

These facilities usually have high level but vague development outcomes. In practice, these facilities focus on undertaking individual projects at the request of partner governments. The facility’s success is judged by the performance of individual projects, and managing contractors are accountable for the success of projects but not high-level development outcomes.

Evaluating the success of these facilities requires a program logic model and M&E framework for the facility as a whole, coupled with more detailed logics and frameworks for individual projects.

Type 3: Consolidated facilities

These facilities are an amalgamation of several separate investments that do not necessarily share a common development outcome. Consolidated facilities are intended to achieve outcomes for each of the stand-alone components plus administrative efficiencies and improved coordination. The facility is largely preprogramed, and the managing contractor is responsible for implementation and the fine-tuning of strategy to achieve outcomes.

Each separate component of the facility requires its own program logic model and comprehensive M&E framework. In addition, the efficiency of the overall facility needs to be examined via evaluation studies. It is worth considering the potential benefits of having an independent entity review the M&E and reporting practices of the managing contractor. If the separate components are intended to jointly contribute to a common higher-level outcome, then an overall program logic model and M&E framework are also required.

Type 4: Adaptive facilities

Adaptive facilities recognise that the key constraint to development change is rarely a lack of technical knowledge or capacity but the constellation of political incentives. An adaptive facility has a small number clearly stated high level development outcomes with a large pool of unallocated funds. The activities and outputs to be produced are not specified up front but are selected in response to local political and contextual considerations. The adaptive changes are not simply to the facility’s choice of activities, but in the underlying logic of how to make change happen. Contractors are held accountable for achieving the facility’s development outcomes. Governance mechanisms are very important but difficult to operationalise in these types of facilities.

Adaptive facilities require a high-level program logic model and M&E framework articulating the desired end of facility and intermediate outcomes. Each emerging stream of work will also require a basic program logic model and M&E framework that links activities and outputs to the identified intermediate outcomes. Adaptive facilities require cycles of rapid feedback similar to action research, reflection workshops to identify and share lessons, coupled with strategy testing. In addition to monitoring implementation progress and development results, the performance of the facility itself needs to be assessed.

Going forward

At present, much is claimed about the presumed benefits arising from the choice of the facility approach, including improved strategic focus, value for money, reduced administrative costs, and program responsiveness. Unfortunately, the necessary data is rarely collected to systematically assess whether any of this is actually happening.

To ensure implemented facilities deliver on their purposes and development outcomes, it’s vital to clarify the role facilities play in the Australian aid program. This in turn will help shape their effectiveness, guide progress in the future, and ensure aid reaches those most in need.