Even when internal evaluation points out failure, do aid agencies hold anyone responsible or change agency practices? It is hard to find out from a review of the World Bank’s evaluation Web site. The OED in 2004 indicated how eight “influential evaluations” influenced actions of the borrower in thirty-two different ways, but mentioned only two instances of affecting behavior within the World Bank itself (one of them for the worse).19
The anthropologist James Ferguson offers a rare detailed case study of an aid project by an independent outsider. A Canadian International Development Agency/World Bank project was to help farmers in the mountains of Lesotho (the Thaba-Tseka region) to gain access to markets and develop modern methods of livestock management and crop production. The project would lend Western expertise at livestock management and cash-crop production and would build roads to enable beneficiaries to carry their products to market.
The only problem was that the beneficiaries were not that interested in farming, since they mainly were migrants working in South Africa. They already had access to markets, where they had long since learned that cash-crop production was not competitive given the region’s poor agricultural conditions. The Canadian/World Bank Thaba-Tseka project sought to improve livestock and food production. The project would improve livestock production by scientific range management, dividing the land into eight controlled grazing areas, with grazing associations to control overgrazing through resting and rotation of livestock. The project designers promised that scientific range management would eventually allow the land to support twice its current animal population, with each animal becoming 20 percent heavier. The only catch was that the project had no legal authority to restrict access to grazing, as land in Lesotho is publicly owned and livestock owners are free to graze the open range. Scientific range management didn’t happen.20
The project promised to increase food yields by 300 percent. Instead, a pilot potato project had losses due to bad weather, disease, and mismanagement. Other cash-crop experiments fell victim to the area’s “killing frosts and hail, and erratic and infrequent rainfall.” The project managers complained that the local people were “defeatist” and didn’t “think of themselves as farmers.” Perhaps the locals didn’t consider themselves farmers because they were not farmers—they were migrant workers in South African mines.21
The main accomplishment of the project was the building of roads that brought South African lorries carrying grain into the region (driving the few existing local farmers out of business).
Aid agency watchers should be tough on such disasters, if only with the aim of strengthening the accountability lobby in foreign aid, so we can shift power from Planners to Searchers. The way forward is politically difficult: truly independent scientific evaluation of specific aid efforts; not overall sweeping evaluations of a whole nationwide development program, but specific and continuous evaluation of particular interventions from which agencies can learn. Only outside political pressure on aid agencies is likely to create the incentives to do these evaluations. A World Bank study of evaluation in 2000 began with the confession, “Despite the billions of dollars spent on development assistance each year, there is still very little known about the actual impact of projects on the poor.22
After years of pressure, the IMF created an Independent Evaluation Office in 2001. The World Bank in 2004 laudably created a Development Impact Evaluation Task Force. The task force will use the randomized controlled trial methodology discussed in chapter 2 to assess the impact of selected interventions on the intended beneficiaries. The task force has started two dozen new evaluations in five areas (conditional cash transfers in low-income countries; school-based management; contract teachers; use of information as an accountability tool for schools; and slum upgrading programs). It remains to be seen if the evaluation results change the incentives to do effective programs in the operational side of the World Bank.
Evaluation—with consequences for a bad evaluation—is one of the keys to accountability of aid. You evaluate what we bureaucrats accomplish, and you hold us responsible for it. Accountability for aid would transfer power from Planners to Searchers—maybe then somebody could figure out how to keep potholes fixed in Tanzania.
Participation Through Planning?
If one problem in foreign aid is that the poor have little power to hold anyone accountable for meeting their needs, the World Bank and the IMF to their credit now show some awareness of this problem. They seek some role for the poor’s choices. They have recently put increased emphasis on “participation by the poor.” It is good that there is some awareness of the feedback problem with the poor.
However, it just shows how stubborn bureaucratic incentives are that the chosen vehicle for bottom-up participation is a detailed central government plan (the PRSP, Poverty Reduction Strategy Paper, already mentioned). The multivolume PRSP Sourcebook of the World Bank suggests some pretty detailed planning—the PRSP needs to include a medium-term expenditure framework (MTEF) in which:
The sector ministries prepare medium-term strategic plans that set out the sector’s key objectives, together with their associated outcomes, outputs, and expenditure forecasts (within the limits agreed upon by the Cabinet). These plans should consider the costs of both ongoing and new programs. Ideally, spending should be presented by program and spending category with financing needs for salaries, operations and maintenance, and investment clearly distinguished.23
Alas, even when we visualize the poor participating, we can’t give up our central planning. Such planning inevitably implies giving more power to the Planners at the top, not less. The last thing poor countries need for greater democratic accountability is a plan that strengthens already strong authoritarian officials.
The officials who talk about “participation” and “local ownership” can’t seem to let themselves shift power to the locals—the bureaucratic incentives against it are too strong. The African founder of a private school where he lives once told me about the frustration in dealing with Planners who paradoxically seek “participation.” The founder had started a high-quality private school in Africa, with many scholarships for poor children. He started it with his own money, but sought to expand activities and scholarships further by getting outside donor funding. He approached one of the world’s largest official aid agencies with a proposal for funding. They turned down his proposal. The founder asked them why. They wouldn’t tell him, saying that he had to demonstrate “local ownership” of the project by coming up with his own ideas without regard to what they would approve or reject. He lost his temper, saying that he could waste a lot of time submitting proposals to try to figure out what they would approve, or they could just have a frank discussion of what their criteria for project approval were. Finally, the aid agency official told him that his proposed pupil-teacher ratios were too low. Their planning incentives were to show as many students as possible reached with an aid grant, and they suggested a pupil-teacher ratio of fifty to one. This contradicted the founder’s objective of fostering high-quality education in Africa, and he told them to get lost.
The appeal of planning to aid visionaries is so strong that they map out top-down plans at the same time that they emphasize “local ownership” and emphatically deny that they favor top-down planning. To supplement the above example, consider the following explanation by the UN Millennium Project of how it goes about implementing the MDGs (which it describes as “open and consultative, involving all key stakeholders”).24 The head of the UN Millennium Project denied to me in person that this is a top-down plan, so I quote from it at length so you can judge for yourself:
In each of these countries, the Project and local research partners built upon international best practices to identify…the input targets that would be needed for the country to achieve the MDGs by 2015. These estimates cover hundreds of interventions…that need to be provided to meet the Goals…. The second stage of the planning process will be for each country to develop a long-term
(10–12 year) framework for action for achieving the MDGs, building upon the results of the MDG needs assessment…. This MDG framework should include a policy and public sector management framework to scale up public spending and services, as well as a broadly defined financing strategy to underpin the plan. The third stage of the planning process will be for each country to construct its medium term (3–5 year) poverty reduction strategy (PRS) and, where appropriate, its Poverty Reduction Strategy Paper(PRSP) based on the long term MDG plan…and should be attached to a Medium Term Expenditure Framework (MTEF)…. Fourth, both the 10-year framework and three-year PRS should include a public sector management strategy…. Bringing together a wide variety of inputs from expert resources, the Millennium Project secretariat has been coordinating a multi-step process to develop a methodology for country-level MDG needs assessments.
Far from promoting “participation,” planning patronizes and diminishes the poor, who have little voice to say what they want and need. Unfortunately, decades of participation rhetoric have not changed the balance of power in foreign aid. At some point the donors just have to trust the recipients to be self-reliant enough to follow their own interests, to seize the opportunities created by aid. Certain kinds of aid can create opportunities for the poor and maximize payoff by letting the poor self-select, that is, show willingness to exert effort toward making the best of the opportunity.
I once got a big boost from such an aid program. The National Science Foundation (NSF) gave me a three-year fellowship to get an economics Ph.D., with tuition paid plus a stipend. Without this, I couldn’t have gotten a Ph.D. The NSF did not send me missions of NSF staff to look over my shoulder as I was studying. They did not ask me to attend meetings of “stakeholders” to write a Ph.D. Promotion Strategy Paper. They had no conditions at all other than my enrolling in school and not flunking out. They allowed me the dignity of self-reliance. I could have wasted the opportunity by skipping my courses and spending all my time going to Woody Allen film festivals. But the NSF trusted the choices of the Ph.D. candidates, who the NSF thought would act in their own best interests with the opportunity given to them to get a Ph.D. The NSF could have this confidence because the applicants for the fellowships were self-selecting: only those who are willing to do the hard work of getting a Ph.D. apply.
Similarly, scholarships or matching grants for poor individuals or entrepreneurs could promote true “participation” in which the poor make their own choices. If you really want to put the poor in the driver’s seat, aren’t there any ways to do so directly? Could you give many more scholarships to poor students? Could you give matching grants to poor entrepreneurs who put their own money at stake to start a new business? Could you have village elections that select (or reject) aid projects? Could you give the poor “aid vouchers” that they could spend on aid agency services of their choice? None of these are easy answers, and all have pitfalls, but new thinking is needed. The final chapter explores these ideas further.
As for aid agency staff, a more effective way to listen to the poor than costing out a Big Plan would be to have aid agency specialists spend time learning about a particular sector in a particular region. In other words, have aid agency staff be sufficiently specialized to be effective Searchers. Aid bureaucracies tend to do the opposite: they reassign staff frequently across countries and across sectors, producing generalists who are much better at producing Big Plans than local solutions. They opt for universality rather than specificity, for worldwide “best practices” rather than what works in each locale. As James C. Scott points out, Planners are impatient with local peculiarities: “the lack of context and particularity is not an oversight; it is the necessary first premise of any large scale planning exercise.25 To change practice, we must persuade the aid agencies to give up their utopian planning in favor of piecemeal intervention. This is not easy when the power and prestige of the existing aid agency managers may depend on keeping the planning approach.
Prior Lives
One characteristic of private markets is that they foster innovation: new products, new business techniques, new financial instruments, in short, new and better ways of doing things. Searchers learn not to repeat prior mistakes; Planners with no feedback keep doing the same failed plans.
In the movie Groundhog Day, a television reporter played by Bill Murray is condemned to endlessly repeat the day on which he must report on whether a groundhog sees its shadow. The aid agencies seem to be stuck in a similar lame cycle of repeating themselves, as table 3 shows. Bill Murray escapes his torment only when he is able to resolve his relationship with his beautiful producer, played by Andie MacDowell.
The lack of historical memory in the aid community inhibits people from learning from mistakes. Moreover, the unchanging approach to many of these desirable objectives shows again that aid agencies keep throwing in more and more resources to try to reach a predetermined, although unattainable, goal.
Differences Among Aid Bureaucracies
I took most of my examples of aid agency bureaucracy from the World Bank because of my personal familiarity with its operations. However, there are differences between the international organizations, which give some insight into when aid succeeds and fails. The International Monetary Fund is somewhat more successful in achieving its narrow goals, although there have been failures (see next chapter). The World Bank is more prone to meaningless frameworks and goal proliferation, but it is actually among the better aid agencies, as some of the positive examples I have given illustrate.
There are also differences between national aid agencies. USAID brazenly states its objective is to further “the foreign policy goals of the United States.26 The United Kingdom’s aid agency, the Department for International Development (DFID), says its objective is helping the world’s poor and it is more committed to independent evaluation of its projects than most other aid agencies.
Going to the other extreme, I have just read some UN documents on what they label “The Open-Ended Ad-Hoc Working Group on Integrated and Coordinated Implementation of and Follow-up to the Outcomes of the Major United Nations Conferences and Summits in the Economic and Social Fields.” This open-ended ad hoc working group faces some challenges, as it coordinates the follow-up to nine reports on country-level coordination, four reports on the PRSPs, eleven reports on the Bretton Woods Institutions (aka the World Bank and IMF), eleven reports on the MDGs, the annual reports of the Administrative Council on Coordination, reports from the five regional commissions of the UN, reports from five other specialized UN agencies, and the follow-up to eighteen UN world conferences.27 The working group labels the background papers for its efforts “non-papers.” (I am not making this up; see http://www.un.org/esa/coordination/ecosoc/wgga/nonpapers.htm.) The first non-paper says that the work of the open-ended ad hoc working group
should be consistent with the provisions of resolution 50/227 and the follow-up mechanisms decided upon by the respective United Nations conferences and summits and should respect the interlinked nature of their outcomes as well as the thematic unity of each conference…. [C]ross-sectoral thematic issues for further consideration throughout the existing structure should be decided upon at the intergovernmental level and should focus on implementation, bearing in mind that the process of integrated and coordinated follow-up to the outcomes of the United Nations conferences and summits in the economic, social and related fields should be fair and balanced and should respect the principle of multilateralism and the principles contained in the Charter of the United Nations.
To be fair, there is incomprehensible language also in private-sector documents, such as investment prospectuses or engineering designs. The difference is that in private-sector documents, the jargon actually has some meaning to specialists. In UN documents, the jargon has no substantive content for anyone.
To help “focus the work of the UN,” non-paper one sets out twelve “cross-cutting themes” previously set out by the Economic and Social Council (for example, theme
twelve is “participation, democracy, human rights, accountability and partnership with major groups and non-governmental organizations”). Non-paper one then goes on to list eleven “areas calling for greater attention” identified by the secretary general (e.g., “greater coherence in United Nations action in support of Africa’s development”). Finally, the non-paper wraps up with ten new “cross-cutting themes” that emerged from recent UN conferences, many with multiple sub-themes. Non-paper three lists all the UN non-conferences on which it is based.
Just as other aid agencies like to produce observable output, the UN holds big world summits. UN officials are travel weary from attending all these utopian summits: Environment and Development (three summits), World Food Summit (two), World Summit for Children (two), World Assembly on Ageing (two), World Conference Against Racism, International Conference on Financing for Development, UN General Assembly Special Session on AIDS, UN Conference on Human Settlements (three), UN Conference on the Least Developed Countries (three), Millennium Summit, World Summit for Social Development (two), World Conference on Women (five), Global Conference on Sustainable Development of Small Island Developing States (two), International Conference on Population and Development (two), and the World Conference on Human Rights. Officials who participate in these summits mean well, but the reason these summits are repeated so often is that previous summits haven’t accomplished their goals.
Another sign of the ineffectiveness of the UN is its dubious economic analysis, in contrast with the higher-quality analysis cited above at the IMF and the World Bank. To take one of the worst examples, the analysis by the UN Commission on Trade and Development (UNCTAD) of the predicament of the least-developed countries (LDCs) is the Rube Goldberg diagram in figure 23. Poor countries may not be in a poverty trap, but UNCTAD itself seems to be trapped in some kind of intellectual maze, with arrows flying in all directions.
B000QJLQXU EBOK Page 21