Randomistas

by Andrew Leigh

  In the Indian state of Andhra Pradesh, Muralidharan worked with the government to randomise the rollout of biometrically identified smartcards.31 From 2010 onwards, India began issuing its citizens with smartcards that were encoded with their fingerprints and photograph, and linked to a newly established bank account. Muralidharan and his co-authors found that the program had precisely the effects intended: more eligible citizens accessed the program, payments were received more quickly, and less money was siphoned off by corrupt officials. In each case, the study’s estimates are extremely precise, since the experiment comprised a whopping 19 million people – more than the entire population of Chile.

  *

  Ill-health is perhaps the most tangible marker of global poverty. Just as new drugs are routinely tested through randomised trials, public health researchers are increasingly using randomised trials to create a safer environment in developing nations. One disease that has received significant attention is malaria, one of the world’s nastiest killers. As the Anopheles mosquito feeds from an infected person, it takes in the parasite. When it bites a new victim, they may contract malaria.32 Malaria claims the life of a young child every two minutes, which means that five children have died of the disease since you started reading this chapter.33

  Because mosquitos are most active at night, a simple solution is to sleep under a bed net. But the challenge for aid workers has been how best to increase the uptake of bed nets. Some people argued that if you simply give out free bed nets, recipients won’t value them. New York University’s William Easterly warned that free nets ‘are often diverted to the black market, become out of stock in health clinics, or wind up being used as fishing nets or wedding veils’.34 So from 2000 to 2005, the World Health Organization focused on subsidising the cost.35 Nets were distributed at a cost of about US$2 to $3, with part of the money retained by the local salesperson.

  As it turns out, economic theory isn’t much help in deciding how to set the right price to maximise the take-up of bed nets. It could be that people who are most willing to pay for a product are those who need it the most. Perhaps paying for a product is a signal of quality, which leads the buyer to make more use of it. There’s also a theory about the ‘power of free’, which sees people consume more when the price falls to zero. You probably see some of these theories at work in your own life. If you overeat at the buffet, you’ve fallen victim to the power of free. But if you open the most expensive bottle of wine at the start of the night, you’re using price as a signal of quality. Like I said, theory doesn’t provide a decisive answer.

  It took a series of randomised experiments to answer the question.36 Offer people a free bed net, and almost everyone takes one. Raise the price of a bed net from zero to just 60 cents, and the take-up rate falls by two-thirds. When the researchers visited the villages, those who received a free bed net were equally likely to be sleeping under it as those who had paid a subsidised price. Free distribution also did a better job of protecting those people most at risk of malaria.

  As the results of the randomised trials became clear, the World Health Organization switched its policy to favour free distribution of bed nets.37 In sub-Saharan Africa, two out of three children now sleep under an insecticide-treated net, compared with just one in fifty in the year 2000. Widespread take-up of bed nets is saving hundreds of thousands of children’s lives annually – and randomised trials have shown how best to achieve the goal.

  Getting the prices right is a central challenge in development economics. Further randomised trials have explored the question of whether free is best for a number of other interventions, including deworming tablets for children and water disinfectant for families. In both cases, the best price turns out to be zero, with coverage rates increasing substantially when households pay nothing.38

  Indeed, when it comes to vaccination, the ideal price may even be negative. A trial in India showed that families were more likely to get their children vaccinated when they received food and metal plates worth about US$3.39 With more children being vaccinated, health centres could cut the cost per child – even after subtracting the cost of the products they gave away.

  But giveaways aren’t always successful. After a visit to Argentina, businessman Blake Mycoskie decided he wanted to do something about the lack of decent footwear in developing nations. A talented entrepreneur, Mycoskie had founded and sold four companies by his thirtieth birthday. Now he was affected by the poverty he saw in villages outside Buenos Aires: ‘I knew somewhere in the back of my mind that poor children around the world often went barefoot, but now, for the first time, I saw the real effects of being shoeless: the blisters, the sores, the infections.’40

  To provide shoes to those children, Mycoskie founded ‘Shoes for Better Tomorrows’, which was soon shortened to TOMS. The company made its customers a one-for-one promise: buy a pair of shoes, and TOMS will donate a pair to a needy child. Since 2006, TOMS has given away 60 million pairs of shoes.41 It has inspired others companies to apply the one-for-one model to eyeglasses, soccer balls, condoms, toothbrushes, flashlights and medical scrubs.42

  Six years in, Mycoskie and his team wanted to know what impact TOMS was having, so they made the brave decision to let economists randomise shoe distribution across eighteen communities in El Salvador. The study showed that the canvas loafers didn’t go to waste: most children wore their new shoes most of the time.43 But the children’s health wasn’t any better, as the TOMS shoes were generally replacing older footwear. Free shoes didn’t improve children’s self-esteem, but did make them feel more dependent on outsiders.

  These were shocking findings. Corporate philanthropy wasn’t an add-on for TOMS – it was the firm’s founding credo. Now a randomised trial showed that among recipients in El Salvador, free shoes weren’t doing much to improve child outcomes, and may even have been fostering a sense of dependency. Yet rather than trying to discredit the evaluation, TOMS responded promptly. As lead researcher Bruce Wydick wrote:

  TOMS is perhaps the most nimble organization any of us has ever worked with, an organization that truly cares about what it is doing, seeks evidence-based results on its program, and is committed to re-orienting the nature of its intervention in order to maximize results. In response to children saying that the canvas loafer isn’t their first choice, they now often give away sports shoes . . . In response to the dependency issue, they now want to pursue giving the shoes to kids as rewards for school attendance and performance . . . Never once as researchers did we feel pressure to hide results that could shed an unfavourable light on the company. By our agreement, they could have chosen to remain anonymous on the study; they didn’t . . . For every TOMS, there are many more, both secular and faith-based, who are reticent to have the impacts of the program scrutinized carefully by outside researchers. Instead of demonstrating the effectiveness of their program on the poor to potential donors, many organizations today continue to avoid rigorous evaluation, relying on marketing clichés and feel-good giving to bring in donor cash. TOMS is different, and we applaud them for their transparency and commitment to evidence-based action among the poor.44

  Ultimately, the TOMS randomised trial doesn’t look like a failure at all. Blake Mycoskie’s goal in establishing the firm was to improve the health of poor children. The company evaluated its approach. It didn’t work. So it changed tack. If all donors were as big-hearted and open-minded as TOMS, there would be less poverty in the world.

  Other experiments have looked at ways to reduce the road toll. If you want to get a vivid sense of the dangers of driving, open up the World Health Organization’s page on the topic. Immediately a timer starts counting down with the caption ‘A road user will die in . . .’45 Every twenty-five seconds, the timer reaches zero, another number is added to the death toll, and the counter resets. Annually, that’s 1.25 million traffic deaths.

  Across the globe, Africa has the highest traffic fatality rates of any continent. Part of the reason is dangerous driving. In
Kenya, where travel by matatu or mini-bus is common, the average passenger reports experiencing a life-threatening event in a mini-bus every three months.

  To encourage passengers to speak up when drivers are behaving recklessly, a team of researchers worked with matatu companies to place stickers in the buses with slogans such as ‘A careless matatu driver is your wake-up call! Stand up. Speak up.’46 In case anyone missed the message, the sticker also included a photograph of a severed foot. Comparing 1000 randomly selected matatus with stickers with 1000 that did not have stickers, the researchers found that a sticker provided the social licence to passengers to speak out against risky driving. They estimated that a simple sign – costing virtually nothing – halved the accident rate. If randomistas promoted their results with bumper decals, this one might say ‘Study shows safety stickers save souls’.

  Defecating outdoors might not seem as dangerous as a speeding bus, but it turns out to be just as life-threatening. Because one-third of the world’s population lack access to a toilet, many people do their number twos in the open, contaminating nearby lakes, rivers and drinking wells. This leads to diarrhoea and parasite infections, which are particularly likely to be fatal for young children.

  Initial strategies to reduce open defecation focused on building toilets, but it quickly became clear that in many cases these new facilities were not being used by villagers. So a movement known as Community Led Total Sanitation sprang up. This strategy begins with locals taking the facilitators on a ‘walk of shame’ – a tour to look at where people defecate. The facilitators then discuss with the community how that faeces makes its way back into the food and water supply. The aim of the project is to generate a sense of disgust, and persuade locals to (in the words of one campaign song) ‘Take the poo to the loo’.

  In randomised trials covering over 400 villages across India, Indonesia, Mali and Tanzania, sanitation education reduced open defecation rates and increased the share of households that installed their own toilets.47 Strikingly, children were taller in the treated villages, because they had avoided the stunting effect of faeces-borne diseases. And because the researchers found similar results in all four nations, we can be pretty confident that the program would work in other countries too.

  *

  Finally, let’s look at how randomised trials are shaping schooling in low-income nations. While better educated children earn more as adults, a perennial challenge has been to ward off parents’ temptation to withdraw their children from school to help feed the family. How can governments create a better incentive for parents to keep their children enrolled?

  One persuasive answer came from a randomised trial in Mexico in 1997, when the government of President Ernesto Zedillo decided to change the way subsidies were provided to poor households. Rather than make food and energy cheaper, the Zedillo government chose to experiment with giving cash to poor households, on the proviso that their children regularly attended health clinics and remained enrolled in school. The program – then known as Progresa – was to be rolled out across 500 Mexican villages. But Santiago Levy, a former economics professor serving as deputy Mexican finance minister, devised a short, sharp randomised trial.48 Rather than doing the rollout in an ad hoc manner, the government deliberately chose to implement Progresa in half the villages in May 1998, and the other half in December 1999.49 This provided a brief experiment – lasting just eighteen months – in which the two groups could be compared.

  The impacts on children were sizable. Thanks to Progresa, secondary school children were 15 per cent more likely to attend school. Preschool children were 12 per cent less likely to be sick. Toddlers were less likely to be stunted. Progresa households also ate more healthily, and were more likely to go to the doctor at the right times. When the government changed in 2000, Progresa became the first major Mexican social program to survive across administrations. President Vincente Fox changed the name to Oportunidades, but then supported the expansion of the program. It is now known as Prospera. Similar programs – known as conditional cash transfers – currently operate in over sixty countries.50

  For developing country governments and donors, Progresa didn’t just provide evidence that conditional cash transfers worked; it also showed that randomised trials could be done quickly and simply. From the standpoint of the Mexican government, it would never have been possible to give every village access to Progresa at the same time – so why not randomise the rollout and learn something about the program’s effects?

  Other randomised trials are ambitious not because of their sample size, but because they randomise things that we might have thought were impossible to test. In Afghanistan, a recent education experiment asked the question: what happens when you build a village school? Previously, the conventional wisdom had been that students were better served by higher-quality regional schools than lower-quality village schools.

  But was it really ethical to randomly assign schools to villages? To run their study, a team of researchers worked with a charity in north-west Afghanistan that planned to open thirty-one schools over a two-year period.51 Because they knew it would not be possible to open all the schools in a single year, the charity was willing to randomly determine which village schools would open in the first year and which would open in the second year.

  At the beginning of the study, no one had access to a village school. After two years, everyone in the experiment had access to a village school. But one year into the experiment, the effects could be studied. At that point, the early-adopter villages served as the treatment group, and the late-adopters as the control. The study received ethics approval because every child in the experiment ended up with access to a village school. The rollout was no slower than it would have been without a randomised trial.

  The results of the study were unexpected. Regional schools, spanning a few villages, had better staff than village schools. Teachers in regional schools were formally trained, while the village school was often run by just one teacher, who hadn’t finished high school. But to get to a regional school, pupils had to travel an average of 5 kilometres each way.

  The researchers found that village schools did more to boost educational attainment, chiefly because they had significantly higher attendance rates. The difference was particularly marked for girls, who were 52 percentage points more likely to attend school when a village school was opened. As one village elder observed, ‘The way is long, so there should be one or two people in the family to take the girls to school . . . and bring them back which is the main reason why they don’t let them go.’ This massive difference in attendance meant that the impact on test scores was significant and positive. Having access to a village school boosted student performance by approximately a full grade level, relative to regional schools.52 As filmmaker Woody Allen once put it, 80 per cent of life is showing up. The Afghan randomised trial disproved the conventional wisdom about village schools.

  Randomised trials have even been used to inform that most taboo aspect of education – sex education. Should children be taught to use contraception, or counselled to avoid all horizontal refreshments? The question is particularly relevant in Africa, where more than 20 million people are HIV positive, and over 4000 people contract the disease each day.53

  In 2009, on a visit to Cameroon, Pope Benedict XVI said that HIV/AIDS is a tragedy that ‘cannot be overcome through the distribution of condoms; on the contrary, they increase it’. Instead, the Pope argued for ‘the humanisation of sexuality, spiritual renewal which brings with it new ways of behaving’.54 Within days, the comments had been condemned by the United Nations, France and Germany. An editorial in The Lancet called them ‘outrageous and wildly inaccurate’ and called on the Pope to retract his words.55 But to many Africans, the world’s most senior Catholic was merely reflecting the way their schools delivered sex education. In Kenya, the ‘Nimechill’ campaign (partly funded by the US government) urged young people to make a two-fingered ‘peace’ sign to show their desire to
abstain from sex until marriage. Young people who were tempted to have casual sex were encouraged to join a ‘chill club’ or pledge to ‘keep it locked’. The official sex education textbook urged students to ‘Avoid sex’ and ‘Say NO to sex before marriage’.

  Does it work? Six years after Pope Benedict’s Cameroon visit, a team of researchers published their randomised evaluation of Kenya’s abstinence campaign. The ABCD campaign – ‘Abstain, Be faithful, use a Condom . . . or you Die’ – turned out to have absolutely no impact on pregnancy rates.56 Five years after receiving the education program, one-third of teenage girls had fallen pregnant – precisely the same proportion as in the control schools. Kenya’s abstinence campaigns were just as ineffective as the US programs we saw in Chapter 3.

  But while Kenyan teens didn’t seem to respond to a simple ‘no sex’ message, a more nuanced information campaign was successful in shaping behaviour. In Western Kenya, students were shown a video about the risks of relationships with older men, commonly known as ‘sugar daddies’.57 Their teacher then wrote on the board the HIV rates among men of different ages: 4 per cent for teenage boys, compared with 32 per cent for men in their thirties.58 In effect, the girls were told, they were eight times more likely to contract HIV from a sugar daddy than from a boy their own age. In schools that were randomly selected to learn about the sour side of ‘sugar daddies’, girls were nearly one-third less likely to become teenage mothers, and more likely to use condoms. Perhaps Pope Francis might shift the focus from criticising condoms to decrying sugar daddies.

  *