by Dean Karlan
On the Right Track
Clearly, the students targeted by the balsakhi program benefited from the extra attention they received. But one might have expected to see impacts across the board—even for students who never needed remedial instruction—since pulling out the low achievers effectively cut class size in half for two hours per day. Proponents of small classes have long argued that fewer pupils per teacher leads to more learning through increased individual attention, and instruction tailored to meet the needs of each student.
When smaller classes are created by dividing students according to their ability, the technique is known as tracking. Advocates contend that tracking allows teachers to more effectively tailor their instruction to the level of the students. In contrast, when faced with a wide range of student ability, they are forced to “teach to the middle,” leaving the struggling students overmatched and the stronger ones underserved. The opposing argument is that everyone benefits from sharing a classroom with the best and brightest, and so dividing classes by ability robs weaker students of a valuable resource.
The jury is still out on the value of tracking in general, but we can say one thing about it: In many contexts, it’s a viable alternative. For a school already hiring additional teachers to reduce class size, tracking—by grouping students according to their grades on a prior year-end test, for instance—is cheap and easy. It can also be very powerful.
The balsakhi program in Mumbai was essentially a part-time tracking scheme. For two hours each school day, classes were effectively tracked when the weaker students were pulled out for remedial instruction. But students who didn’t meet with the balsakhis appeared not to benefit much. In fact, the researchers couldn’t rule out the possibility that the program had no effect on them at all. As we saw, there were some significant positive impacts from the program in general, but the evaluation can’t say whether tracking in particular was an important part of the explanation, since it looked at the whole package: two hours’ daily instruction from a specially trained balsakhi for some students, plus smaller classes and part-time tracking for all. To say more, we need an RCT that isolates the impact of tracking by itself.
Unsurprisingly, Esther Duflo, Pascaline Dupas, and Michael Kremer were hot on the trail, this time in Kenya. They partnered again with ICS Africa, now familiar from the uniform giveaway and deworming programs, and designed another RCT. ICS was rolling out a program that identified primary schools with just one first-grade teacher and gave them grants to hire an additional one, effectively dividing each first grade class into two sections.
It was a golden opportunity to test tracking directly. In half the schools, students were assigned to sections based on their grades in the previous term. In the remaining schools, students were randomly assigned to sections. So the only difference between schools was the method of assignment into sections—by rank versus by chance—which gave the researchers just what they wanted.
The program was a success. In tracking schools, students’ test scores in both sections improved more, on average, than those of their counterparts in nontracking schools. So, unlike in the balsakhi program, benefits seemed to accrue to all students—not just lower achievers. Another powerful piece of evidence came from students near the middle of the ability spectrum: They improved similarly, whether they were assigned to the higher or lower section. The top students in the lower sections and the bottom students in the higher sections fared equally well. That was a big win for tracking, as it suggested no students were losing out.
Which is not to say that opponents’ arguments don’t hold water; in fact, the study found evidence that smart kids do positively affect their classmates’ learning. Presumably, students in the low sections of the tracking schools were missing out, but it appeared that their losses were overshadowed by the gains from teachers tailoring instruction to the level of students. The test scores supported this story: low-section students improved more in basic competencies, while high-section students improved more on advanced topics.
This approach is now one of IPA’s scale-up efforts, with a recent launch of a large pilot in Ghana under the leadership of research director Annie Duflo. If it proves successful in this context, the groundwork has been laid for a nationwide scale-up and for replication in other countries, with the generous support and enthusiasm of the Children’s Investment Fund Foundation in the United Kingdom.
Another Surprise Knockout
Most education programs—and all the ones we’ve seen thus far—focus on getting teachers and students into schools. It does stand to reason. As I said at the beginning of the chapter, those are two ingredients virtually everyone can agree on.
Mark Twain was always an oddball, and, were he alive today, he might not be part of the consensus. He famously admonished, “Never let your schooling interfere with your education.” Maybe he knew something others didn’t about the elusive key to learning, or maybe he was just talking about the importance of life experience. But he probably wouldn’t have guessed how apposite his comment would be to some corners of the world a century after his utterance. If he could see the decrepit schoolhouses of Uttar Pradesh, India, I bet he would have some even stronger things to say.
The school system in Uttar Pradesh was broken. There were spectacular failures of education across all subjects and grade levels. A 2005 survey of children aged seven to fourteen produced some dismal figures: One in seven kids couldn’t recognize a written letter, one in three couldn’t read numbers, and two-thirds were unable to read a short story written for first-graders. The survey also found that students’ deficiencies went largely unnoticed by their parents. In the most severe cases, where children couldn’t recognize written letters, only a third of parents knew the extent of the problem. Most thought their children could read just fine.
This was in spite of a government program that sought to get local communities involved in making schools better. The vehicle for participation at the local level was the Village Education Committee, made up of three parents, the head teacher of the village, and the head of the village government. As the primary bridge between the villagers and the district-level education administration, the committees served many functions, from monitoring and reporting on classroom activities to hiring and firing teachers and allocating federal funds to schools. There seemed to be opportunities for regular people to make a difference in education, either by working through committee members or simply by joining.
Maybe those opportunities were mirages, or maybe people were just apathetic. Maybe both. Whatever the case, it is no surprise that parents were also neglecting the Village Education Committees, given the extent to which they were misinformed about their own kids’ schooling. Ignorance about the committees was almost universal, with fewer than one in twenty parents aware of their existence.
Incredibly, this ignorance extended to the committee members themselves! When asked which organizations they belonged to, barely a third of members mentioned the Village Education Committee; when specifically prompted about it, one in four still had nothing to say. What little awareness members had about the committees was mostly superficial. Almost nobody understood the roles and responsibilities of the committee. Only one in five members knew that they were entitled to government money at all, and just one in twenty-five knew they could request funds to hire additional teachers. The upshot was that Village Education Committees were utterly ineffectual, and students were deprived of a valuable advocate.
Pratham, India’s largest educational NGO, wasn’t content to let the children of Uttar Pradesh suffer. They believed that if the villagers (including committee members) learned about the powers and duties of the committees, maybe they would respond. So Pratham partnered with researchers Abhijit Banerjee, Esther Duflo, and Rachel Glennerster, and with Stuti Khemani of the World Bank, to test three programs designed to kick village education into gear.
In the first and most basic program, Pratham organized a series of neighborhood-level meetings t
hat culminated in a village-wide meeting to discuss the state of education, the role of the Village Education Committee, and the educational resources available from the federal government.
The second program had all the elements of the first, plus training on a testing tool that let villagers assess the level of students’ learning. The assessments were conducted in each neighborhood and compiled in “report cards,” which were discussed at the village-wide meeting. Villagers were also trained on a monitoring tool that allowed them to track progress in students’ achievement over time.
The third program had all the elements of the second, plus training in Pratham’s “Read India” program, a group-based reading skills curriculum. Once trained, villagers were encouraged to set up reading camps for local students and run them as volunteers.
Pratham chose 280 villages in Uttar Pradesh and randomly assigned a quarter to receive each program. The remaining quarter were monitored as a control group. After the programs had been running for a year, they surveyed to see what had changed.
Their first finding was encouraging: Across all three programs, the village meetings had been well-attended, with over a hundred villagers participating on average. On closer inspection, though, it looked like the meetings might have been a waste of time. There was a marginal increase in awareness of the Village Education Committees, but it was tiny relative to meeting attendance figures—so small, in fact, that it actually implied that many of the villagers who showed up never even learned of the committees’ existence.
Whatever the impact on awareness, the functioning of the committees and the schooling situation—the real objects of the initiatives—were unchanged. There was no increase in the hiring of teachers, no change in parents’ engagement with schools (e.g., visiting, volunteering, donating), no evidence of children switching schools, and no change in student or teacher attendance. It is hard to conclude anything except that the meetings had failed.
Fortunately there were some bright spots in all the darkness, and perhaps a key to the secret ingredient in the recipe for education. While the committees remained utterly useless, reading camps thrived. Of the sixty-five villages that had been offered training in Pratham’s Read India program, fifty-five had started camps, serving an average of 135 children per village. The camps were tremendously successful, especially for the people who needed it most. Children who, at the outset, couldn’t identify written letters, got a huge boost from the reading camps—they all learned to do so. By comparison, less than half of comparable children from non-reading-camp villages made the jump.
The wild success of the reading camps gives us reason to be hopeful. Even where schools are practically useless, and where parents are at best unable to coordinate their noble efforts at improvement (and at worst utterly apathetic), there are still ways to help. We just have to think outside the box—or, in this case, outside the schoolhouse.
Finding the Secret Ingredient
Students and teachers are easy to agree on; the pixie dust that makes the whole thing work is not. Some of the most promising results come straight out of left field.
We have to cast a wide net for solutions in education. The incredible impact of deworming on students’ attendance and the power of reading camps to boost reading levels is proof positive that routes to learning don’t all start and end in the classroom.
We in developed countries feast on quality education all the time, but in some sense we don’t know what we’re eating. One reason why it’s so hard to identify the secret ingredient(s) is that school systems in rich countries typically have a great many things that their poorer counterparts lack, from well-appointed classrooms to healthier students to functioning PTAs. That means teasing out the effect of any one input alone just by looking at a well-functioning system is often impossible. (Indeed, the same difficulty also applies to research about improving farming, banking, health care, and other areas of life that touch us all, rich and poor alike.)
What we can do is go to the field and test. Little additions and changes, one or two at a time, to find out what makes education tick. We have touched here on some innovative ideas—but this is only the beginning. For starters, millions of schools around the world are still in dire need of the two most basic components: students and teachers. How many stories like Anthony’s are out there?
Of course, questions also remain about textbooks, school lunches, classrooms, desks, and countless other inputs. The more we learn through rigorous testing and evaluation, the more of these we start to get right, and the closer we come to a recipe for education that nourishes everybody.
10
TO STAY HEALTHY
From Broken Legs to Parasites
For just over a year, Jake lived in a tidy compound house on Ring Road in Accra, Ghana. On a friend’s recommendation he hired a housekeeper to clean and do laundry twice a week. Her name was Elizabeth, and in January 2008, she hurt her leg.
Jake found out about the incident a few weeks after it happened, when he called Elizabeth to ask why she hadn’t been coming by the house to clean. He said, “Elizabeth, I haven’t been seeing you recently.”
“Oh, Brother Jake. I’m sorry I haven’t been coming. I broke my leg.”
“Elizabeth! What happened?”
“I was at market and I fell inside a ditch.”
“Oh! I’m so sorry. Have you seen a doctor?”
“Yes, I went to hospital.”
“And the doctor told you your leg is broken?”
“Yes. He said I have twisted it. Near the foot.”
“Oh, so it is twisted. But is the bone itself broken?”
“Yes. The bone is not broken.”
They had reached the limit of their ability to communicate over the phone. It was clear they would need the benefit of hand gestures to get the point across. Elizabeth said she was well enough to come the following Monday to clean. Then she would tell him the whole story.
When he came home from work the following Monday he found her sitting on the front porch. Her left leg was extended awkwardly in front of her, swollen below the knee and wrapped tightly in a ratty Ace bandage from the shin down to the foot. She wore her usual wide smile and greeted him kindly.
As they talked she described the accident. She had been at an outdoor market, crossing a ditch on a wide plank that had been laid as a bridge. The plank broke under her and she fell a few feet onto the gravelly dirt below. Her two-year-old son, Godswill, cinched tightly to her back by a fabric wrap in the typical Ghanaian fashion, was lucky not to be crushed. A few days later she went to the hospital, where the doctor suggested a few possible courses of treatment.
Elizabeth chose the most traditional. She went to see an herbalist. He put her on a regimen of daily applications of a topical cream and weekly checkups. The whole package cost sixty cedis (about sixty dollars) per month, half her salary. But for all that money, Elizabeth couldn’t be sure exactly what she was getting. Herbal clinicians in Ghana rarely let their patients know the contents of the balms, salves, poultices, and tinctures they prescribe, because the ingredients can usually be found at the local market for pennies. Whatever it was, Elizabeth liked it—at least enough to keep taking it. When the first month was up, she signed on for a second.
Unfortunately her leg was not so convinced. After two months of treatment, there were still good days and bad days. Sometimes it felt completely normal and there was no pain or soreness; other times, when it was swollen, she had to walk on it very tenderly, or not at all. She would unwrap it and rewrap it tighter. The pain on these days came “from inside,” she would say, pointing to the area above her ankle.
It was a dismal situation—two months gone by, pockets half empty, and leg still a mess—but, one must concede, not a completely surprising one. If her leg was actually broken, then the herbalist’s ointment by itself couldn’t be expected to do much good.
Whatever was happening under the skin, it was painful just to watch her. She did her level best to carry on,
coming to clean on most of the appointed days, often limping while she swept, sitting at the laundry basin with her leg puffed up and jutting out to the side. Jake told her she had better go back to the hospital for an X-ray and a consultation.
Nobody said it would be easy. Elizabeth went to Korle Bu Teaching Hospital Monday morning, signed herself in, and waited. Around midday she was told by a nurse that the doctor wasn’t coming in; she should come back Wednesday. So she was there again Wednesday morning, name on the sign-in sheet, sitting in the folding chair. She sat there through lunch. In the afternoon a woman came out from behind the reception counter and told Elizabeth she had seen her name on the sheet with “X-ray” written next to it, and had watched her sitting all morning. Didn’t she know she was at the wrong hospital? X-rays were done at Ridge Hospital across town. By now it was too late to go to Ridge, though, so she should go tomorrow morning, first thing. She did. Thursday at Ridge the doctor should have been in—he hadn’t called in sick—but nobody could find him. Surely, the receptionist said late that afternoon, he would come tomorrow. (As we will see, this is not a freak occurrence. As with teachers, simply getting doctors and nurses to show up is a big part of the problem with health care in developing countries.)
In fact he did come on Friday, and Elizabeth was there waiting for him. It must have been an exciting moment, as the anticipation had been building for about twenty hours in various waiting rooms over the course of the week.