The Boy Who Could Change the World

by Aaron Swartz

  “Well, I felt challenged,” Mazur recalls. “My reaction, you can probably already predict this: ‘Not my students!’ After all, I was at Harvard—maybe this was some problem that was in the Southwest of the United States, right? . . . I wanted to show that my students could ace this test. . . . At that time we were dealing with rotational dynamics, and the students had to calculate triple integrals of complicated bodies with different moments of inertia. You know, we were so way beyond Newtonian mechanics there was no comparison between [this test] and what we were actually doing in class.

  “But I was so desperate to get this data, I walked into class and told my students I was going to give them this quiz. I called it a quiz because I didn’t want to scare them—you know how pre-meds are. . . . But I had to give them some incentive to take this test seriously, so I told them, ‘Look, if you take this test seriously, you can use your score to help you study for the upcoming midterm examination.’ Now, I told you, the midterm examination dealt with far more complicated materials, and I realized as soon as I said that, it was actually a huge lie. And I was worried that as soon as I said that my students would be offended by the simplicity of this test as soon as they started on it.

  “Oh, boy, were my worries quickly dispelled. Hardly had the first group of students taken their seats in the classroom when one student raised her hand and she said, ‘Professor Mazur, how should I answer these questions? According to what you taught me, or according to the way I usually think about these things?’” How was he supposed to answer that?

  Sure enough, the results came back and Mazur’s class wasn’t very different from any of the others. “When I saw how poorly my students had done, my first reaction was ‘Well, maybe you’re not such a great teacher after all.’ But that could obviously not be true, right? So I didn’t think about that too long. Well, what’s another reason the score could be low? Dumb students. But that’s pretty hard to say at [Harvard]; we have a very selective group of students. So I thought about it a little bit more and then, my mind, my twisted mind came up with the perfect excuse: . . . the test! There had to be something wrong with the test!

  “Take this question about the heavy truck and the light car, right? You don’t need to have taken physics to know you’re much better off in the heavy truck than the light car. So maybe students were confusing damage or acceleration with force—maybe it’s just a matter of semantics!

  “So I decided to do some testing of my own. I decided to pair, on an exam, two questions of different types on the same subject. One was a typical question out of the textbook, on which I knew students would do well, and another was a word-based question a little bit like the one with the heavy truck and the light car. And I decided to stay away from Newtonian mechanics, because we all have some intuitive notions of Newtonian mechanics before taking physics. I decided to do some testing in DC circuits, direct current circuits. I think very few people have any intuitive notions about circuits.”

  All right, so here’s the standard question (don’t worry if you don’t understand it):

  5.For the circuit shown, calculate (a) the current in the 2-resistor and (b) the potential difference between points P and Q.

  To you, this question may seem impenetrable. But for the physics students, this was the standard sort of problem they were used to answering. “This is straight out of the textbook. It’s not a particularly hard problem, it’s about 2/3 of a page of cranking numbers—but it’s not a completely trivial question either.”

  Now, for comparison, here’s the conceptual question:

  1.A series circuit consists of three identical lightbulbs connected to a battery as shown here. When the switch

  S is closed, do the following increase, decrease, or stay the same?

  (a)The intensities of bulbs A and B

  (b)The intensity of bulb C

  (c)The current drawn from the battery

  (d)The voltage drop across each bulb

  (e)The power dissipated in the circuit

  This question does not involve any numbers at all. “If you understand DC circuits, it takes 30 seconds to answer this question. And 25 of those 30 are spent on part I.

  “Now at Harvard, large courses are taught by two faculty members. So in order to put this on the exam, I had to convince my colleague that this was a good exam problem. So I showed him the problem and after reading it he looked at me and said, ‘Eric, you’re out of your mind.’ . . . He said, ‘Eric, we only have 5 problems on this exam. We cannot give away 20% of the exam!’ . . . We argued and argued . . . finally, he reluctantly agreed, mostly because we didn’t have any other problem. And we made it problem number one—the warm-up problem.

  “Well, it turns out the students overheated. ‘Professor Mazur, this problem number one is the hardest problem on the exam!’ Another student said, ‘I didn’t know how to get started on this problem.’ What do you mean, getting started? I mean, if you’re started you’re done! . . . Students had freaked out. Some had taken up more than six pages in their blue books, writing down absolutely everything they knew about DC circuits in hopes of somehow covering the right answer somewhere. And I had to read through all of it, hunting for the right answer!”

  A few words about this physics problem. The basic question is pretty simple: when you close the switch, the current now has two ways to form a circuit instead of just one. It can take its old path, all the way around (including through lightbulb C), or it can just go through the switch.

  Now, one of the most basic things about circuits is that the current takes the shortest path it can. (Current is lazy, you might say.) If you close the switch, the current travels through that path (the short) and lightbulb C turns off. This is why things go off when they short-circuit. But that’s not what the Harvard students thought. Most students figured that when the current had two ways it could go, it split half and half and took both. Thus, in their view, lightbulbs A and B stayed the same, while C decreased to half brightness.

  You can’t say this is simply a semantic argument—anyone with a little basic circuit equipment lying around can wire this up and see what happens. (But don’t; short-circuiting things is a little dangerous.) Either lightbulb C goes out or it doesn’t—and one would think that a student who aced circuits at Harvard would know which. But they didn’t. When he looked at the results, Mazur was shocked to see that there were students who aced the traditional question but flunked the conceptual one. Even more shocking, there were no students who did the reverse—there was nobody who answered these basic questions perfectly and then went on to fail the harder parts of the test. No one.

  But this is just the tip of the iceberg, even in physics. In one experiment, Andrea DiSessa had kids play a computer game that simulated basic Newtonian physics. The goal was to kick a simulated ball into a goal. Psychologist Howard Gardner describes one typical subject:

  Consider what happened to an MIT student named Jane, who was studied intensively by DiSessa. Jane knew all the formalisms taught in freshman physics. She could trot out the equation F = ma under appropriate textbook circumstances, she could faithfully recite Newton’s laws of motion, and she could employ the principles of vector summation when asked to do so in problem sets. Yet as soon as she began the game, she adopted the same practices as the naive elementary school students, assuming that the turtle would travel in the direction of the kick. For half an hour she stuck to this inappropriate strategy. Only when she was convinced that this strategy would not work did she make the crucial observation that an object will not lose its prekick motion just because she applies a kick in a certain direction. This realization finally led to experimentation in which the velocity (or speed in a particular direction) of the dynaturtle was at last taken into account.

  As the experimenter noted:

  We have already discussed the remarkable similarity of [Jane’s] cluster of strategies to those exhibited by 11- and 12-year-old children. But what is equally remarkable is the fact that she did not, indeed fo
r a time could not, relate the task to all the classroom physics she had had. It was not that she could not make the classroom analyses; her vector addition was, by itself, faultless. It is more that her naive physics and classroom physics stood unrelated and in this instance, she exercised her naive physics.

  But, as a battery of studies have shown, Jane’s errors are fairly typical of college physics students. When asked what happens to a ball shot out of a curved tube, students predict it will keep curving in the same way, as if the ball somehow absorbs the curve. When asked about the forces acting on a coin tossed into the air, 90% of engineering students say there are two: the upward force of the hand and the downward force of gravity (in reality, there is just gravity once the coin has left the hand). Students who have studied relativity ignore what they’ve learned when asked about the behavior of distant clocks.

  I could go on, but let’s move to biology. Even students who have studied biology for years continue to think that characteristics an animal acquires in one generation can be passed down to its children (like the giraffe who stretches its neck further to reach more distant food). They assume that all changes in animals are a result of some change in the environment and they believe that evolution has a particular direction rather than stumbling around randomly. They believe that animals behave intentionally: parasites are trying to destroy their hosts, chameleons intentionally change colors to disguise themselves. They think that plants suck up soil through their roots and that their genetic traits are distributed in precise ratios, exactly three to one.

  You might hope that things are better in math, where there are fewer everyday misconceptions. But even basic algebra turns out to be a problem. When told to write an equation representing that there are six students for every professor, most college students write: 6s = p. But this is exactly backwards: it says the number of professors (p) is six times the number of students (s). And this isn’t simple carelessness; even when students are warned about this problem, they keep on making it.

  This is just one example of a larger problem—students don’t really seem to know what the symbols mean, they just know some basic operations that can be performed on them. When given a problem they’re not sure how to solve, students simply start adding all numbers in sight. Asked to add two fractions, they just add the numbers on the top together and then add the numbers on the bottom. And their understanding of decimals isn’t much better: they refuse to believe that .6 is bigger than .5999 yet somehow less than .6000001.

  Students in computer science have an almost opposite confusion: they don’t seem to understand that the computer is simply rigidly following rules and instead expect it to understand what they’ve written, like any human reader would. Thus, for example, they are puzzled as to why the computer doesn’t simply put the largest number in the variable LARGEST, since that’s so obviously what they intended.

  College students who have studied economics seem to approach economic issues very similarly to those who have not. Both made claims like “The more they sell, the lower the price should be, because you can still keep the profit the same”—a statement wildly at odds with the role of profit in economic theory. College in general seems to make little dent in this kind of basic reasoning. One study found that students took pretty much the same approach to reasoning social and political issues before they went to college as they did after.

  Turning to the softer subjects, a famous experiment by I. A. Richards found that when asked to summarize poems, even literary undergraduates turned out to wildly misunderstand them. Not only did they not grasp the poetic implications, they seemed incapable of following its basic meaning. As Richards wrote, “They fail to make out its sense, its plain, overt meaning, as a set of ordinary intelligible English sentences, taken quite apart from any poetic significance.”

  Furthermore, when asked to rate poems from which the author’s name had been removed, they gave low ratings to most famous poets and instead preferred a terrible unpublished poem by an unknown poet. Why? Instead of looking at the meaning, they simply gave high ratings to poems that were positive, rhymed well, and used a sensible vocabulary.

  In every case, we see the same phenomenon at work: children may be able to memorize enough formulas and facts to pass the test, but they literally have no idea what they’re talking about. When asked the question in a slightly different way or with a practical application, the appearance of understanding simply collapses.

  Schools do something. We all know that getting a degree increases your wages, even if there hadn’t been “literally thousands of published estimates” of this effect.* But what exactly is it that schools do?

  Weiss, www.Jstor.org/stable/2138394.

  The standard theory, of course, is that schools teach. We go there, we learn things, and they make us better at our jobs, which causes employers to pay us more. But evidence for this theory turns out to be rather hard to find.

  Economist Joseph Altonji tried to calculate the benefits of education by looking at the benefits of each individual high school class. He compared the wages of people who took a class with those who didn’t take it to try to calculate how much more the average student made by taking that class. From that, he could work backwards to try to determine how much money a student would have lost had they taken no classes at all. The result was shocking: taking no classes has no statistically significant effect on wages; indeed, it might even increase them!

  A similar study by different researchers done with different data in a completely different way came to basically the same result: students who took no classes while in school would make around $0.12 an hour more.

  The same problems persist when we look at how well a student does in a class. “There is a long history of researchers failing to find an economically significant relationship between scores on achievement tests and wages,” notes economist Andrew Weiss. Success on standard school tests of vocabulary, reading ability, math skills, and so on have no noticeable effect on wages. Nor does getting good grades seem to be an indicator of success in the workplace. “Most students realize few benefits from studying hard while in school,” complains economist John Bishop. “Performance in high school as assessed by student grades explains almost nothing about job success . . . higher grades [do] not improve the probability of getting either a job or competitive wages once one has a job.”*

  http://digitalcommons.ilr.cornell.edu/cahrswp/400/.

  A final piece of evidence is the GED. If schools were simply about educating people, students with a GED would fare little differently from students who had graduated high school. Indeed, students with a GED are, on average, more educated than high school graduates—after all, most kids don’t have to pass a high school achievement test to pass high school. But all this education doesn’t buy them much in the labor market—students with a GED fare little differently than other high school dropouts.

  I should note that the researchers are not happy with these results. John Bishop, for example, considers them an outrage. But despite their best efforts, they cannot make the facts go away.

  So what is it schools are really doing if not educating the next generation? Well, just look at what’s left over: schools are places where kids must show up every day at 8 a.m. for years on end, sit at uncomfortable desks under fluorescent lighting with a group of relative strangers, and obey arbitrary instructions from their superiors about the appropriate way to carry out repetitive intellectual assignments. Even a casual glance at a modern office will show you that these are skills very much in demand.

  Ask employers what they want from their employees, and they don’t say academic brilliance. Indeed, in the 1970s employers were complaining that their workers were too educated, causing “unrealistic job expectations.” The resulting “poor worker attitudes” led to “productivity and quality problems and (in some cases) to outright sabotage.”† Instead, employers ask for “character”: “a sense of responsibility, self-discipline, pride, teamwork, and enthusiasm.” I
n other words, employers want people they can rely on to do their work with pride and enthusiasm—and certainly not people who would engage in misbehavior and sabotage.

  Capelli, 5.

  Looking at new hires who don’t “make the cut” and get fired in their first few weeks, one can see where the problem really lies. Despite all the talk about how we need better schools to compete in a global economy, a survey of employers found that only 9% of workers were dismissed because they couldn’t learn to do their jobs.

  And looking at workers who are liked by their bosses finds they have basically the same traits as those students who are liked by their teachers: “consistent attender,” “dependable,” “identifies with job/school,” a willingness to quit, and “prosocial attitudes”—i.e., a willingness to do more for the boss.•

  Edwards 1977.

  In short, schools don’t really teach kids anything because they’re not about really teaching kids anything. They’re about teaching kids to stay quiet, do their work, and show up on time.

  This isn’t an accident. This was the plan all along.

  It’s difficult to even imagine what America was like before the industrial revolution. Their notion of freedom was far stronger than the one we have today. For many Americans, life wasn’t about showing up at a job at a specified hour, following orders all day, and returning home for a couple hours of “free time”—that would be considered slavery. A free American was one who worked on their own or with their family, worked from home, worked whatever hours they liked, and got paid based on what they accomplished.

  Under the putting-out system, for example, merchants would deliver raw materials like cotton to your house. When you felt like it, you’d card, spin, and weave the raw cotton into cloth. And then the next week the merchant would come by to buy from you whatever cloth you had produced. If you wanted to make more money, you simply did more work or figured out how to work more efficiently. If you wanted to take a vacation, there was no one stopping you—you just wouldn’t get paid that week.