Eight years of quiz scores footnoted and time-stamped? Facebook friends? Television-watching habits? What teacher in 2014 has the time to figure out the relevance of all that information? Not when lesson plans need writing, parents need to be called, and quizzes need grading. Thankfully, this isn’t 2014.
Your daughter’s teacher opens the link on her phone and downloads the relevant files. They’re automatically run through a modeling app that sends her a notification. She suddenly knows exactly how well your kid will do on the first four quizzes, right down to which errors she’s going to make. “It seems your daughter keeps reversing second- and third-order operations. We’ll start drilling on those tomorrow. I’ll schedule a half-hour online tutoring session for the evening, right after Teen Mom?”
“That would be great,” you answer, thankful that you aren’t being asked to come between your daughter and your daughter’s violent devotion to her favorite show on TV.
“There’s one more thing,” says the teacher. “The profile shows that when Becky is confronted by a particularly hard problem she’ll switch over to Drawsomewords 8 for five minutes or so. She seems to have great spatial-representation skills. I have a friend that designs drafting freeware at a studio downtown. I think that if we can get your daughter an internship, it might help her make the connection between math and drawing and then she’ll exhibit a bit less resistance to second- and third-order operations.”
This offer seems generous, perhaps too much so. “Isn’t she a bit young for an internship? I mean, it’s her first year of high school.”
The teacher nods politely. “She’s a bit late, actually. The average student her age has already started a company. But I think I can pull some strings.”
• • •
HOW does the above scenario become reality (and do we want it to)? For starters, the feedback loop between a teacher administering a lesson and a student taking a test needs to collapse to the second or two it takes a student to click a mouse. More important, the time and convenience costs of keeping records on individual student performance would need to fall to virtually zero. Finally, teachers, state education secretaries, administrators, parents, and employers would have to be willing to accept new performance metrics in place of what we today call grades. Every item on that list, except for the last one, exists in 2014.
But the most important step is philosophical. We need to acknowledge what education is today: essential, expensive, and in terrible shape. The United States spends more than $10,000 a year per elementary and secondary student; that’s $2,000 above Japan and $4,000 above South Korea, two countries where students are outperforming us in science and math.
Even if we don’t know how to invest in school, we understand its importance. We’ve absorbed the fact that high school should prepare students for college because a college degree has never been a more essential credential to join the middle class. People with different education levels experience the same national economy in dramatically different ways. Unemployment among people with a high school degree was 8 percent in December 2012. Among people with a bachelor’s degree it was 4 percent. Statistically, people with a master’s degree or higher saw no employment collapse during the Great Recession. While it’s true that nearly half of all 2012 college grads in the United States were either unemployed or, far more likely, underemployed in low-wage jobs (and carrying an average of $27,000 in school debt), they were still faring better than their peers who did not have a degree.
This speaks to a national skills gap that’s growing along the lines of economic class. Low-skilled jobs are partly being replaced by a smaller number of high-skilled jobs. Even as GM parts factories were shuttering in Michigan, kids in Silicon Valley were seeing their start-ups bought out in a matter of months. In many cases it wasn’t because of the products or services those fledgling companies were building but because of the talent contained therein, a phenomenon sometimes called acqui-hiring.
Our nation’s response to our education challenge (both locally and nationally) embodies the worst aspects of an obsolete mind-set. A slavish devotion to lecturing has been compounded by a nascent obsession with testing. Whether it’s the Adequate Yearly Progress (AYP) reports mandated by No Child Left Behind, SAT scores, or just finals, the effect is the same: at the end of a designated interval—a week, a semester—teachers ask students to take a test. Too often we accept whatever result comes back as an objective and useful indication of the students’ command of the material (administered via lecture). We do this despite the fact that history is full of intelligent people who didn’t perform well on standardized tests and we know people forget information they’ve been successfully tested on. A lot of this testing is purely for the sake of identifying failing schools and teachers. Increasingly little of it has to do with helping students learn. Lectures make testing necessary. Testing makes lectures important. Testing is the big data present.
The naked future looks very different.
The Teacher as Superstar
The year is 2007 and Stanford professor Andrew Ng is in front of four hundred students, giving his famous and highly rated lecture on machine learning. He asks a question of the undergrads assembled before him and observes three distinct behaviors in response.
Ten percent of the class is slumped back; these students are texting, checking Facebook, or recovering from hangovers. They’re what you might call “zoned out.” About half the students are still madly typing the last thing said, displaying the sort of dedicated academic seriousness that propelled them through AP courses to get to Stanford. But they aren’t raising their hands. Thirty percent or so sit quietly, waiting for someone else to answer. Only a few kids near the front, less than 1 percent by Ng’s estimation, ask to be called on. If one of them gets the question right, Ng can breathe a sigh of relief and move on to the rest of the material.
The predictable dreariness of this lecture hall exchange began to depress Ng. It’s a scene you could find in virtually any lecture hall today. Indeed, the lecture has changed relatively little from the time of Socrates, as evinced by the fact that Plato spends most of The Republic following Socrates around taking notes. It’s a method of teaching that has endured because it’s functional, which is not exactly a compliment.
When Ng looked out over that horde of four hundred students, he recognized himself among them, one of the quiet kids, neither waving his hand nor asleep, simply sitting, passive and indecipherable.
“I was a shy kid back in school. So raising your hand and asking a question, or answering a question, I did that sometimes, but not always,” he tells me in his office on the Stanford campus.
Andrew Ng, it turns out, was fortunate to be a quiet student. If not for this quality of bashfulness, he would never have started his company, Coursera, which is remaking education for the twenty-first century.
Today, anyone in the world can familiarize themselves with the fundamentals of machine learning through Andrew Ng’s massively open online course (MOOC). It boasted more than one hundred thousand alumni by July 2012. In his interactive instructional videos, Ng comes across very much as he does in real life. He is polite, serious, attentive, constrained in his movements, but friendly. He is not as shy as he was as an undergraduate at Carnegie Mellon but he remains an exceptionally soft-spoken man. Though he lectures quite successfully to auditoriums, he is clearly an instructor who thrives on one-on-one exchanges. His online course affords him the opportunity for this type of interaction with tens of thousands of people.
Coursera offers a huge departure in the way student performance is measured and understood. Instead of tests at the end of the week or semester, short, interactive quizzes are interspersed throughout the lesson, in keeping with the human attention span as science actually understands it (not how headmasters want it to be). Every student must interact with the material as they’re studying it, not afterward. This allows Ng’s online platform to be not onl
y an information distribution system but a telemetric data collection system.
“We can log every mouse click, every time you speed up or slow down the video, every time you replay a particular five-second piece of the video. Every quiz submission, be it right or wrong, we know exactly how many seconds you took to do every quiz, and every post you read or posted. We’re starting to look at this data, which is giving us, I think, a new window into human learning,” Ng told me.
He admits that the subject matter in his machine-learning course is not easy. In fact, without a good understanding of linear algebra and at least some familiarity with statistics, the course is impossible. Chris Wilson from the online magazine Slate attempted the course and noted despairingly, “Avert your eyes, Mom, because I have a confession to make: I’m not entirely certain I’m going to pass.”
Writing code for learning algorithms doesn’t become intuitive just because we want it to, or because the White House has a renewed interest in science, technology, engineering, and mathematics (STEM) education, or because someone designed a video game to teach it. Computer science will remain a difficult, multistep, and rule-filled domain because such is science. Though we are prone in the Internet era to lionize technology wizards the way we used to venerate rock stars, science and music aren’t interchangeable. Science will never feel natural because it is not natural. For all his genius, Andrew Ng can’t change this.
What telemetric education offers is the chance for all students to raise their hands and be heard. That opportunity doesn’t come easily in a crowded classroom and especially not for women or minority students, many of whom feel that if they ask the wrong question or display ignorance, they’ll confirm some unflattering, broadly held perception about their social group. We now understand this to be a real phenomenon, one that plays out in classrooms around the world every day, called stereotype threat.
It turns out other people’s bad expectations are holding you back.
Here’s how we know this is true. In 2006 Smith psychology researcher Maryjane Wraga and a few colleagues gathered together fifty-four female students and paid them $20 apiece to perform a series of spatial tests. Wraga divided them into three groups and told the first group that these were the sorts of tests women were expected to do well on and then told the second group that they were not expected to do well. In essence, two-thirds of the participants were told that they were going to confirm or refute either a positive or negative stereotype about all women. She didn’t tell the third group (the control group) anything.
Each subject took the test under functional magnetic resonance imaging (fMRI). The women who were told they were being examined to confirm a negative stereotype showed activity in the part of the brain associated with processing anger and sadness (the rostral-ventral anterior cingulate) and the part of the brain charged with learning about social and interpersonal relationships (the right orbital gyrus). In other words, the subjects themselves encoded the stated premise of the experiment—that women were more likely to perform negatively on the test—as fact.
Conversely, the second group of women, the ones who were told that the test was intended to validate a positive stereotype about their sex, displayed activity in the portion of the brain associated with working memory (right anterior prefrontal cortex) as well as the portion of the brain associated with egocentric encoding (middle temporal gyrus), which is how we perceive objects in relation to us.
We’ve known that confidence can affect test performance but until Wraga’s study, science didn’t know exactly how large a role social stigma and stereotyping play in education. Wraga and her colleagues found that the women with the positive stereotype stimulus did 14 percent better on the challenge than the women with negative stereotype stimulus.1
Stereotype threat could be a contributing factor in the fact that just 30 percent of African Americans and fewer than 20 percent of Latinos have an associate’s degree (among those currently in their twenties). It may also be one of the reasons why the United States now has a higher college dropout rate than any highly developed country.2
Consider the implications of Wraga’s findings, particularly that 14 percent performance differential between the two subject groups. Bad performance doesn’t result in stereotyping; rather, the situation is reversed. When a person is continuously exposed to negative predictions about how she’ll perform on a test as a factor of group affiliation it’s the prediction that has a deleterious effect on performance. Coursera creates an environment where students are shielded from the effects of these predictions.
So far, the system’s biggest asset has been a collapse in the cost to do certain types of education and curriculum experimentation. “In a traditional education study, you may have twenty students in your experiment group and twenty students in your control group. And if you’re really lucky, maybe you get something that’s just barely statistically significant,” says Ng. An extremely large education study, encompassing on the order of hundreds of students, can cost thousands of dollars and sometimes won’t produce actionable results for months or years. On Coursera, every interaction can become an A/B test in which one-half of the test-taking population is shown one lesson and the other half is shown a different one. On any given day Ng can run such a test on twenty thousand students. All of this data is helping him understand the process of learning in a way that is specific to any individual and yet broad enough to be applicable to any student in any country.
In 2012, when two thousand of his students submitted the same wrong answer to a question, Ng realized immediately the problem wasn’t the kids taking the course. Rather, the malfunction lay in the question itself: “In a normal class, if two students out of a hundred submit the same wrong answer, you probably won’t even notice. But when two thousand out of a hundred thousand students submit the same wrong answer, that’s a very strong signal to the instructor that some clarification is needed.” Ng checked the question again and at first didn’t see any error. Most of the students got it right. But when he applied a learning algorithm to the data set, he discovered that the wrong students were all making the same type of error: they were reversing two steps in the formula. In effect, he learned to predict errors before they occurred. He created a customized error message so now every student who misses that question is given a clue, a message urging him or her to go back and reconsider the order of the operation, which allows the students to correct missteps and move on much more quickly.
These sorts of discoveries are increasingly common in online environments that actually collect and use student data. When MIT physicist David Pritchard first analyzed the results of a widely used concept test given to about a thousand students, he found that most of the carefully designed problems tended to challenge the problem students but were a breeze for the better-prepared kids in the class, as anyone might have expected. But two problems in particular produced a counterintuitive result. The A students did well and the B students did less well; however, the C and especially the D students did better than the B students! He realized that the wording of the question was ambiguous, but the A students seemed to know what was being asked. Many of the students doing less well, after misunderstanding the question, had a common physical misconception that resulted in the correct answer—two errors combined had a canceling effect. It’s the sort of event that a regular teacher lecturing to a class of thirty would never notice but because Pritchard had hundreds taking the concept test, the size of the data set enabled analysis that made the error visible.
Rapidly adjusting lessons on the basis of new incoming information is only possible in the Internet era. But Ng is careful to point out that he’s not actually an advocate of Web-only education. He’s a fan of what’s sometimes called the flip model, which is in-person education with a heavy online component.
Far more students today have seen Ng’s lectures because of the popularity of the machine-learning course, but he actually spends much les
s time lecturing than he used to. He uses his class time to teach high-level concepts, try out new material, and workshop big problems. “The instructor can look at a Web site and see what the students are getting right and getting wrong, so that [the instructor] can focus the classroom discussion on what the students are actually confused about.”
Some of the most pioneering work in this field was conducted by Kansas State University physicists Dean Zollman and Sanjay Rebello, who handed PDAs to their students in 2005 and instructed them to actually text in class (so long as the subject was physics). Zollman and Rebello were able to quiz every student in real time and then alter lesson plans accordingly.
In many ways the Rebello-Zollman classroom provided a snapshot of what Ng is doing, and, perhaps, all classrooms of the future. In a press release Rebello remarked that the system worked well to address issues of minority student engagement. “I find that even in a small class it can give me a feeling for how this silent majority or silent minority of students is thinking about things that I wouldn’t [normally] get.”
Ng already has a lot of competition in the online learning space. On the East Coast the edX program, which features online interactive courses from MIT and Ivy League schools, has already attracted thousands of participants. On the West Coast, two of Ng’s colleagues at Stanford, Sebastian Thrun and Peter Norvig, put their own interactive course in artificial intelligence online at around the same time and are now spearheading a company called Udacity.
Norvig also serves as a director of research at Google and is one of the most senior executives of the demographically young company. Prior to joining the search giant he led the computer science division at the NASA Ames Research Center where he worked on sending robots into space. He’s considered one of the world’s top minds in AI but is also known for his oddball sense of style. If you catch Norvig at a big event, he’ll likely be in one of his trademark Hawaiian shirts. In public he speaks in a slow and swampy-deep voice that seems to emanate from the very bottom of his six-foot-and-then-some frame. The mathematical genius and the inner joker seem to have reached a strange but solid equilibrium. In teaching his online course, he found that little mistakes and moments of goofiness that can screw up a live lecture actually work well online.
The Naked Future Page 16