Bjork and Soderstrom would compare students’ scores on pretest-related questions to their scores on non-pretested ones on the cumulative final. The related questions were phrased differently but often had some of the same possible answers. For example, here’s a pair of related questions, one from the pretest and the next from the cumulative exam:
Which of the following is true of scientific explanations?
a. They are less likely to be verified by empirical observation than other types of explanations.
b. They are accepted because they come from a trusted source or authority figure.
c. They are accepted only provisionally.
d. In the face of evidence that is inconsistent with a scientific explanation, the evidence will be questioned.
e. All of the above are true about scientific explanations.
Which of the following is true of explanations based on belief?
a. They are more likely to be verified by empirical observation than other types of explanations.
b. They are accepted because they come from a trusted source or authority figure.
c. They are assumed to be true absolutely.
d. In the face of evidence that is inconsistent with an explanation based on belief, the belief will be questioned.
e. b and c above
The students tanked each pretest. Then they attended the relevant lecture a day or two later—in effect, getting the correct answers to the questions they’d just tried to answer. Pretesting is most helpful when people get prompt feedback (just as we did on our African capitals test).
Did those bombed tests make any difference in what the students remembered later? The cumulative exam, covering all three pretested lectures, would tell. Bjork and Soderstrom gave that exam two weeks after the last of the three lectures was presented, and it used the same format as the others: forty multiple-choice questions, each with five possible answers. Again, some of those exam questions were related to pretest ones and others were not. The result? Success. Bjork’s Psych 100B class scored about 10 percent higher on the related questions than on the unrelated ones. Not a slam dunk, 10 percent—but not bad for a first attempt. “The best way you could say it for now,” she told me, “is that on the basis of preliminary data, giving students a pretest on topics to be covered in a lecture improves their ability to answer related questions about those topics on a later final exam.” Even when students bomb a test, she said, they get an opportunity to see the vocabulary used in the coming lectures and get a sense of what kinds of questions and distinctions between concepts are important.
Pretesting is not an entirely new concept. We have all taken practice tests at one time or another as a way of building familiarity—and to questionable effect. Kids have been taking practice SATs for years, just as adults have taken practices MCATs and GMATs and LSATs. Yet the SAT and tests like it are general-knowledge exams, and the practice runs are primarily about reducing anxiety and giving us a feel for format and timing. The research that the Bjorks, Roediger, Kornell, Karpicke and others have done is different. Their testing effect—pre- or post-study—applies to learning the kind of concepts, terms, and vocabulary that form a specialized knowledge base, say of introductory chemistry, biblical analysis, or music theory.
In school, testing is still testing. That’s not going to change, not fundamentally. What is changing is our appreciation of what a test is. First, thanks to Gates, the Columbia researcher who studied recitation, it appeared to be at least equivalent to additional study: Answering does not only measure what you remember, it increases overall retention. Then, testing proved itself to be superior to additional study, in a broad variety of academic topics, and the same is likely true of things like music and dance, practicing from memory. Now we’re beginning to understand that some kinds of tests improve later learning—even if we do poorly on them.
Is it possible that one day teachers and professors will give “prefinals” on the first day of class? Hard to say. A prefinal for an intro class in Arabic or Chinese might be a wash, just because the notations and symbols and alphabet are entirely alien. My guess is that prefinals are likely to be much more useful in humanities courses and the social sciences, because in those courses our minds have some scaffolding of language to work with, before making a guess. “At this point, we don’t know what the ideal applications of pretesting are,” Robert Bjork told me. “It’s still a very new area.”
Besides, in this book we’re in the business of discovering what we can do for ourselves, in our own time. Here’s what I would say, based on my conversations with the Bjorks, Roediger, and others pushing the limits of retrieval practice: Testing—recitation, self-examination, pretesting, call it what you like—is an enormously powerful technique capable of much more than simply measuring knowledge. It vanquishes the fluency trap that causes so many of us to think that we’re poor test takers. It amplifies the value of our study time. And it gives us—in the case of pretesting—a detailed, specific preview of how we should begin to think about approaching a topic.
Testing has brought fear and self-loathing into so many hearts that changing its definition doesn’t come easily. There’s too much bad blood. Yet one way to do so is to think of the examination as merely one application of testing—one of many. Those applications remind me of what the great Argentine writer Jorge Luis Borges once said about his craft: “Writing long books is a laborious and impoverishing act of foolishness: expanding in five hundred pages an idea that could be perfectly explained in a few minutes. A better procedure is to pretend that those books already exist and to offer a summary, a commentary.”
Pretend that the book already exists. Pretend you already know. Pretend you already can play something by Sabicas, that you already inhaled the St. Crispin’s Day speech, that you have philosophy logic nailed to the door. Pretend you already are an expert and give a summary, a commentary—pretend and perform. That is the soul of self-examination: pretending you’re an expert, just to see what you’ve got. This goes well beyond taking a quick peek at the “summary questions” at the end of the history chapter before reading, though that’s a step in the right direction. Self-examination can be done at home. When working on guitar, I learn a few bars of a piece, slowly, painstakingly—then try to play it from memory several times in a row. When reading through a difficult scientific paper, I put it down after a couple times through and try to explain to someone what it says. If there’s no one there to listen (or pretend to listen), I say it out loud to myself, trying as hard as I can to quote from the paper its main points. Many teachers have said that you don’t really know a topic until you have to teach it, until you have to make it clear to someone else. Exactly right. One very effective way to think of self-examination is to say, “Okay, I’ve studied this stuff; now it’s time to tell my brother, or spouse, or teenage daughter what it all means.” If necessary, I write it down from memory. As coherently, succinctly, and clearly as I can.
Remember: These apparently simple attempts to communicate what you’ve learned, to yourself or others, are not merely a form of self-testing, in the conventional sense, but studying—the high-octane kind, 20 to 30 percent more powerful than if you continued sitting on your butt, staring at that outline. Better yet, those exercises will dispel the fluency illusion. They’ll expose what you don’t know, where you’re confused, what you’ve forgotten—and fast.
That’s ignorance of the best kind.
Part Three
Problem Solving
Chapter Six
The Upside of Distraction
The Role of Incubation in Problem Solving
School hits us with at least as many psychological tests as academic ones. Hallway rejection. Playground fights. Hurtful gossip, bad grades, cafeteria food. Yet at the top of that trauma list, for many of us, is the stand-up presentation: being onstage in front of the class, delivering a memorized speech about black holes or the French Resistance or Piltdown Man, and wishing that life had a fastforwa
rd button. I’m not proud to admit it, but I’m a charter member of that group. As a kid, I’d open my mouth to begin a presentation and the words would come out in a whisper.
I thought I’d moved beyond that long ago—until early one winter morning in 2011. I showed up at a middle school on the outskirts of New York City, expecting to give an informal talk to a class of twenty or thirty seventh graders about a mystery novel I’d written for kids, in which the clues are pre-algebra problems. When I arrived, however, I was ushered onto the stage of a large auditorium, a school staffer asking whether I needed any audiovisual equipment, computer connections, or PowerPoint. Uh, no. I sure didn’t. The truth was, I didn’t have a presentation at all. I had a couple of books under my arm and was prepared to answer a few questions about writing, nothing more. The auditorium was filling fast, with teachers herding their classes into rows. Apparently, this was a school-wide event.
I struggled to suppress panic. It crossed my mind to apologize and exit stage left, explaining that I simply wasn’t ready, there’d been some kind of mistake. But it was too late. The crowd was settling in and suddenly the school librarian was onstage, one hand raised, asking for quiet. She introduced me and stepped aside. It was show-time … and I was eleven years old again. My mind went blank. I looked out into a sea of young faces, expectant, curious, impatient. In the back rows I could see kids already squirming.
I needed time. Or a magic trick.
I had neither, so I decided to start with a puzzle. The one that came to mind is ancient, probably dating to the Arab mathematicians of the seventh century. More recently, scientists have used it to study creative problem solving, the ability to discover answers that aren’t intuitive or obvious. It’s easy to explain and accessible for anyone, certainly for middle school students. I noticed a blackboard toward the back of the stage, and I rolled it up into the light. I picked up a piece of chalk and drew six vertical pencils about six inches apart, like a row of fence posts:
“This is a very famous puzzle, and I promise: Any of you here can solve it,” I said. “Using these pencils, I want you to create four equilateral triangles, with one pencil forming the side of each triangle.” I reminded them what an equilateral triangle is, one with three equal sides:
“So: six pencils. Four triangles. Easy, right? Go.”
The fidgeting stopped. Suddenly, all eyes were on the blackboard. I could practically hear those mental circuits humming.
This is what psychologists call an insight problem, or more colloquially, an aha! problem. Why? Because your first idea for a solution usually doesn’t work … so you try a few variations … and get nowhere … and then you stare at the ceiling for a minute … and then you switch tacks, try something else … feel blocked again … try a totally different approach … and then … aha!—you see it. An insight problem, by definition, is one that requires a person to shift his or her perspective and view the problem in a novel way. The problems are like riddles, and there are long-running debates over whether our ability to crack them is related to IQ or creative and analytical skills. A knack for puzzles doesn’t necessarily make someone a good math, chemistry, or English student. The debate aside, I look at it this way: It sure doesn’t hurt. We need creative ways of thinking to crack any real problem, whether it’s in writing, math, or management. If the vault door doesn’t open after we’ve tried all our usual combinations, then we’ve got to come up with some others—or look for another way in.
I explained some of this in the auditorium that morning, as the kids stared at the board and whispered to one another. After five minutes or so, a few students ventured up to the blackboard to sketch out their ideas. None worked. The drawings were of triangles with smaller triangles crisscrossing inside, and the sides weren’t equal. Solid efforts all around, but nothing that opened the vault door.
At that point, the fidgeting started again, especially in the back rows. I continued with more of my shtick about math being like a mystery. That you need to make sure you’ve used all available information. That you should always chase down what seem like your stupidest ideas. That, if possible, you should try breaking the problem into smaller pieces. Still, I felt like I was starting to sound to them like the teachers in those old Charlie Brown movies (WAH-WAH WAH WAAH WAH), and the mental hum in the room began to dissipate. I needed another trick. I thought of another well-known insight problem and wrote it on the board beneath the chalk pencils:
SEQUENC_
“Okay, let’s take a break and try another one,” I told them. “Your only instruction for this one is to complete the sequence using any letter other than E.”
I consider this a more approachable puzzle than the triangle one, because there’s no scent of math in it. (Anything with geometric shapes or numbers instantly puts off an entire constituency of students who think they’re “not a math person”—or have been told as much.) The SEQUENC_ puzzle is one we all feel we can solve. I hoped not only to keep them engaged but also to draw them in deeper—put them in the right frame of mind to tackle the Pencil Problem. I could feel the difference in the crowd right away, too. There was a competitive vibe in the air, as if each kid in that audience sensed that this one was within his or her grasp and wanted to be the first to nail it. The teachers began to encourage them as well.
Concentrate, they said.
Think outside the box.
Quiet, you guys in the back.
Pay attention.
After a few more minutes, a girl near the front raised her hand and offered an answer in a voice that was barely audible, as if she was afraid to be wrong. She had it right, though. I had her come up to the board and fill in the answer—generating a chorus of Oh man! and You’re kidding me, that’s it? Such are insight problems, I told them. You have to let go of your first ideas, reexamine every detail you’re given, and try to think more expansively.
By this time I was in the fourth quarter of my presentation and still the Pencil Problem mocked them from the board. I had a couple hints up my sleeve, waiting for deployment, but I wanted to let a few more minutes pass before giving anything away. That’s when a boy in the back—the “Pay attention” district—raised his hand. “What about the number four and a triangle?” he said, holding up a diagram on a piece of paper that I couldn’t make out from where I was standing. I invited him up, sensing he had something. He walked onto the stage, drew a simple figure on the board, then looked at me and shrugged. It was a strange moment. The crowd was pulling for him, I could tell, but his solution was not the generally accepted one. Not even close. But it worked.
So it is with the investigation into creative problem solving. The research itself is out of place in the lab-centric world of psychology, and its conclusions look off-base, not in line with the usual advice we hear, to concentrate, block distractions, and think. But they work.
• • •
What is insight, anyway? When is the solution to a problem most likely to jump to mind, and why? What is happening in the mind when that flash of X-ray vision reveals an answer?
For much of our history, those questions have been fodder for poets, philosophers, and clerics. To Plato, thinking was a dynamic interaction between observation and argument, which produced “forms,” or ideas, that are closer to reality than the ever-changing things we see, hear, and perceive. To this, Aristotle added the language of logic, a system for moving from one proposition to another—the jay is a bird, and birds have feathers; thus, the jay must have feathers—to discover the essential definitions of things and how they relate. He supplied the vocabulary for what we now call deduction (top-down reasoning, from first principles) and induction (bottom-up, making generalizations based on careful observations), the very foundation of scientific inquiry. In the seventeenth century, Descartes argued that creative problem solving required a retreat inward, to an intellectual realm beyond the senses, where truths could surface like mermaids from the deep.
This kind of stuff is a feast for late night dorm room di
scussions, or intellectual jousting among doctoral students. It’s philosophy, focused on general principles and logical rules, on discovering “truth” and “essential properties.” It’s also perfectly useless for the student struggling with calculus, or the engineer trying to fix a software problem.
These are more immediate, everyday mental knots, and it was an English intellectual and educator who took the first steps toward answering the most relevant question: What actually happens when the mind is stuck on a problem—and then comes unstuck? What are the stages of solving a difficult problem, and when and how does the critical insight emerge?
Graham Wallas was known primarily for his theories about social advancement, and for cofounding the London School of Economics. In 1926, at the end of his career, he published The Art of Thought, a rambling meditation on learning and education that’s part memoir, part manifesto. In it, he tells personal stories, drops names, reprints favorite poems. He takes shots at rival intellectuals. He also conducts a wide-ranging analysis of what scientists, poets, novelists, and other creative thinkers, throughout history, had written about how their own insights came about.
How We Learn Page 11