Mind in Motion

by Barbara Tversky

  One fundamental role for points is to bring the world, the here and the now, into the conversation. Points simultaneously direct attention to something in the world and refer to that something in the world. Airplane . Eat . Go . Daddy . So much conversation, especially with children, is about the here and now. But pointing to bring something in the world into the conversation isn’t just for children. Adults point: to show someone which way to go, to indicate which dessert they want, to designate whose turn is next.

  Points are often regarded as the simplest gestures. What could be simpler than extending a finger in the direction of the focus of one’s thought? That’s their meaning, right there in front of our eyes. Babies point early and proficiently. But simple points are not. Suppose while talking I point to a book. I could be referring to any old book, to an object that could be used as a door stop, to a recent purchase, to something a friend forgot, or to that specific book. If to that book, I could be referring to its title, to its contents, to its author, to the pleasure it brought me, to its influence, to its size or its cover or to countless other features associated with the book. Context can clarify.

  To complicate things further, pointing is not a single gesture. It needn’t even use the pointer finger. We can point with a finger or a hand or our heads or our shoulders or even with our eyes. How we point varies. Perhaps we’ve been told that pointing isn’t polite or perhaps pointing with the head or the eyes is more private and can’t easily be seen by others. A sweep of the eyes toward the door can signal a companion to see what’s going on there or that it’s time to leave. How we point depends on so much, on who the point is for, on what is pointed to, on the surrounding context, physical, social, and conversational.

  Even odder, points can be directed at something that isn’t there at all. A nod in the direction of the place of someone who has left the room or a dish that has been removed from the table can refer to that absent someone or that absent dish. But more than that, I can set up an imaginary world with points, a remembered world or a completely hypothetical world, a concrete world or an abstract one. And I can continue to point to the imaginary things I’ve arranged in my imaginary world, even rearranging them using moving points. Setting up an imaginary world and animating it is in fact a feature of American Sign Language.

  Iconic gestures depict. They show properties of objects, spaces, or actions. The prototypic iconic gesture is the “big fish” gesture, the drawing out of the hands to indicate the impressive length of the fish that was caught or that got away. Iconic gestures do not and cannot exhibit all the features of an object or action. The big fish gesture shows the length and horizontality of the fish, but it doesn’t show the shape of the fish or its swimming motion. Iconic gestures can also represent actions as in “he walked into the room looking as if he owned the place” while swaggering and strutting.

  Metaphoric gestures express depictable but nonliteral properties or abstract concepts. There are big fish and there are big ideas. Of course, ideas can’t literally be big. Ideas might be big because they are inflated or because they encompass many other ideas or because they have many implications. A gesture accompanying big idea would be different from one accompanying big fish. Fish have shapes and orientations; ideas don’t. How do you indicate that something, an idea, can be regarded as an entity but that it has no particular shape or orientation? A sphere. So, a big idea is more likely to be conceived of as something rounded rather than something elongated like a fish. For big idea, the fingers might be curved as if holding a ball. Actions can also serve as metaphoric gestures. A head bobbing this way and that can depict a person who bounces from idea to idea. A flattened hand wobbling up and down signals uncertainty, like a teeter-totter.

  Metaphors of all kinds permeate our thought and our talk, and our gestures as well. One reason metaphors work is that they use something that is familiar to represent something that is unfamiliar, something that is concrete to represent something that is abstract, something understood to represent something that is not. Many metaphors are so common—often called, metaphorically of course, “dead”—that we don’t notice them as such: the heart pumps, the brain computes, life is a journey, political candidates are at war. During a recent presidential campaign, a political commentator remarked that one of the major candidates had a driver’s license, the other not even a learner’s permit, and a third-party candidate was without a car. Shakespeare was the master metaphor maker: life’s a stage, Juliet’s the sun, life’s a web. Of course, metaphors do not transfer all senses to their targets. What transfers from web to life is a complex network, a network of events and relationships, not of strands of filament extruded from the nether parts of a spider. Similarly, Juliet lights up Romeo, but she isn’t a glowing ball in the heavens. So, too, for metaphoric gestures: only some features transfer. Gesture can make explicit which ones.

  GESTURES REVEAL THOUGHT

  My husband was a paratrooper in the Israeli army. One of the training exercises was to be dropped alone in the desert in darkness without a map. You found your way back or… He had an uncanny sense of direction. Many years later in more benign environments that were paved and well lit on the rare occasions when I was the driver and he was the navigator, he would tell me to “turn right” and point left. Or vice versa. It didn’t matter what language he was speaking. Since the body is faster than the mind, I knew to go with his hand, not the words. The relationship between words and action is arbitrary, but the relationship of pointing to action is direct, it’s in the body and the world. You point the way you want to go. Sometimes people’s gestures contradict their speech. In those cases, pay close attention to their gestures.

  This is true in spades for children because they are often less adept at explaining in words. Here’s an example from young children in a standard Piagetian conservation task. Two equal rows of checkers are lined up in front of the child. The experimenter spreads out one row and asks, “Are there more checkers now [pointing to the widened row] or are they just the same?” The experimenter also asks why. Very young children say more; older ones correctly say same. But some children say one thing and gesture another; the researchers called these discordances mismatches. For example, a kid might say more but make a gesture pointing to corresponding pairs of checkers in the two rows. That one-to-one gesture suggests the kid is on the cusp of grasping conservation. In this case, the mismatched gestures don’t contradict the words, as in the case of my husband and left and right. In many mismatches, the gestures and the words simply carry different information.

  The same happens in school-aged children learning to solve arithmetic equations. Some kids calculate incorrectly but point to both sides of the equation with a V gesture, suggesting nascent understanding that the two sides of an equation must be equal. Significantly, children’s mismatches predicted leaps in understanding in both cases. That is, a child pointing to two sides of an equation will soon understand that the two sides of an equation must be equal or that stretching the row of checkers doesn’t change the quantity of checkers. What’s more, teachers seem to pick up the discrepancies between words and gestures and use them in teaching, by helping the child articulate their understanding. Teachers sense that these are teachable moments and give more instruction to children who produce mismatches.

  Students’ gestures provide other information that is helpful to teachers, notably their problem-solving strategies. When children are asked to gesture as they explain how they solve equations, their gestures reveal strategies not explained in speech, for example, which numbers in the equations they are summing. It’s a bit like asking students to show their work. They are then more likely to benefit from instruction.

  Conversely, children learn better when their teachers provide two different problem-solving strategies, one in their words and another in their gestures, than when their gestures and speech match or when both strategies are conveyed by speech.

  GESTURES PUT
THOUGHT ON A STAGE

  Gestures reveal thought, often far better than words do. This turns out to be especially important for really big thoughts, like the (Kantian) Big Three: space, time, and causality. Each of these is a multifaceted concept that can be spatialized, and spatialized in different ways. Setting up a schematic space of ideas is one of the great powers of gesture. Much research on gesture has analyzed single gestures focusing on hand form or simply counted gestures. Insightful as that work has been, that narrow focus overlooks the force of an integrated sequence of meaningful gestures that put ideas on a stage, poised to interact.

  What’s missing from the Kantian Big Three is emotion. Emotion was not one of Kant’s fundamental a prioris, space, time, and causality. If space, time, and causality are successively more abstract, then emotion is even more so, though not on the same conceptual continuum. Emotion is on its own conceptual continuum. Or continua. If expressing space, time, and causality uses sequences of integrated gestures, usually of the hands, emotion often takes only a single gesture, typically of the face. Still, emotion is part of every perception and every thought, and this cannot be forgotten.

  Countless nuanced and nameless emotions can be expressed by the body and the face, even just the eyes and eyebrows. Raised eyebrow, either as action or an expression, has become a synonym for skepticism. We discussed emotion in Chapter Two, when we populated the world around the body. Here, we only give lip service to emotion. And the lips play roles, they smile and frown and yawn and pucker. Words emit from them. Suffice it to say that we often experience the emotions of others the way we experience actions of others directly, through mirroring by the body and the brain.

  Space. Using space to represent space is a no-brainer. Nevertheless, if you’re in psychology nobody trusts you unless you do an experiment. So, we did. We brought people into the lab, gave them schematic maps to study, and asked them to describe the environments represented in the maps to a video camera so that someone watching the video would know where everything was. As expected, most (but not all) gestured. Many produced a long string of integrated gestures that laid out the places and paths in the environment in a spatial array, some on a virtual vertical blackboard, some on a virtual horizontal table. Predominating the gestures were lines for paths and points for places.

  Now time. Time is usually abstracted to a single dimension, a line. But which one? Depending on the language and the situation, the line might be gestured from left to right or right to left, it might go sagittally from the front of the body to behind the body or vice versa. The direction depends on how time is conceived. In some languages, the future is in front because conceptually we are moving toward it or it is coming at us. In other languages, the past is in front because it is known and the future is behind because it cannot yet be seen. In Mandarin, time might be gestured vertically, earlier up and later down, like a calendar. Arraying time from left to right or right to left is convenient on a page or in some social situations where the sagittal front/back is complicated to represent. Whether time goes from left to right or vice versa appears to depend largely on reading/writing order.

  Causality. Causality is much, much harder; there are so many different kinds of causes and so many of them are invisible. But many causes and consequences of causes are actions, inviting iconic gestures. Back to the lab to see how people gesture when they explain causal systems. In one experiment, students studied the rock cycle or the workings of the heart and then made a video explaining the system. Typically, they first used gestures to create a large virtual diagram that located the parts of the systems, much like the way people create a map of locations in space or a timeline of events using gestures. For causality, gestures can do more than map in space or time. Gestures were used to show the actions of the parts of the system and the causal chain of actions in the system. Thus, gestures do double duty in representing causality, making them all the more important in explanations of causality.

  We’ll stop here, with examples from the Big Three, but it should be clear that this is only the beginning of ways that gestures can put thought onto a stage. But gestures representing space, time, and causality do far more than put thought on a stage—they have the power to change thought in those who make them as well as those who see them.

  Second General Fact Worth Remembering: Representations created by hands and by words are wildly different.

  Presumably by now I have convinced you that people spontaneously gesture and that gestures can express a multitude of different kinds of ideas more directly than words can. Yes, all over the world. And, yes, there are also cultural differences. As for just about everything. Now I need to convince you that gestures make a difference, that they are effective and effective beyond words in communicating both to others and to one’s self. Fortunately, there’s plenty of evidence for both. And that research gives more insight into how gestures work.

  GESTURES HELP US TALK

  Try this. Sit on your hands. Then explain out loud how to get from your house to the supermarket, train station, your office or school. This isn’t just a thought experiment; it works in highly controlled laboratory experiments. When people are asked to explain or describe spatial relations while sitting on their hands, they have trouble speaking. They can’t find words.

  People blind from birth, both children and adults, gesture, even when speaking to each other. They have never seen gestures nor have their conversation partners. They seem to gesture for themselves. Gesturing by people who are blind, as for the people with sight in the previous experiments, seems to help them speak. But it turns out it isn’t just word finding people have trouble with when they can’t use their hands. Preventing gesturing doesn’t just disrupt speaking, it disrupts thinking.

  GESTURES HELP US THINK

  There is a perhaps apocryphal story about the venerable poet Wallace Stevens, who walked to his work in an insurance company. As he walked, he wrote poetry, in rhythm with his thoughts. The story is that when he revised a line, he walked backward to where that line had begun in his mind, and then forward again as he rewrote.

  Now from poetry to a far more mundane activity of the mind, counting. Try counting a bunch of pennies sprawled on a table without pointing to or moving each one as it is counted. Children are taught to point to each object in turn as they count, and doing so makes counting more accurate and faster. When adults’ hands are tied as they count, they count with their heads. And, undoubtedly, if the head were immobilized, people would count with their eyes. Pointing while counting allows keeping track of the count. Is pointing to count an action or a gesture? It seems to be both.

  Making the case that gestures help thinking requires gestures that represent thought. And more: that people gesture when they are thinking, but not talking, that when they do so, they think better, and that preventing gesturing disrupts thinking. There’s an added bonus. Seeing the kinds of gestures people make when they are thinking also reveals the thinking and does so directly, without the use of machines that peer into the brain.

  To do all that means going into the laboratory. We began a research program in which people were alone in a closed room and given problems to solve or complex descriptions to remember. We know that people gesture when they talk about such things, but in our studies, there was no one to talk to.

  First, we gave people problems to solve. Here’s one of them: There’s a row of six glasses. The three on the left are empty, the three on the right are full. By moving only one glass, change the configuration to empty-full empty-full empty-full.

  Did you figure it out? While thinking about that problem, the majority of students gestured. Their gestures represented the problem, three empty glasses, three full glasses in a row, but in different ways. Some put out three fingers on each hand, side-by-side. Others used an index finger to lay out two separate groups of three along a row on the table. Either way, their gestures represented the problem. These gestures aren’t single gestures like deictic or iconic or metaphoric ges
tures. They are much more; they are a coordinated sequence of gestures that form a spatial representation of the problem, a virtual diagram of the problem. That was an interesting finding in and of itself. But there was another finding, much more surprising. The people who gestured were more likely to solve the six glasses problem than those who didn’t gesture. Why should gesturing help problem solving?

  Before trying to understand why gesturing helps problem solving, we need to know how general the phenomenon is: Will people gesture to understand and learn other kinds of information? Because it’s known that people gesture when they describe environments, we turned to those, roads and landmarks in a small town or the configuration of various exercise rooms in a gym. Environments are inherently spatial, but they are abstracted both in the mind and on the page—into the paths and places of sketch maps. Dots and lines. Would people, alone in a room, gesture to represent and remember descriptions of environments, and would their gestures form sketch maps? The answer to both questions is yes.

  Just as they did while reading problems to solve, most (but not all) people gestured while reading spatial descriptions to remember. Whether or not they gestured didn’t depend on whether the environment was indoors or outdoors, large or small. It didn’t matter if the description took a perspective from above or from within the environment. Just as for the six glasses problem, people’s gestures made virtual sketches of the environments, but their styles of gesturing differed. Some gestured on the table, some in the air, some under the table. Some traced lines or pointed with the index finger, some used an entire hand. But the gestures were similar at a semantic level. Everyone used line-like gestures to represent paths and point-like gestures to represent landmarks. Other features of the environments, like parks or schools or weight rooms or pools, were rarely represented. Only the skeleton, much like sketch maps.