FIGURE 5.1. Guidonian hand, a device for notating and conducting music, from the eleventh century and often still in use.
Continuing with music, not the piano, but singing. Another fascinating use of the hand was to represent musical notes and to lead a chorus, the Guidonian hand, so-called because it has been attributed to an eleventh-century monk, Guido of Arezzo. You can see a version in Figure 5.1.
Guido is also responsible for the way music is annotated to this day: do re mi fa so la ti do. Leading a chorus meant inscribing the notes on the fingers and palm, and pointing to the appropriate notes for the singers. Although the Guidonian hand fell out of use with the availability of printed music, it is making a comeback today.
The gestures used to calculate sums or to lead a chorus are not spontaneous; they are highly codified, even more than language. Nevertheless, like spontaneous gestures, they are intricately involved with thinking.
GESTURES AS SOCIAL GLUE
Conversation
That gestures are social glue is apparent from watching conversation. Gestures of the head, face, hands, and body keep conversation going. You say something and pause, looking at me. I nod my head to indicate I’m with you, or if I’m not, I raise my eyebrows or tilt my head and squint in puzzlement. When I’m ready to give you the floor, I lean back. Instead of answering a question I can’t answer, I might shrug my shoulders. If we don’t give this silent feedback to each other, conversation is awkward.
Collaboration
Conversation is one form of collaboration, but there are other explicit collaborations where gesture is key, notably where collaboration is facilitated by something in the world that can be pointed to or manipulated. Here’s one example: pairs of students were asked to find the best route to rescue wounded people after a hypothetical earthquake. They were given a map of the campus with locations of wounded people and blocked roads. They were asked to sketch a map of the best route. Some pairs worked side-by-side huddled over the same map. Their conversation tools were hands, map, and voice. Although they were deep in conversation, they rarely looked at each other’s faces. Instead they looked at their hands. Their hands took turns suggesting and editing routes on the map, annotated by the voice. The voice said things like: go here, turn there, now this way, not there—expressions that made sense only by looking at what the hands were doing on the map. Their gestures got abbreviated. Initially, participants would trace a whole route; as collaboration proceeded, they just pointed to the successive intersections. They picked up on each other’s gestures, a common phenomenon termed entrainment that is evident in words as well. Other pairs also worked side-by-side with identical maps but with a thin curtain between them. For those pairs, the only conversational tool was the voice. The first group of pairs, those who could gesture on a shared map, were more interactive, enjoyed the experience more, and produced better maps. The pairs separated by a curtain worked hard to agree on a route—students took this task seriously and enjoyed it. Yet, fully a third of the pairs came up with routes that differed substantially between partners.
Words can be and all too often are ambiguous, even words describing something as basic as space, the surroundings we carry with us at all times. Gestures, by contrast, are explicit. They show the exact places and trace the turns and paths. You already know what gestures predominated: point-like gestures for places, line-like gestures for paths. Zero-dimensional and one-dimensional. A third kind of gesture was also used, a two-dimensional sweep of the hand, to denote an area. The gestures didn’t work alone, they worked with an external representation of the task, in this case a map. But remember that we saw earlier that gestures can create a virtual external representation that serves as a platform for gestures. Whether actual or virtual, an external representation creates common ground and serves as a stage for the hands to reason and deliberate.
Design
Here’s another case, design. Small, experienced teams of designers were asked to redesign a device that detected properties of materials. Each team was seated around a table and given an engineering drawing and eventually a tangible model to work with. Needless to say, there was considerable use of gesture, on the drawings and the object. A few of the groups had “radical breaks,” sudden changes in design ideas. Insights. Flashbulb moments. These were accompanied by a cascade of new metaphors and new ideas, and notably an explosion of gestures, a switch from small ones on the table to large ones walking around the room enacting interacting with the object. Sometimes the narrated enactments were sketched rather than gestured, again showing the close relations between the two forms of expression, gesture and graphics.
Dance
The entire body participates in dance—dance is inherently and fully embodied. As such, it can readily represent itself. Yet choreographers and dancers have developed other embodied ways to represent dance when they are referring to dancing, techniques they call marking. Marking is often done with the fingers of one hand on the palm of another. The fingers dance as the legs would, showing a step to someone else. The whole body can also mark, performing what is essentially an embodied sketch of a dance or a segment of dance, for example, to sketch out position on the stage for lighting or to show in exaggerated fashion a dip of the torso or the stretch of a leg to a dancer. Surprisingly, sketching by marking an entire sequence of dance moves turns out to be a better way to remember the sequence than fully dancing the sequence. Sketching the sequence allows dancers to concentrate on the sequence, per se, rather than on both the sequence and the full expression of the movements.
Conducting
Now consider conducting, a vivid case for which gestures are the social glue. If the job of gestures on maps was straightforward, to express places and paths and areas, the job of gestures in conducting is straightforward but also nuanced and subtle. Among so many other things, conductors synchronize the tempo, oversee the dynamics of the music in space and in time, control the volume and intensity, and cue the entrances and exits of musicians. It is said that the left hand typically sets the tempo, and the right hand does the rest. But actual practice is far more complicated, and in no way systematized. Hand gestures vary wildly across conductors, many use a baton, some don’t. Many use their faces, backs, legs, and even their lungs, the rhythm of their breathing. Leonard Bernstein famously conducted Haydn, entirely with his head and plastic face, especially his eyebrows, with hardly a movement of his arms or body. Van Karajan conducted with his eyes closed. Esa-Pekka Salonen dances. The diversity of styles is astounding, all the more so because, according to some research, hundreds of years of social interactions of this sort should have converged on a common language.
Conductors conduct the audience as well as the orchestra. Audience perception of features of music such as expressiveness, articulation, and dynamics are enhanced when emphasized by conductors. Even though the contribution of a conductor is entirely visual, when audio is kept constant and conducting varied, the experience of audiences changes accordingly. For example, conductors could lead listeners to attend to overall melody or to a repeating theme (ostinato). When conductors emphasized overall melody, listeners were more likely to describe the piece as connected and regular, but when conductors emphasized repeating themes, listeners were more likely to describe the same piece as disconnected and irregular.
Although conductors only create visual performances, musicians create both sound performances and visual performances. Remarkably, sometimes the visual is more powerful than the sound. Here’s a dramatic case: determining the actual rankings of the top three finalists of a piano competition from audio alone, video alone, or both. Make your own prediction about which is best, but be prepared to be surprised. Both experts and novices were most correlated with the actual rankings when they had viewed only the video! This is, of course, despite their beliefs that the sound is more important than the visual.
Moving from music to art. Let me end the discussion of gestures as social glue with da Vinci’s iconic Last Supper. You ca
n find many copies in Google images. Better yet, go to Milan, to the church of Santa Maria delle Grazie and spend half an hour enjoying the real thing. A double session because the fifteen-minute standard session isn’t enough time. Follow the complex interactions of the bodies, the eyes, and the hands. You can see who’s talking to whom, how they are related, what they are referring to, how they are reacting. You feel the intensity of the interactions of the groups of guests and the calm detachment of Jesus. This is where we began, observing social interactions from afar, now exquisitely captured by da Vinci.
AFTERTHOUGHT
We can think with our bodies, but can we think great thoughts with our bodies? Yes! There’s da Vinci thinking through designs for flying machines and bridges and parachutes. There’s Einstein, imagining himself flying on a beam of light, the imagined flight that enabled his insights into spacetime. There are magicians who imagine making knots that undo themselves and surgeons who imagine one-handed knots that don’t come undone. Houdinis imagining escapes from locked boxes and thieves imagining cracking safes. Then there are choreographers, football coaches, fashion designers, military strategists, wrestlers, artists, engineers, actors, and mathematicians. All (or most) thinking great thoughts with their bodies.
CHAPTER SIX
Points, Lines, and Perspective: Space in Talk and Thought
In which we consider how linear language describes space, using a perspective, either an inside, body-centered perspective or an outside, world-centered perspective. For insider perspectives, we show that surprisingly taking another’s perspective is sometimes easier and more natural than taking your own.
The goal is not only the destination but also the path that takes you to it.
—PAUL ANDREU, from Lao Tzu
TALK AND THOUGHT
Talking isn’t thinking. Talking can reveal thinking and talking can change thinking, but it shouldn’t be confused with thinking. Talk is only one way to express thought; there are others. Laughing, gasping, and screaming emerge from the mouth bursting with meaning but are not talk. The face, the hands, the body—all express thought. As do sketches and diagrams and models and arrangements of things in space. Then there are times that our thoughts get stuck in our heads—they can’t seem to find their way out. We are speechless, at a loss for words.
But talk can take us far; it’s where we’ll begin. Talk is words, one after another. Words are symbols, arbitrary and indirect expressions of meaning, expressions of meaning that are highly condensed. There simply aren’t enough words. Words do not show their meanings in the ways that facial expressions, gestures, and depictions do. Unfettered by meaningful perceptual features, words reach effortlessly to abstraction. Because of the abstractness of words, there are so many more meanings than words. Talk still provides a window to thought. That window is a narrow one but, under scrutiny, a revealing one.
We begin with talk about space. Talking about space is quite like exploring space. Talking about space takes us on a journey from one place to another. Space should be one of the easiest things to talk about—we spend our lives in it. Navigating in space is essential to survival. We have nouns for places and kinds of places, adjectives for describing them, verbs to express exploring them, prepositions to convey spatial relations within and between them. Despite the solidity and tangibility and ubiquity of space, talk about space is subtle and prone to ambiguity. Watch my words and yours. Talk and thought about space serve as foundations for talk and thought about so much else. That foundation is embedded in the brain. If you missed that, go back to Chapter Three, the Sixth Law of Cognition: Spatial thinking is the foundation of abstract thought. We saw the foundation in the brain; now we show it in talk.
TALK AND THOUGHT ABOUT SPACE
Perspective
We start with an ordinary, everyday request: someone asks you where something is. Your bike. Their keys. Cell phone. Glasses. Your house or office. Think how you would answer. “On the kitchen table” or “leaning against the right side of my house.” But you can’t give those answers without being sure that you and that someone have a shared understanding of which kitchen table and house, and where they are located. That is, you need a shared perspective. You need a common way of looking at the world as a starting point. In fact, it’s likely you can’t talk about anything without taking a perspective, implicit or explicit, and making sure that perspective is shared (n.b.: shared does not mean agreed). A number of different disciplines arrived at the same two basic perspectives if by different routes: egocentric, that is, with respect to a specific body, typically yours, and allocentric, that is, with respect to the surrounding world. The most familiar allocentric perspective is north-south-east-west.
Egocentric perspective. Let’s say a friend wants to borrow your bike and you’re not home, so you need to tell your friend where your bike is. You might say, “If you’re facing my house, you’ll see the bike leaning against the right side of the house.” Here the perspective is explicit, it’s facing the house. If you had said, “The bike’s to the right of the house” it wouldn’t be clear whether right was from the point of view of someone facing the house or someone leaving the house. You also need a frame of reference. Here, it’s your friend’s egocentric perspective, your friend’s right-left front-back above-below, those imaginary axes extending from the body that we talked about in Chapter Three.
But that presupposes that your friend knows where your house is. Explaining how to find your house may require a longer description. Like all spatial descriptions, that one has to begin with a shared perspective, one that is known to your friend, one your friend can take. You might say something like, “From your hotel, turn right onto Cowper. Go down Cowper about half a mile until you get to Embarcadero. Turn right on Embarcadero, then left at about the third light onto El Camino. Go down about half a mile. Turn right onto Stanford Avenue…” Here, the perspective is still the egocentric perspective of your friend, the traveler. Although “your” viewpoint keeps changing, the changes are explicit and the frame of reference is always “your” body, your left-right front-back head-feet.
For good reason, perspective and reference frames are core concepts in linguistics, psychology, geography, and more—history, literature, art, everywhere you look. The perspective illustrated above, that of an observer embedded in a space, and the frame of reference described above, extensions of the axes of the body, are typically called egocentric. The center, the starting point, of the perspective is the ego. Ego keeps moving along the route, and the reference point moves along with ego. Now for a technical term, from linguistics. That center, ego’s position in space and time, is called the deictic center. Deictic, right, Greek again, “to point or show.” Knowing the deictic center is essential for understanding words like here, now, there, next, this, and that. Put differently, grasping those terms depends on knowing ego’s location in space and time. No wonder that the first question we usually ask the caller on the cell phone, “Where are you?”
The extended use of an egocentric perspective to give directions, to tell someone how to get from A to B, is often called a route perspective. Route descriptions are procedures, explanations, directions for getting from A to B. They take you from where you are to where you need to be in space. But also in time. Writers address their readers as you when they want to put you smack in the middle of a scene, when they want you to experience the action as if you were there, as if the action were unfolding step-by-step before your eyes.
Route perspectives can be used not just to give directions but also to give an overall impression of an environment, to describe the layout of your apartment or locate the major landmarks of a city. Here’s one of midtown Manhattan. “Start with Lincoln Center at your back and make a right onto Broadway. Go down about seven blocks until you get to Columbus Circle at 59th Street. You’ll see the statue of Columbus in the center, surrounded by fountains. To your right, you’ll see Time-Warner Center and on your left the edge of Central Park. Turn left onto 59th St
reet and walk along the park until its end, at Fifth Avenue. Go right on Fifth Avenue and continue to 53rd Street. Go right again on 53rd and you’ll see the Museum of Modern Art half-way up the street on your right.” I’ve taken you from landmark to landmark, place to place, point to point, along a path. Travel guides do this more vividly. Take my word for it, it’s a nice walk.
You probably noticed that route descriptions are built from pieces, like Lego blocks. And like Legos, each piece has two parts, a link to the previous part and a link to the subsequent one. The parts of each route segment are a path and a place. The path is a link from the previous part and the place becomes a link to the subsequent one. Just as Lego parts can be connected to create smaller or larger edifices, so route parts can be connected to create smaller or larger descriptions. Paths are the way you get from place to place, the actions you take to get from place to place. At places, actions can change; for routes, the change is typically change of direction. Places, choice points, are often street intersections but could be other landmarks, churches or piazzas, subway stops or restaurants. Another way to think about paths and places: actions and consequences of actions, the places actions bring you to. “Make a right onto Broadway.” “Turn left onto 59th Street.” Segments of places and paths can be added, as many as needed, like links in a chain. Points and links, places and paths, form the skeleton of routes.
Mind in Motion Page 15