The typical (Western) calendar of today arrays time linearly and hierarchically: months left to right, and within each month, weeks in a vertical stack top to bottom, with the days of each week arrayed left to right. Time lines within time lines organized by Western reading order, left to right, top to bottom. As we saw in Chapter Seven, we think about time as a line dotted with events, the way we think about space as a plane dotted with places.
Causality
Understanding causality crucially depends on time. Except for baffling arcane cases, causes precede effects. Understanding causality is fundamental to understanding ourselves and the world. If I let go of a cup, it will fall. Babies in high chairs love to practice that maneuver, to their delight and to the annoyance of their caretakers. Watching a sequence of events is the first step in understanding the causality, and controlling it the second. An excellent research method. Causality is crucial for understanding our own behavior, for understanding the behavior of others, for understanding phenomena in the world, for controlling the world.
EVENTS, PEOPLE, PLACES, AND THINGS
Depictions of events are widespread in archaeological and historic places all over the world, stelae in ancient Egypt, caves in Europe and South Africa, frescoes in Crete, temple friezes in Angor Wat, petroglyphs in Hawaii, arches and columns in ancient Rome, scrolls in China, tapestries in Europe, vases in Greece. They show people, tools, animals, and the like engaged in everyday activities, mythical events, or historic ones. A petroglyph found recently in Kashmir records a single striking event that occurred around five thousand years ago, a supernova, that is, an explosion of a star. To those on the ground, this apparently looked like two suns in the sky. The petroglyph depicts an entire scene: the two round heavenly bodies emitting rays above a hunting scene; to the left, a stick figure of a man aiming a bow and arrow at a large animal with antlers, to the right, another stick figure pointing skyward and a smaller animal, perhaps a dog.
FIGURE 8.9. Wall painting from the tomb of Menna in the Valley of the Kings in Egypt from 1420 to 1411 BCE, showing scenes of daily life in Egypt.
In contrast to the petroglyph depicting a single dramatic event, wall paintings in Egyptian tombs often portray repeated events, such as the one in Figure 8.9, from the tomb of Menna in the Valley of the Kings, dated back to 1420–1411 BCE. It is essentially a step-by-step diagram showing how to make bread, from sowing the wheat to the finished product. This single painting also portrays many of the different kinds of activities that constituted the complex society of Egypt in that early era, from agriculture to measurement and taxation. The depictions of the events are stylized, and the different events are distinguished by their backgrounds, the spatial groupings of the participants, and their tools, activities, and dress.
Recall from Chapter Two that these features of perception and action characterize different scenes and events. What will appear in later depictions are explicit frames or boxes that include all the features of one scene or event that separate one from another as well as move in a consistent direction in space.
Events take time to happen, some glacial, some instantaneous. Either way, depictions suspend events—they capture a critical moment or string of moments that epitomize the event. We distinguish event from action: events have a beginning, middle, and end. Actions don’t. Running is an action; running a race is an event.
The mind, too, freezes events into critical moments. The mind transforms the continuity of events to sequences of steps. When people are asked to segment videos of everyday events such as making a bed or putting a piece of furniture together, they easily segment the continuous set of actions into units and subunits, and agree with one another on where the segment boundaries are. The segment boundaries correspond to large visual changes in the action that beautifully correspond to accomplishments of goals and subgoals. Thus, the segments link perception with meaning so that one can be used to infer the other. Notably, time has only a background role in segmentation. When asked to describe what is happening, people report a sequence of actions on objects, for example, spreading the bottom sheet, tucking in the corners, spreading the top sheet, tucking in the bottom corners, putting on the blanket, stuffing the pillowcases. Those units and subunits may take different and variable amounts of time; what counts are the accomplishments. The event units are connected in a causal chain that accomplishes the overriding goal, making the bed. When people are asked to list the steps for making a bed or assembling a piece of furniture, they list the same steps.
EXPLANATIONS
It’s a short hop from describing events step-by-step to creating instructions to do them. The wall painting from the Valley of the Kings has both. Instructions, visual ones especially, should be simple and straightforward, but all too often, they are not. Many of us have spent frustrating days and nights trying to put together a kid’s bike or a new barbeque, swearing at the confusing instructions that came in the box. Many of the instructions that come in the box are just exploded diagrams of the objects. So many parts and nary a hint as to the order of assembling them, the sequence of actions.
The separate groups that had worked on map design joined forces. We thought we could do better, and even dared to think that we could discover design guidelines for creating effective instructions. Assembly instructions seemed a perfect paradigmatic task that would be familiar to most people and that most could accomplish, and one that represented myriad similar tasks that required following instructions. We chose an easy one, a piece of knock-down furniture, a TV cart. Not from IKEA, but similar. Hundreds of undergrads assembled the TV cart across the many variants of the experiments we ran, and we have a closet-full of broken ones to prove it. We think putting together a piece of furniture was an important part of their education.
The initial set of experiments had a similar format. Undergraduates first put together the TV cart using only the photograph on the box. Everyone succeeded, some more efficiently than others. Once they were experts, they were asked to act as designers, to create instructions so that others could easily put together the TV cart. Some were asked to use only diagrams, some only words, some a combination. We also assessed their spatial ability. One fascinating finding was that those high in spatial ability not only assembled more efficiently, they also produced better diagrams and even better verbal instructions. That is, people high in spatial ability seem to better understand the spatial transformations involved in assembly, and they are better able to articulate that understanding both in words and in diagrams.
The assembly diagrams of one high-spatial participant appear in Figure 8.10. As you can see, this set of diagrams leads you through step-by-step, they show the perspective of action, and they show how to perform each action using arrows and guidelines. Each new step is adding a new part. Not this one, but many others began with a “menu” of parts, like a recipe, and ended with sparkly lines, so the better diagrams had a narrative, told a story, there was a beginning, middle, and end. The diagrams of those low in spatial ability were flat, no perspective, no action. Sometimes nothing more than a menu of parts.
FIGURE 8.10. Instructions for assembling a TV cart made by a participant high in spatial ability.
Although the diagrams of the high spatials made the rules of thumb quite clear, we followed the Three Ps and went on to test preference and performance. Indeed, the qualities of the high-spatial diagrams—show each step, show action, show perspective—were preferred by a new group of participants asked to rate a large and variable set of instructions, high and low spatials alike. Finally, we held our breath and tested performance. We began with a new group of participants, only low spatials, as the high spatials were whizzes. Half saw the instructions that came in the box, which weren’t bad, but not nearly as good as ours. Those who used ours performed better and faster—whew!
Note the beauty of “our” (that is, the computer scientists’) instructions in Figure 8.11. They were created by a clever algorithm developed by the computer scientists. The al
gorithm began with a model of an object and decomposed it into parts. Then it created assembly instructions using the rules of thumb. Transparency in the diagrams affected the prescribed order of actions; the diagrams and the actions had to be designed together because they work together. The algorithm tackled other objects, including Legos, the gold standard for visual instructions, and produced the Lego instructions. I’d like to say whatever works for kids all over the world should work for grown-ups, especially grown-ups who used to be kids, but that would be reckless. Many of us put our kids to work assembling knock-down furniture—they’re better.
FIGURE 8.11. Instructions for assembling the TV cart produced by the algorithm following the cognitive design principles.
These, then, are three Rules of Thumb for assembly instructions:
Show each step: Each new part is a new step.
Show the actions: Use arrows and guidelines.
Show the perspective of action.
These rules of thumb have far more generality. With minor variations, they can be applied to visual explanations of how things happen as well as instructions for how to do something, explanations of how the heart works, how rain happens, how laws are passed, how revolutions take place. And you’ve probably already picked up that the same rules of thumb apply to verbal instructions and explanations as well as visual. Implicitly or explicitly, IKEA instructions follow these rules of thumb. You can search the Web to find a tantalizing video of a pair of robots using IKEA instructions to assemble an IKEA chair.
SEMANTICS OF DIAGRAMS
We are now ready to sketch a theory of diagrams. Diagrams use place in space and marks in space to convey meaning. Place in space is primarily left-right, up-down, center-periphery. Rows and columns. Marks in space are meaningful graphic forms, depictions, icons, words, symbols, dots, lines, boxes, arrows, circles, networks, and the like. Often, the marks are referred to as glyphs. Icons acquire meaning through resemblance or through figurative correspondences such as metaphor (scales of justice) and synecdoche (part representing whole, as in crown for king). Dots, boxes, lines, and similar simple abstract forms acquire meaning through their geometric and gestalt properties. They can get combined to create familiar forms like networks, flow charts, and decision trees.
We have already seen many of the common meanings combinations of these visual spatial elements express—space, number, time, creatures, objects, events, causality—to describe, explain, or retell.
Diderot
Depictions came early, but diagrams came late, perhaps surprising because representations of space, time, events, and number came early. The turning point for diagrams was a principled set of diagrams in the ambitious and remarkable oeuvre edited by Diderot and d’Alembert, the Encyclopedia, or a Systematic Dictionary of the Sciences, Arts, and Craft, affectionately known as L’Encyclopédie. It was printed and published in secret over some twenty years in the late eighteenth century under the shadow of the political and social upheavals that preceded the French Revolution. It is regarded as epitomizing eighteenth-century Enlightenment values. The subtitle of a recent book on the topic put them perfectly: reason, science, humanism, and progress.
FIGURE 8.12. A pin maker’s factory from L’Encyclopédie of Diderot and d’Alembert, late eighteenth century. Remarkably, the diagram does double duty: it shows the activities of the factory and it provides a visual explanation of what a diagram is.
More than three thousand diagrams were created for L’Encyclopédie and they were created systematically, many with a uniform design. They were elegant visual explanations of various industries, like the one in Figure 8.12, a pin-making factory.
The editors apparently thought the concept, diagram, needed explaining and invented a system for explaining the concept of diagram within the diagram. Each diagram, then, is both a diagram and a lesson in diagrammatology. You can see that this diagram has two parts, each enclosed in a box. The top part depicts a scene, a large room with light entering the windows, a door, a fireplace, and decorations on the wall. Inside, factory workers are carrying out their tasks with the appropriate instruments. The scale and locations of the figures and tools are proportional, the lighting is natural, the scene is in depth. Like a painting of a landscape or a scene in a home, art that would be familiar to readers. The bottom part is quite different, not at all a scene. It is flat like a wall; it shows only the tools, and they are lined up neatly in columns and rows. They are sized so viewers can see their parts, not proportional to their natural sizes. They are grouped by their function, not their locations in the factory. The light and shadows are inconsistent, meant to show the features of the tools, not to reflect natural light. There are labels and there is a key to information outside the diagram. Some measurements are added.
None of this is difficult for our eyes, but it might have puzzled eighteenth-century eyes. It’s a catalog or a web page. For eighteenth-century eyes, the diagram taught how to see and interpret diagrams by contrasting a diagram to a natural scene, which would have been familiar to eighteenth-century eyes.
Appropriately, L’Encyclopédie began with a set of tree diagrams, dividing knowledge into three branches: memory, reason, and imagination, and each of those further. Remember that trees are a special case of networks, networks with a single origin: large ideas and subdividing into smaller ones. That a monumental treatise of the Enlightenment regarded memory, reason, and imagination as the fundamental branches of knowledge is at once thrilling and at the same time daunting to the current enterprise, indeed to cognitive science as a whole.
As you have seen, Diderot’s diagrams used many of the ingredients of diagrams: place in space, array in space, marks in space, depictions in space, words and symbols all deliberately remodeled for communicative ends. What’s more, they explained what they had done and how to interpret it diagrammatically.
Place in space
Many of the spatial structures that reflect language and capture thought get put out into the world to represent thought. Like lines and boxes and networks. Anyone who’s looked at graphs, and there’s no escaping them, has noticed that time ordinarily goes left to right and that increases in any quantity go from down to up. This seemed to me more than a convention; it seemed to reflect how we think about time and how we think about more, of anything. As we saw earlier, people think about time as a line running horizontally. The direction of that line varies but often corresponds to reading and writing direction, a cultural artifact. Upward reflects the resources needed to counter gravity; more of just about anything does that—height, strength, health, wealth. Trees and elephants and people get stronger as they grow higher, healthy people stand erect, more money makes a higher pile. Good reasons why most good things go up. On good days, we’re on top of the world, on bad days, down in a slump. For the most part, neutral dimensions like time generally run horizontal and value-laden dimensions like health and wealth run vertical.
Spontaneous graphing: Vertical carries value, horizontal is neutral, reading order matters
If these correspondences are natural, time running horizontal in reading order and increases running upward, perhaps they would show up in young children and cross-culturally. Years ago, when I first became fascinated with the ways people put thought into the world, I found myself on sabbatical in Israel. Israel provided a unique opportunity as most of the population are schooled in languages that are read and written right to left, Hebrew and Arabic. On my return to the United States, we added English readers, for a sample size of more than twelve hundred, from four years old—prereaders—to college age. We asked them to array temporal, quantitative, and preference concepts, concepts that are increasingly remote from spatial thinking but that can be and are spatialized, even by preschoolers. We sat side-by-side with the children, asking them to place stickers on a square sheet of paper to represent the concepts. For example, for time, we would say, “Think about the times you eat breakfast, lunch, and dinner. I’ll put a sticker down for lunch and then you’ll p
ut down stickers for breakfast and dinner.” The experimenter would place the lunch sticker in the middle of the page, and then ask the child to place a sticker for the other meals, one-by-one, in counterbalanced order. For quantity, we asked about the amount of candy in handful, bagful, shelfful. For preference, about a food they were indifferent to or loved or hated. We asked children to map two examples for each concept.
First, we wanted to know whether children would map these abstract concepts to space. The answer was a robust yes. The kids arrayed stickers on the page with little hesitation and, for the most part, systematically. Then we wanted to know if they would see time, quantity, or preference as a dimension. If so, they would array the stickers on a line. If not, they might put the stickers on top of each other or all over the page. Placing stickers randomly rather than on a line suggests categorical thinking, an easier form of thinking, rather than dimensional thinking. Some of the four- and five-year-olds did exactly that, but most arrayed the stickers on a line, meaning they understood that the instances were ordered on an underlying dimension. They lined up time at a younger age than quantity, and quantity at a younger age than preference; that is, more abstract concepts were mapped on lines in space at later ages.
Mind in Motion Page 23