Understanding Context

by Andrew Hinton

  And, of course, action is required for this information pickup. In nature, a tree branch has structural properties that we detect through the way light and other energy interact with the substance and texture of the branch. We are always in motion and perceiving the branch from multiple angles. Even if we’re standing still, our bodies are breathing, our eyes shifting, and able to detect the way shadows and light shift in our view. This is an unconscious, brain-body-environment dynamic that results in our bodies’ detection of affording properties: whether a branch is a good size to fit the hand, and whether it’s top-heavy or just right for use as a cane or club.

  Perception evolved in a natural world full of affordances, but our built environment also has these environmental properties. Stairs, such as those shown in Figure 4-9, are made of surfaces and materials that have affordances when arranged into steps. Those affordances, when interacting with energy such as light and vibration, uniquely structure that energy to create information that our bodies detect.

  Our bodies require no explanation of how stairs work, because the information our bodies need is intrinsic in the structure of the stairs. Just as a hand pressed in clay shapes the clay into structure identical to the hand, affordance shapes energy in ways that accurately convey information about the affordance. Organisms rely on this uniquely structured energy to “pick up” information relevant to what the organism can do.

  Figure 4-9. Stairs at City Lights Bookstore in San Francisco (photo by author)

  For a person who never encountered stairs before, there might be some question as to why climbing up their incline would be desirable, but the perceiver’s body would pick up the fact that it could use them to go upward either way. Humans evolved among surfaces that varied in height, so our bodies have properties that are complementary with the affordances of such surface arrangements.

  For most of us, this is a counterintuitive idea, because we’re so used to thinking in terms of brain-first cognition. In addition, some of the specifics of exactly what affordance is and how Gibson meant it are still being worked out among academics. The principle of affordance was always a sort of work in progress for Gibson, and actually emerged later in his work than some of his other concepts.[48] He developed it as an answer to Gestalt psychology theories about how we seem to perceive the meaning of something as readily as we perceive its physical properties; rather than splitting these two kinds of meaning apart, he wanted to unify the dualism into one thing: affordance.[49]

  We won’t be exploring all the various shades of theory involved with affordance scholarship. But, because there is so much talk of affordances in design circles, I think it’s valuable to establish some basic assumptions we will be working from, based on Gibson’s own work.

  Affordance is a revolutionary idea

  Gibson claimed in 1979 that affordance is “a radical departure from existing theories of value and meaning.”[50]And in some ways, it is still just as radical. As an empiricist, Gibson was interested in routing around the cultural orthodoxies and interpretive ideas we layer onto our surroundings; he wanted to start with raw facts about how nature works. If we take Gibson’s theory of affordances seriously on its own terms, we have to take seriously the whole of his ecological system, not pick and choose parts of it to bolt onto mainstream, brain-first theory.[51]

  Affordances are value-neutral

  In design circles, there is sometimes mention of anti-affordances. But in Gibson’s framework there’s no need for that idea. Affordances offer the animal opportunities for action, in his words, “for good or ill.” He writes of affordances having “positive” or “negative” effects, explaining that some affordances are inconvenient or even dangerous for a given agent, but they are still affordances.[52] An affordance isn’t always good from the perspective of a particular organism. Water affords drowning for a terrestrial mammal, but it affords movement and respiration for a fish.[53] Fire affords comforting heat, light, and the ability to cook food, but it also affords injury by burning, or destruction of property. Affordance is about the structural and chemical properties that involve relationships between elements in the environment, some of which happen to be human beings. Separating what a structure affords from the effect on the perceiver’s self-interests helps us to remember that not all situations are the same from one perceiver to the next.

  Perception of affordance information comes first; our ideas about it come later

  Gibson argued that when we perceive something, we are not constructing the perception in our brains based on preexisting abstract ideas. He states, “You do not have to classify and label things in order to perceive what they afford.” When I pick up a fork to eat food, my brain isn’t first considering the fork’s form and matching it to a category of eating utensils and then telling my arm it’s OK to use the fork. The fork affords the stabbing of bites of food and bringing them to my mouth; my body extends its abilities by using the fork as a multipointed extension of my arm. That is, my body appropriates the fork based on its structure, not its category. The facts that it is a dinner fork and part of a set of flatware are based on categories that emerge later, from personal experience and social convention.

  Affordances exist in the environment whether they are perceived or not

  One contemporary theoretical stance argues that we do not perceive real things in the world, but only our brain’s ideas and representations of them. Gibson strongly disagreed, insisting that we couldn’t perceive anything unless there were a real, physical, and measurable relationship between the things in the world and our bodies. He allowed room for a sort of “life of the mind” that might, in a sense, slosh about atop these real foundations, but it exists only because it was able to emerge from physical coupling between creature and world.

  The properties that give something affordance exist whether they are perceived in the moment or not; they are latent possibilities for action. Affordance is required for perception; but an affordance doesn’t have to be perceived to exist. I might not be able to see the stairs around the corner in a building, but that doesn’t mean the stairs’ ability to support climbing doesn’t exist. This idea complicates the commonly taught concept that a mental model drives behavior. Affordance means the information we need for action does not have to be “mental” and is actually in the structures of the environment. I perceive and use the stairs not because I have a mental model of them; no model is needed because all the information necessary is intrinsic to the shape and substance of the stairs. Prior learned experience might influence my usage in some way, but that’s in addition to perception, not perception itself.

  Affordances are there, whether they are perceived accurately or not

  A Venus flytrap exists because it can get nutrition. And it gets nutrition because it “tricks” its prey into thinking it is a source of food for the prey, rather than the other way around. For a fly, the affordance of the flytrap is being caught, dissolved, and absorbed by a plant. What the fly perceives, until it is often too late, is “food.” Likewise, we might perceive ground where there is actually quicksand, or a tree branch that is actually a snake. Perception is of the information created by the affordance, not the affordance itself. This is an important distinction that has often been misunderstood in design practice, leading to convoluted discussions of “perceived” versus “actual” affordances. The affordance is a property of the object, not the perception of it.

  Affording information is always in a context of other information

  No single affordance exists by itself; it’s always nested within a broader context of other affording structures. For example, even if we claim to “add an affordance” by attaching a handle to a hammer head, the hammer is useful only insofar as it can bang on things that need to be banged upon. Stairs afford climbing, but they’re always part of some surrounding environment that affords other actions, such as floors and landings, walls, handrails, and whatever is in the rooms the stairs connect. In digital devices, the physical buttons and
switches mean nothing on their own, physically, other than “pushable” or “flippable”—what they actually affect when invoked is perceived only contextually.

  Affordances are learned

  Human infants are not born understanding how to use stairs. Even if we allow that perception couples with the information of the stairs’ surfaces to detect they are solid, flat, and go upward, we still have to learn how and why to use them with any degree of facility. Infants and toddlers not only inspect the stairs themselves, but also were likely carried up them by caregivers and saw others walking or running up and down them long before trying out the stairs for themselves. Learning how to use the environment happens in a densely textured context of social and physical experience. This is true of everything we take for granted in our environment, down to the simplest shapes and surfaces. We learned how to use it all, whether we remember learning it or not. In digital design, there is talk of natural interfaces and intuitive designs. What those phrases are really getting at is whether an interface or environment has information for action that has already been learned by its users. When designing objects and places for humans, we generally should assume that no affordance is natural. We should instead ask: is this structure’s affordance more or less conventional or learnable—keeping in mind that “learnable” is often dependent upon how the affordance builds on established convention.

  Directly Perceived versus Indirectly Meaningful

  The Gibsons continued to expand their theories into how affordances function underneath complex cultural structures, such as language, cinema, and whole social systems. Other scholars have since continued to apply affordance theory to understanding all sorts of information. Likewise, for designers of digital interfaces, affordance has become a tool for asking questions about what an interface offers the user for taking action. Is an on-screen item a button or link? Is it movable? Or is it just decoration or background? Is a touch target too small to engage? Can a user discover a feature, or is it hidden? The way affordance is discussed in these questions tends to be inexact and muddled. That’s partly because, even among design theorists and practitioners, affordance has a long and muddled history. There are good reasons for the confusion, and they have to do with the differences between how we perceive physical things versus how we interpret the meaning of language or simulated objects.

  The scene from Rosemary’s Baby serves as an apt example of this simulated-object issue. In the scene depicted in Figure 4-7, moviegoers can’t see Minnie’s mouth moving behind the door frame, even if they shift to the right in their seats, because the information on the screen only simulates what it portrays. There is no real door frame or bedroom that a viewer can perceive more richly via bodily movement. If there were, the door and rooms would have affordances that create directly specifying information for bodies to pick up, informing the body that moving further right will continue revealing more of Minnie’s actions. Of course, some audience members tried this, but calibrated by stopping as soon as their perception picked up that nothing was changing. Then, there were undoubtedly nervous titters in the crowd—how silly that we tried to see more! As we will see later, we tend to begin to interact with information this way, based on what our bodies assume it will give us, even if that information is tricking us or simulating something else, as in digital interfaces.

  Perception might be momentarily fooled by the movie, but the only affordance actually at work is what is produced by a projector, film, and the reflective surface of the screen. The projector, film, and screen are quite real and afford the viewing of the projected light, but that’s where affordance stops. The way the audience interprets the meaning of the shapes, colors, and shadows simulated by that reflected light is a different sort of experience than being in an actual room and looking through an actual door.

  In his ecological framework, Gibson refers to any surface on which we show communicative information as a “display.” This includes paintings, sketches, photographs, scrolls, clay tablets, projected images, and even sculptures. To Gibson, a display is “a surface that has been shaped or processed so as to exhibit information for more than just the surface itself.”[54] Like a smartphone screen, a surface with writing on it has no intrinsic meaning outside of its surface’s physical information; but we aren’t interested in the surface so much as what we interpret from the writing. Gibson refers to the knowledge one can gain from these information artifacts as mediated or indirect—that is, compared to direct physical information pickup, these provide information via a medium.

  Depending on where you read about affordances, you might see Affordance used to explain this sort of mediated, indirectly meaningful information. However, for the sake of clarity, I will be specifying Affordance as that which creates information about itself, and I will not be using the term for information that is about something beyond the affordance. Images, words, digital interfaces—these things all provide information, but the ultimately relevant meaning we take away from them is not intrinsic. It is interpreted, based on convention or abstraction. This is a complex point to grasp, but don’t worry if it isn’t clear just yet. We will be contemplating it together even more in many chapters to come.

  This approach is roughly similar to that found in the more recent work of Don Norman, who is most responsible for introducing the theory of affordance to the design profession. Norman updates his take on affordance in the revised, updated edition of his landmark book, The Design of Everyday Things (Basic Books). Generally, Norman cautions that we should distinguish between affordances, such as the form of a door handle that we recognize as fitting our hand and suited for pulling or pushing, and signifiers, such as the “Push” or “Pull” signs that often adorn such doors.[55] We will look at signifiers and how they intersect with affordance in Part III.

  This distinction is also recommended in recent work by ecological psychologist Sabrina Golonka, whose research focuses on the way language works to create information we find meaningful, “without straining or redefining original notions of affordances or direct-perception.” For affordance to be a useful concept, we need to tighten down what it means and put a solid boundary around it.[56]

  That doesn’t mean we are done with affordance after this part of the book. Affordance is a critical factor in how we understand other sorts of information. Just as a complex brain wouldn’t exist without a body, mediated information wouldn’t exist without direct perception to build upon. No matter how lofty and abstract our thoughts are or how complex our systems might be, all of it is rooted, finally, to the human body’s mutual relationship with the physical environment.

  As designers of digitally infused parts of our environment, we have to continually work to keep this bodily foundation in mind. That’s because the dynamic by which we understand the context of a scene in a movie—or a link on a web page—borrows from the dynamic that makes it possible for us to use the stairs in a building or pick a blackberry in a briar patch. Our perception is, in a sense, hungry for affordance and tries to find it wherever it can, even from indirectly meaningful information. That is, what matters to the first-person perspective of a user is the blended spectrum of information the user perceives, whether it is direct or indirect. It’s in the teasing apart of these sorts of information where the challenge of context for design truly lies.

  Soft Assembly

  Affordance gives us one kind of information: what I’m calling physical information. But, what we experience and use for perception is the information, not the affordance that created it. Cognition grabs information and acts on it, without being especially picky about technical distinctions of where the information originates.

  Cognition recruits all sorts of mechanisms in the name of figuring out the world, from many disparate bodily and sensory functions. The way this works is called soft assembly. It’s a process wherein many various factors of body-environment interaction aggregate on the fly, adding up to behaviors effective in the moment for the body.[57] Out of all that activity
of mutual interplay between environment and perceiver, there emerges the singular behavior. Now that we can even embed sensors and reactive mechanisms into our own skin, this way of thinking about how those small parts assemble into a whole may be more relevant than ever.[58]

  We’re used to thinking of ourselves as separate from our environment, yet an ecological or embodied view offers that the boundary between the self and the environment is not absolute; it’s porous and in flux. For example, when we pick up a fallen tree branch and use it as a tool—perhaps to knock fruit from the higher reaches of the tree—the tool becomes an extension of our bodies, perceived and wielded as we would wield a longer arm. Even when we drive a car, with practice, the car blends into our sense of how our bodies fit into the environment.[59]

  This isn’t so radical a notion if we don’t think of the outer layer of the human body as an absolute boundary but as more of an inflection point. Thus, it’s not a big leap to go from “counting with my fingers” to “counting with sticks.” As author Louise Barrett explains, “When we take a step back and consider how a cognitive process operates as a whole, we often find that the barrier between what’s inside the skin and what’s outside is often purely arbitrary, and, once we realize this, it dissolves.”[60] The way we understand our context is deeply influenced by the environment around us, partly because cognition includes the environment itself.