Understanding Context

by Andrew Hinton

  [76] Polanyi, Michael. The Tacit Dimension. Chicago: University of Chicago Press, 1996:4.

  [77] http://en.wikipedia.org/wiki/Lewin’s_equation

  [78] Gibson, J. J. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin, 1979:17.

  [79] ———. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin, 1979:130.

  [80] Banakou, Domna, Raphaela Groten, and Mel Slater. “Psychological and Cognitive Sciences: Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes.” Biological Sciences PNAS 2013;110(31):12846-51; published ahead of print July 15, 2013, doi:10.1073/pnas.1306779110.

  [81] Hogenboom, Melissa. “Adults become more like children in a virtual world,” (http://bbc.in/1uuRqFI).

  [82] Shokur, Solaiman, Joseph E O’Doherty, Jesse A Winans, Hannes Bleuler, Mikhail A Lebedev, and Miguel AL Nicolelis. “Expanding the primate body schema in sensorimotor cortex by virtual touches of an avatar.” Biological Sciences – Neuroscience PNAS 2013; published ahead of print August 26, 2013, doi:10.1073/pnas.1308459110.

  [83] Thaler, Richard H., and Cass R. Sunstein. Nudge: Improving Decisions about Health, Wealth, and Happiness. New Haven, CT: Yale University Press, 2008:181.

  [84] Schacter, Daniel. Searching for Memory the brain, the mind, and the past. New York: Basic Books, 1996.

  [85] ———. Searching for Memory the brain, the mind, and the past. New York: Basic Books, 1996:4.

  [86] McCall, Becky. “Memory surprisingly unreliable, study shows.” Cosmos magazine, September 15, 2008. http://bit.ly/1uDR42i

  [87] Hayasaki, Erika. “How Many of Your Memories Are Fake?” The Atlantic (theatlantic.com) November 18, 2013

  [88] Wilson, Andrew. “What Does The Brain Do, Pt 2: The Fast Response System” Posted in Notes from Two Scientific Psychologists, August 2, 2011 (http://bit.ly/1Fs0TWF).

  [89] Malone, Michael S. The Guardian of All Things: The Epic Story of Human Memory. New York: St. Martin’s Press, 2013:14.

  [90] Wikimedia Commons: http://commons.wikimedia.org/wiki/File:Memory.gif

  [91] Gibson, J. J. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin, 1979:202.

  [92] ———. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin, 1979:253.

  [93] Barrett, Louise. Beyond the Brain: How Body and Environment Shape Animal and Human Minds. Princeton, NJ: Princeton University Press, 2011:183, Kindle edition.

  [94] Barrett, 2011:214, Kindle edition.

  [95] Quoted by Barrett p. 238 [Gibson (1970), p. 426.] J. J. (1970). On the relation between hallucination and perception. Leonardo 3:425–427.

  [96] In cognitive studies, the terms “online” and “offline” refer to the difference between cognition about a situation that is at hand, in the present moment, versus cognition that’s about a situation not at hand and remembered from some earlier experience. Climbing a tree to get at some apples in higher branches would involve cognition “online.” Later, when away from the tree but planning how to get higher apples out of it the next time you’re there requires “off-line” cognition.

  [97] Baranauckas, Carla. “Eleanor Gibson, 92, a Pioneer in Perception Studies, Is Dead.” New York Times January 4, 2003. Retrieved October 29, 2013.

  [98] Alberini, Cristina M., ed. Memory Reconsolidation. Waltham, MA: Academic Press, 2013.

  [99] http://www.radiolab.org/story/91569-memory-and-forgetting/

  [100] Lindsay, D. S., L. Hagen, J. D. Read, K. A. Wade, and M. Garry. “True photographs and false memories.” Psychological Science 2004;15:149–154. doi:10.1111/j.0956-7976.2004.01503002.x.

  [101] Wade, K. A., M. Garry, J. D. Read, and D. S. Lindsay. “A picture is worth a thousand lies: Using false photographs to create false childhood memories.” Psychonomic Bulletin & Review 2002;9:597–603. doi:10.3758/BF03196318.

  [102] Gibson argues that “Information does not have to be stored in memory because it is always available” for pickup. Gibson, 1979:250.

  [103] Guibert, Susan. “Walking through doorways causes forgetting, new research shows.” Notre Dame News November 16, 2011 (http://ntrda.me/1Fs0XFX).

  [104] I should point out that Norman is not an embodied cognition theorist; I am (I think, accurately) appropriating some of his more embodiment-compatible ideas.

  [105] Osberg, Molly. “Hug it out: can art and tech ever be friends?” The Verge (theverge.com) May 8, 2014.

  [106] McGaugh, J. “Involvement of the amygdala in memory storage: interaction with other brain systems.” Proceedings of the National Academy of Sciences 1996. Available at http://www.pnas.org/content/93/24/13508.short.

  [107] Norman, Don. The Design of Everyday Things: Revised and Expanded Edition. New York: Basic Books, 2013:100, Kindle edition.

  [108] ———. The Design of Everyday Things: Revised and Expanded Edition. New York: Basic Books, 2013:79, Kindle edition.

  [109] Nielsen, Jakob “User Expertise Stagnates at Low Levels” September 28, 2013 (http://www.nngroup.com/articles/stagnating-expertise/).

  [110] “Schindler Installs PORT at Hyatt Regency New Orleans” schindler.com January 26, 2012 (http://bit.ly/1wg5F4s).

  [111] Much to my delight, many months after first drafting this passage, I learned that Donald Norman’s new edition of The Design of Everyday Things also explores this elevator system (pp. 146–149). Norman comes to similar conclusions (though, as an engineer, he seems to find more to like about the system than I did as an annoyed conference attendee).

  Chapter 6. The Elements of the Environment

  The earth is not a building but a body.

  —WALLACE STEVENS

  Invariants

  WE’VE LOOKED AT HOW our basic functions of perception and cognition work in an environment and how affordances form that environment. Given that context is largely about how one thing relates to another thing, let’s now look at how we perceive elements and their relationships.

  Luckily, our friend James J. Gibson, the ecological psychology theorist, created an elaborate yet straightforward system describing the structures that make environments. We won’t be exploring all its details, but there are some major portions we can borrow for making a sort of building-block kit for purposes of design. These elements start with the most basic: what Gibson calls invariants.

  Invariants are persistently stable properties of the environment; they persist as unchanging, in the midst of change.[112] These are not permanent properties in the scientific sense of permanence.[113] A hill might erode; a fallen tree might rot; the sun will eventually burn out, but they still involve invariants because they have properties that have been “strikingly constant throughout the whole evolution of animal life.”[114] Invariance is, then, about the way the animal perceives the environment, not an objective measurement of permanent structure.

  The only reason we can do anything is because some parts of our environment are stable and persistent enough to afford our action. The laptop keyboard I’m typing on right now is solid, with little square keys that spring up and down in response to my fingers. The keys wouldn’t be able to do this if the body of the laptop and the bed of the keyboard were made of, say, helium. Only because it’s a solid surface with a particular kind of shape can this activity take place.

  Likewise, when I get up from my writing spot and go to make a cup of coffee, I can get to the kitchen because the surface of my floor is made of wood, which is solid and supportive of my weight. That is, it’s enough like the ground my species evolved on that I can walk on it, as well.

  There are also walls around me, and they have openings that are windows and doors. The doorway between my current writing spot and the kitchen is large enough for me to walk through; I know this because I’ve walked through many other doors and have a good feel for which ones will afford passage and which ones won’t. I’ve also grown up in a culture where conventional doorways afford walking through for someone of average size such as mys
elf. I take them for granted as persistent structures in the built world, the way a squirrel takes for granted that trees will afford climbing.

  Perception itself originates with our perceiving of invariants.[115] “The persisting surfaces of the environment are what provide the framework of reality.”[116] We can intellectually know that an earthquake or a hurricane isn’t actually changing “reality,” but at an ecological level, these rare events disrupt our most deeply embodied knowledge of how reality should work.

  Invariants exist along a wide scale of the environment, from the most basic components to large structures. The invariant properties of stone are such that a human can pick up a fist-sized rock and throw it at a bird’s nest or use it to bash open a walnut. The same properties of stone make it so that a mountain serves as a landmark for generations of people, not to mention a source of fresh water and a habitat for millions of creatures that evolved there. But, invariance is ultimately about individual perceiving agents and how structure persists in their environment.

  Context is impossible to comprehend without invariants. We understand something only in relation to something else, and the “something else” has to be invariant enough to be “a thing” to begin with. (Or, as Richard Saul Wurman so often says, “You only understand something relative to something you already understand.”)[117]

  I know I am outside my house when I can see the outside of the structure and look up and notice that I am on uncovered ground, and the sky is above me. If these structures were not invariant, I’d have no idea where I was (not to mention they would no longer be structures).

  Invariance is a simple idea that we don’t think about much because it’s so intrinsically a part of our world. It’s why we can say something “is” or that we are “here.” Our cognition evolved in a world in which invariance just comes along, automatically, with the places and objects we experience.

  Examples of Invariants

  There are dozens of invariants explored in Gibson’s work, but here are just a few:

  Earth and sky

  The earth “below” and the air “above.” The ground is level, solid, affording support for walking, sitting, lying down. The air is the opposite of the ground, affording locomotion, allowing light and sound to penetrate, and giving room for living things to grow and breathe. In Gibsonian terms, this earth/sky pairing forms the most basic invariant human-scale structure, a shell within which all other structures are structured, comprehended, and acted upon. Humans didn’t evolve in outer space, so like it or not, we have ingrained in us these structures of earth and sky, and what is up or down.

  Gravity

  We tend to think of gravity as a mathematically expressed property of physics, but our bodies don’t comprehend gravitation as a scientific concept. Long before Newton, our bodily sense of up and down was written into our DNA through natural selection. Our embodied experience of gravity isn’t concerned with purely elastic elementary bodies interacting in space; the planet’s actual environment is made mostly of surfaces that don’t behave much at all like the mathematical ideals of Newtonian physics.[118]

  The occluding edge of one’s nose

  Human vision is framed most of all by the edge of the human nose; “of all the occluding edges of the world, the edge of the nose sweeps across the surfaces behind it at the greatest rate whenever the observer moves or turns his head” providing an “absolute base line, the absolute zero of distance-from-here.”[119] In English, we often turn to an embodied verbal expression to describe something as “right in front of my nose” as a way to mean it’s as present and obvious as possible (even though we may not notice it). The edge of the nose is, to our vision, the purest instantiation of what “here” means. This reminds us of how our bodies are a crucial element of our environment. The perception-action loop depends on the “landscape” of the body as much as any other of the invariant features in the experienced world.

  Other, more complex invariants Gibson describes run the gamut from “the unchanging relations among four angles in a rectangle” to “the penumbra of a shadow” and “margins between patches of luminance.”[120] These are as important as the aforementioned more basic invariants, because they set the foundation for how we know a table is within our reach or whether we can fit through a doorway. A more detailed inventory of Gibson’s invariants is beyond our scope here, but they are central to the key challenges of much industrial and interaction design.

  Invariants significantly affect how learnable an environment is. They are what contribute to what we tend to call “consistency.” In the natural world, we come to depend on the patterns we learn about with respect to what different substances and objects do and don’t do; we quickly pick up on the rules of cause-and-effect. It’s crucial to reemphasize, though, that invariants are not necessarily permanent, “frozen” structures; they are invariant because they are experienced that way by the perceiver, in contrast to the changing parts of the environment: variants.[121]

  Variants

  In contrast to invariants, variants are the environmental properties that tend to change in our perception. The shadows cast by a tree will change as the direction of sunlight changes through the day. The shape of water changes as wind interacts with it, or as it is poured into a differently shaped container. Although these changes might be predictable and follow some consistent patterns, we perceive them as changing in relation to the relatively invariant surfaces and objects around them. Elements all around us have variant properties that we don’t count on for affording action. Imagine a tribe of humans that lived for generations on an island where most of the ground was actually quicksand that was impossible to perceive as such without stepping on it. One could be sure they would walk differently than the rest of us, testing each step as they went.

  Compound Invariants

  Invariants can be simple properties of simple structures, such as the hardness of a substance. But, there are also combinations—or compound invariants—that we experience as a singular “unit” of invariant properties.[122] A bicycle is a combination of invariants providing many affordances: pedals support feet, but they also move to pull a chain, and the chain turns a wheel; the seat supports my weight; the handlebars support my body leaning forward, but also afford turning the front wheel for steering. Yet, as soon as I’ve grown used to riding a bicycle, I don’t approach the vehicle as a pedal-handlebar-seat-wheel-turning-etc. object, but as an object that affords “riding a bicycle.” The invariant affordances combine into a unit that I learn to perceive as a single invariant entity.

  In the science literature, this idea of compound invariants is still being debated and fleshed out, but I think it’s safe for us to think of it as a basis for much of what we make with technology. Most of what we design is more complex than a stone or stick; it combines invariants and variants in multiple ways. As we’ll see, with language and software, invariants can be compounded to the point that they’re not directly connected to physical properties at all; rather, they’re part of a self-referential system distantly derived from our physical-information surroundings.

  Invariants Are Not Only About Affordance

  Here’s an important point to make regarding invariants and affordances: although affordance involves perception of invariant information, not all invariant information is about affordances. We are using Affordance for physical information. Yet, we encounter all sorts of other invariant information that is meaningful to us without being directly perceived. A stop sign means “stop,” but its affordances have only to do with how the object is visible or how it is sturdy enough to lean against while waiting to walk across the street. The meaning of “stop” is one of the most stable invariants in English (and red, octagonal stop signs have a conventional, invariantly informative presence in many non-English-speaking places, as well). But the most important meaning of the stop sign is not its physical affordances. Because digital environments depend on such continuities of meaning, this distinction between language-based in
variants and physical invariants is central to the challenges of context in a digital era.

  Digital Invariants

  In digital interfaces and objects, we still have need of invariants because perception depends on them. However, this sort of information doesn’t naturally lend itself to the stability we find in physical surfaces and objects. The properties of user interfaces need to be consistent for us to learn them well. We hunger for stable landmarks in the often-ambiguous maze of digital interfaces.

  The Windows operating system’s innovation of a Start button was, in part, a way to offer an invariant object that would always be available, no matter how lost users might become in the complexities of Windows. Likewise, websites have retained the convention of a Home page, even in the age of Google, when many sites are mainly entered through other pages. It provides an invariant structure one can cling to in a blizzard of pages and links.

  These examples exist partly because the rest of these environments tend to be so variant and fluid. Digital technology gives us the capability to break the rules, in a sense, creating variants where we expect invariants. There are huge benefits to this physically unbound freedom; we can handle scale and make connections that are impossible in the physical environment. Folders can contain more folders to near-infinite capacity; a desktop can multiply into many desktops. But there are equally substantial dangers and pitfalls.