Understanding Context

by Andrew Hinton

  Figure 14-4. The user begins shopping on the Web—a place without location—but must be placed in a “store” to see necessary information about products

  Conventions are starting to emerge for disambiguating the context of product availability. The approach adopted by Home Depot, shown in Figure 14-5, is becoming more common: for the product-listing view, the interface presents options to view All Products as well as Online Only and In My Store. This strategy seems pretty straightforward, but it turns out there are numerous complications when we unpack what these contextual labels really mean.

  Figure 14-5. Home Depot’s attempts to give semantic structure to the overlap of online and in-store shopping

  All Products

  In many online stores, “all products” can actually mean “all products we list online, but not everything you could buy in the store.” This issue is improving, but lots of big retailers are still working to get their services infrastructure to handle the same inventory across stores and online. For stores that sell regionally sourced products such as lumber, this can be a special challenge.

  Online Only

  At first glance, this is a clear label that indicates products displayed on that tab are available only by ordering from the website. However, on some retail sites, what it really means is “these aren’t available in ‘Your Store,’ so you need to order them from the site instead,” even though they might be available in a store only a few miles away from “Your Store.”

  In My Store

  I never selected a store and agreed to make it “mine.” The website chose a store for me, using a third-party service that guesses my location based on the IP address the system thinks is assigned to my device’s net connection. I know this only because I’ve learned it in my consulting work. However, many regular customers don’t have that insider knowledge; they might not notice the choice was made for them at all. Sometimes, when systems make these choices for us, it’s convenient. But it works only when it’s accurate. Unfortunately, these geolocation-by-IP services are notoriously flakey, because there’s no guarantee my IP address reflects my real location. For example, many customers do shopping from their workplace or via a coffee shop’s WiFi connection. Corporate computers often use proxy connections for security reasons, and the proxy can be located many miles away. You can easily buy something thinking it will be ready for pickup at the store on the way home, only to find that it’s actually waiting for you in another city.

  The language is trying to create stability where systems are unable to promise it. There’s always a point at which simplifying the environment obscures too much important information. The best way to handle a situation like this is not to fake simplicity but to embrace the complexity and clarify it by making it more understandable. For example, Lowe’s Home Improvement has been experimenting with how to make the complex rules more clear, without obscuring important information that the customer really should understand, as depicted in Figure 14-6.

  When the customer gets to a point on the site at which accurate location is relevant to action—such as in product lists with prices and availability displayed, or in the shopping cart—the site gently but unmistakably provides a message panel that urges the customer to take a moment to confirm the store choice. It’s not a pop-up window that interrupts and obscures—the customer can keep shopping whether dismissing the embedded message or not, but it will continue to show up at important moments to ensure that the customer has confirmed the store before buying anything.

  This is a digitally encoded business rule that the user needs to understand in order to use the environment. Like the ontology of “calendar” that we looked at earlier, here the ontologies of “store” and “shopping” and “purchase,” and even “product,” are still being sorted out for retailers who need customers to shop both in the cloud and in a store. Until systems can magically not only read your mind but know the exact location you want to get a product from, some version of this translation will be needed.

  Figure 14-6. Lowes goes to some trouble to explain the complexity and avoid costly confusion about location

  Location, Location, Location

  We see digital influence at work even in subtle ways in our various online platforms. For example, as demonstrated in Figure 14-7, there are many ways to describe the location of a person.

  Figure 14-7. My Twitter profile, temporarily sporting an iPhone coordinate location

  Physically

  If I can see the friend, the friend is an object in the environment, exhibiting surface patterns that I recognize as that friend’s face and body, and event-specifying information in the way the friend moves. The friend could be out of sight—occluded by another environmental surface—but I might still be able to hear the friend speak or make some other recognizable sound. Either way, the friend is within a relatively close distance to where I am.

  Semantically

  Semantic information gives us the ability to extend our sensemaking of location far beyond our human-scale perception. Because of the cultural system I’m part of, I understand that if my friend is “in Detroit,” Detroit is a city in Michigan, which is situated on a map that I’ve seen thousands of times in my life. I probably couldn’t tell you exactly where, but I know it’s in the Midwest and close to the Canadian border. It’s not precise, but it’s a good-enough, satisficing concept that gets the job done.

  Digitally

  Suppose that I’ve met this person only online, over Twitter. At some point I wonder where he lives, so I check his Twitter profile only to find that it lists the person’s location as iPhone: 37.404922,-5.992203, as in Figure 14-8. This is coordinate information that doesn’t correspond with any semantic or physical reference point for me. Technically, this is semantic information, but in the context in which I’m seeing it, it’s bound up with digital influence—it requires that I tap something to ask the software to translate it into a city name or a simple map.

  Ambient Agents

  There are physical objects and events in our surroundings that are digitally enabled; and these physical-digital agents are proliferating at a wild rate. Digital agent-objects are already more prevalent than most people realize, because they’ve been introduced into our lives in mostly mundane ways. For years we’ve been getting comfortable with on-screen digital agents such as an email client that decides on its own which messages are spam and which aren’t. Thus, it’s a small step to having a clothes dryer that stops when it senses the moisture is gone, or a thermostat that learns our habits and makes best-guess adjustments to home temperature. The consumer space is already awash in these gadgets, and many are networked so that we can control them from a kitchen console or personal smartphone.

  These are all seemingly benign agents on their own, but as they add up, they become a layer of digital sentience pervading our surroundings. This can have a downside, as consumers discovered when refrigerators and other smart devices were hijacked by spam bots in late 2013,[278] or when electronics manufacturer LG had to admit it was collecting private viewing-habits data without customer knowledge.[279] These digital-information undercurrents transform our very dwellings from private enclaves to porous nodes on global networks—they change what “home” means. Generally, these technologies are created with the best of intentions. Yet, like all technologies, we have to learn their limits and consequences, and improve along the way.

  Likewise, the networked objects of the urban landscape transform what cities are to us. In William Mitchell’s ever-prescient words, “Rooms and buildings will henceforth be seen as sites where bits meet the body—where digital information is translated into visual, auditory, tactile, or otherwise perceptible form, and, conversely, where bodily actions are sensed and converted into digital information.”[280] Since those words were published in 1996, much of the world has found itself in exactly the situation Mitchell describes: our inhabited places are fundamentally different, whether online or offline, through the emergence of networked consumer
technology and government infrastructures. We treat them like the invariant processes of nature, even though digital influence means these processes don’t have to stay true to any natural laws at all.

  Our cities and towns are becoming sensor-studded, agent-suffused environments that can improve our lives immensely, such as the cityscape of systems depicted in Figure 14-8. But they also require careful design and translation into people’s everyday, tacitly aware understanding.

  As inhabitants of these places satisfice their way through each day, they need the environment to let them know when objects aren’t going to behave the way natural objects do. Retail store shelves can track body movements, age, and gender.[281] Safety-aware, cooperatively smart chemical drums can track how well their handlers are following safe-handling policies.[282] These are creatures of a sort that are not especially legible to us.

  Figure 14-8. Detail portion of an infographic about smart cities, from Internet-of-Things platform provider, Libelium[283]

  These are known challenges in computing fields. In a 2010 IEEE paper titled “Smart Objects as Building Blocks for the Internet of Things,” the authors raise important questions about digital-agent objects:

  The vision of an Internet of Things built from smart objects raises several important research questions in terms of system architecture, design and development, and human involvement. For example, what is the right balance for the distribution of functionality between smart objects and the supporting infrastructure? How do we model and represent smart objects’ intelligence? What are appropriate programming models? And how can people make sense of and interact with smart physical objects?[284]

  The model, presented in Figure 14-9, describes smart objects across several dimensions, one of which the authors describe as “fundamental design and architectural principles: activity-aware objects, policy-aware objects, and process-aware objects.”

  Figure 14-9. A model describing smart objects across several dimensions[285]

  This and other models like it will be important for breaking down the types of agency these objects have, informing the design approaches we should take in creating and including them in designed environments.

  In their book Code/Space: Software and Everyday Life, a particularly important idea explored by Kitchin and Dodge is that, not only do hardware and software objects have this kind of agency, but the “space” (in our terms, the places) we inhabit can have agency, as well—a quasi-sentience that’s woven into the surfaces and layouts around us. Robots aren’t only in the form of objects that behave like people or animals; entire buildings and cities can be “robots” of a sort.

  This might sound ominous, and for good reason. Without diligent attention and foresight of its creators and users, the agency of pervasive digital infrastructures can erode human agency, independence, and privacy. In Everyware: The Dawning Age of Ubiquitous Computing, Adam Greenfield offers a thorough analysis of the implications of pervasively digital environments. One of Greenfield’s theses states: “Everyware produces a wide belt of circumstances where human agency, judgment, and will are progressively supplanted by compliance with external, frequently algorithmically-applied, standards and norms.”[286] We are used to living in environments in which, other than nature, everything around us happens based on the present or planned choices of human beings, but that arrangement is rapidly changing. We’ve created systems that use light-speed decision logic based on rules that are increasingly written by the systems themselves. Greenfield has dedicated himself to ongoing efforts at bridging the “black box” dimension of digital agency with the human-scale world that people can actually perceive and understand. The socio-political implications of “everyware” are deeply important. Although that’s not our focus for this discussion, Greenfield’s work is a good starting place for considering these issues.

  When designing a system that depends on the perception of a human agent, the system should present invariant information that meets the expectations that human perception brings to the experience—the human umwelt (one’s perceived environment). It’s difficult enough to understand what all the “levers” in our lives actually do. Now, though, we are faced with a world bristling with levers that do not look like levers; they are hidden from view or look and behave like something else entirely. These levers can create events too fast for our slow, embodied perception to pick up, much less understand.

  Likewise, when designing a system that needs to perceive human context in helpful and accurate ways, the system itself should clearly signal its own umwelt—how it perceives and understands the human trying to interact with it. People tend to assume a digital agent is going to know more than it actually is capable of knowing; users of these agents risk making too many assumptions about the agents’ intelligence. That’s partly because intelligence is, itself, a reification. There is no actual thing we can point to that is “intelligence.” But reification means we see intelligence where there often isn’t any, perhaps to our great disappointment or risk. Digital agents need to be transparent about their limitations rather than present a simplified front that inaccurately promises human-like coherence.

  This is why it’s so important to understand how humans experience context. The parts of our environment that have been given digital agency should have an ontology that the human agent can comprehend, so users can understand what sort of creature the agent is, how to interact with it, and what can be expected of it.

  * * *

  [273] Thanks to Richard Dalton (http://mauvyrusset.com) for this tidbit.

  [274] An informative take on the filtered-place problem is in the book, The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think, by Eli Pariser. Penguin Books, 2011.

  [275] Bates, Marcia J. “The design of browsing and berrypicking techniques for the online search interface.” Online Review. October, 1989;13(5):407–24.

  [276] Bates, Marcia J. “Toward an Integrated Model of Information Seeking and Searching.” (Keynote Address, Fourth international Conference on Information Needs, Seeking and Use in Different Contexts, Lisbon, Portugal, September 11, 2002.) New Review of Information Behaviour Research. 2002; Volume 3:1-15.

  [277] Pirolli, Peter, and Stuart Card. “Information Foraging” Xerox Palo Alto Research Center. Psychological Review. 1999; 106(4):643–75.

  [278] “Fridge sends spam emails as attack hits smart gadgets.” BBC News Technology January 17, 2014 (http://www.bbc.com/news/technology-25780908).

  [279] Wakefiled, Jane. “LG promises update for ‘spying’ smart TV.” BBC News Technology November 21, 2013 (http://www.bbc.com/news/technology-25042563).

  [280] Mitchell, William J. City of Bits: Space, Place, and the Infobahn (On Architecture). Cambridge, MA: MIT Press, 1996:913–14, Kindle locations.

  [281] Ungerleider, Neal. “The Future of Shopping: Shelves That Track the Age and Gender of Passing Customers,” Fast Company (fastcompany.com) October 15, 2013 (http://bit.ly/1r8RsSw).

  [282] Kortuem, Gerd et al. “Smart objects as building blocks for the Internet of things,” Internet Computing, IEEE. 2010;14(1):44–51 (http://usir.salford.ac.uk/2735/1/w1iot.pdf).

  [283] Courtesy, Libelium (libelium.com)

  [284] Kortuem, Gerd et al. “Smart objects as building blocks for the Internet of things.” Internet Computing, IEEE. 2010;14(1):44–51.

  [285] ———. “Smart objects as building blocks for the Internet of things.” Internet Computing, IEEE. 2010;14(1):44–51.

  [286] Greenfield, Adam. Everyware: The Dawning Age of Ubiquitous Computing. Berkeley, CA: New Riders; 2004:148.

  Part V. The Maps We Live In

  Information Architectures for Places and People

  FOR THE REMAINDER OF THE BOOK, the main focus will be about how we use information to create architectures—environmental structures that shape the experience of “place.” Prior chapters provided many examples of close-focus structures at the level of interaction; now, we will adjust our “zoom level�
� to be from a higher altitude, as we see how all those objects participate in systems of meaning. Like the natural environment, there are no places without objects, and objects ultimately define the moment-to-moment experience of place. But the overall coherence of the system brings architectural challenges to our work.

  Figure V.1. A map, with people inside it. In this case, on the side of a favorite pizza joint in; Phoenixville, PA[287]

  Information architecture is a discipline and practice well suited for attending to these challenges. It has been working with them in one way or another for decades. Early on, information architecture started as a way to make complex information about the world and its places understandable. Over the years, it has grown to also be a way to use information for creating places in the world itself. The framing device for understanding how semantic information forms environments will be “maps.” But these are not just representational maps—they are maps we inhabit together.