When We Are No More

by Abby Smith Rumsey

  Biology is neither personal nor cultural destiny. But as we explore the early-stage findings of memory science, we discover that the relationship between the part and the whole, the individual and society, personal and collective memory, are intimately intertwined and evolve together. We may never be able to map cultural history directly onto our biological legacy, but we cannot afford to ignore it. To the extent that our cultural life extends and elaborates on our essential biological capacities, what we learn may serve as a guide, if not a set of detailed instructions, on how to design digital memory.

  HOW WE LEARN THE WORLD

  The crowning achievement of memory is the model of the world that we carry in our heads, a kind of diorama that closely resembles reality and allows us to respond to events in real time. As Frith points out, what we perceive in any moment is a combination of real-time perception and stored information, our memory of the world. No creature is able to process enough information in real time to react appropriately to events as they transpire. So it prepares a miniature map of its environs and annotates each spot with detailed information about people, places, and things found there. Like the sea slugs and rats studied in labs, we are all born cartographers who draw mental maps to orient ourselves to our surroundings. In split seconds, the mind runs through its geo-checklist: Where am I? How did I get here? Where am I going? Who else is here? Like Simonides, we spatialize that information, sometimes in way-finding maps that tell us how to get from point to point, sometimes creating a mappa mundi, a map of the world that represents our conception of space and time. Humans are also aware of existing in the fourth dimension of time. We create narratives that make sense of our passage through time just as our maps make sense of our passage through space. The model we assemble has to be flexible and easily modified over time so that our behaviors can be calibrated to the current scenario.

  How our brains decide what information is important to remember is hidden from our direct observation. Most of the processes of attention and retention of information are directed by our instincts or emotions and processed unconsciously. As neuroscientists Eric Kandel and Larry Squire note, “By virtue of the unconscious status of these forms of memory, they create some of the mystery of human experience. For here lie the dispositions, habits, attitudes, and preferences that are inaccessible to conscious recollection, yet are shaped by past events, influence our behavior and our mental life, and are a fundamental part of who we are.” The unconscious nature of these nondeclarative memories makes them essentially off-limits to analysis by machine intelligence, at least at our present state of knowledge. Unlike their facility with logical processing, computers and robots do not process information with emotions. Nor do they store emotional memories. As we will see, emotions are essential to decisions involving values, such as making choices between leaving work at the end of the day to spend time with family or staying late to finish a pressing assignment, and between telling a friend a white lie or seriously hurting his feelings.

  We are not born as blank slates, having to learn literally everything from scratch. Living creatures come with preprogrammed memory, the genome, that encodes the history of the species and provides full instructions on how to become an ant if you are born with ant genes, a marmot if with marmot genes, a human if with human genes. Scientists will often refer to the genome with rhetorical flourish as the book of life and modern biology as a science of information because so much of their work lies in deciphering how cells communicate with other cells and send orders from one part of the body to another that are instantly heeded. Because genes retain modifications over time, scientists identify the genome as an archive of the past. “The new genome in a fertilized egg is not the new person nor even an encoded version of the person; it is the archive of information that the developmental clock will use to form a new and genetically unique body from the descendants of one fertilized egg cell.” In other words, the egg is like an executable file: Add food and water and watch it grow. We are programmed to be curious and acquire information about the specific environment into which we are born. The more complex the environment, the steeper the learning curve and longer the education we experience. Social animals like Homo sapiens and elephants require long gestation periods and decades of on-the-job training before they reach maturity.

  Conceptually, memory formation and retrieval is relatively straightforward, the very model of curation. It boils down to just a few steps: selection, acquisition, categorization, storage, and preparation for ready retrieval on demand. That said, each step of mental curation is intricately detailed and involves global coordination and synchronization with other processes in the brain. How this happens is beyond the horizon of present-day science. But the basic contours of each process are coming into focus.

  We scan the environment for information that catches our attention. Given that the brain’s primary job is to keep us alive, it is highly attuned to perception of novelty. We become inured—habituated, as scientists say—to the familiar. Something new and unexpected will grab our perceptual attention, and the brain will make inferences about everything else, based on what it has laid down in its memory. As a consequence, most of the information we perceive in any given moment is disregarded—effectively thrown away—because it is redundant. What we acquire through our senses is instantaneously processed through emotional and cognitive centers for value and sensemaking. The brain looks for matches against similar information stored in the brain, and then temporarily parks it for ready reuse. Short-term memory is packaged first as read-only, as it were, and is easily discarded if not transferred into long-term storage. At the end of the day, everything taken in gets parked in the overnight parking lot, so that it can be sorted and processed during sleep before going into its assigned address. The crucial processing steps that occur during our sleep are inaccessible to our consciousness. By day, our strategy for sensemaking is to create a narrative of events that suggests cause and effect and identifies a context in which all the elements cohere and find meaning. By night, the linear timelines we observe during waking hours are not operative. In dreams the rules of cause and effect are suspended, our internal censors fast asleep and vast realms of reality normally inaccessible to us take center stage. In sleep the mind sorts the rubbish of the day from the riches we grab and keep forever. Adrienne Rich said poems are “like dreams: in them you put what you don’t know you know.” But the reverse is true as well. Dreams are like poems: In them what we know is represented by symbols whose meanings are ambiguous, multivalent, open-ended.

  Information has value to the extent that it has the potential for reuse. Erwin Schrödinger noted: “Biological value lies only in learning the suitable reaction to a situation that offers itself again and again, in many cases periodically, and always requires the same response if the organism is to hold its ground.” What is useful gets converted to long-term memory through a consolidation process before taking up permanent residence in our mind. During consolidation, content is abstracted from its native context and made available for reuse in other contexts. After once experiencing the scorching heat of an open flame on flesh, we abstract that pain into an infinitely reusable lesson about fire, and further, all forms of intense heat and, metaphorically, about pain associated with emotional intensity.

  Information processing may occur in very specific parts of the brain, but the storage itself is likely to be organized and distributed in a way that facilitates an almost infinite number of possible uses in a wide variety of scenarios. Our uncanny ability to see patterns everywhere allows us to interpolate information from our memory banks into present perceptions. What we “see” are inferences the brain makes on the basis of what it already knows—that is, remembers. The patterns we see are based on samples of the world, the data within each sample are chemically tagged with specific molecules that express value, and those with the greatest salience command our attention. You may be walking down a strange street in a foreign town when out of the corner of your eye, yo
u see a fast-moving blur approaching you. Before you have time to look properly and assess what it is, your mind has already commanded your body to protect itself against a collision by stepping back. Your mind doesn’t know what it is, but it has already deferred to memory and taken evasive action. These mental patterns, whether of self-protection or pursuit of pleasure, are like the flexible yet sturdy and resilient lattices of a pergola, serving as architectural supports for the buds of thought that sprawl like vines. This is how we wind up seeing a complete picture of the world from split second to split second, even when our senses have time only to record fragments of what we perceive in any given moment. Like poetry, mathematics, and music, good memory relies on patterns that simultaneously constrain content and suggest meaning.

  During recollection, a memory is opened up like a book or computer file, gets reworked, then reencoded and stored in a slightly modified way. “The retrieval of the consolidated memory is a dynamic and active process in which remodeling or reorganization of the already-formed memories occurs to incorporate new information.” Recall is literally a rebuilding process, executed chemically, and new perceptions are incorporated into the old. The more often a memory is called up for use, the stronger it gets, be it declarative (factual) memory such as an event or word or somatic (physical) memory such as a smell, a sound, a golf swing or keyboard skills. Memory consolidation, “the progressive post-encoding stabilization of the memory trace,” is the phase in which connections between the new data coming into the brain find their home in existing mental contexts or webs of associations. In other words, memory consolidation creates meaning by putting information into an appropriate context. Once in context, it can be used again in the future. And it is in this phase that a memory is most vulnerable to losing its way, never finding a meaning or context that can hold it stable and available for future use. Because every memory is tweaked and fortified in use, we live in a state of nonstop historical reinterpretation. The upshot is that the past itself changes in the process of remembering. The representations of things that were laid down previously—whether five decades ago or five minutes—are modified simply by being used. In use, they are placed within the environment of the present moment and that use becomes an intrinsic part of memory itself—the past plus.

  Before the twenty-first century, we relied on a system of preserving memories that complemented our internal memory. One of the breathtakingly simple advantages of the cuneiform, scroll, or printed page was that the memories inscribed on them were not easily changed, overwritten, or erased. On the contrary, these durable objects acted in exactly the opposite way our brains work. If kept in reasonably good physical shape, the words and images on a piece of paper would not change one whit for hundreds of years, no matter how many times they were read. Digital memory operates much more like biological memory. It is not really fixed and is easily overwritten or updated without leaving much trace of the changes made. With digital memory we lose one of the crucial advantages of fixed and stable physical memory—the fixed and passive retention of information. How we re-create the advantages of physical storage in the digital realm is an important consideration in designing our memory systems to stabilize digital data over long periods.

  Deep learning and creativity, on the other hand, rely on the transformation of one day’s intake of perceptions to something sustained over time, embedded within a network of existing associations. What we call creativity is the use of mental content in contexts wildly unrelated to its source. (Humor is putting the content into wildly incongruous contexts.) But this deep absorption of content into the mind can exact a price. One common by-product of preparing content for multiple uses is source amnesia, whereby we remember content strongly but not where we first acquired it. You may know a certain film was recommended to you, but you cannot remember by whom. You may have read how many people were killed in an accident, but not where you read it. In each case the source was familiar or trustworthy enough (your friend who recommended the film) that you did not bother to store information about it, or the information so arresting (the fatalities) that the content eclipses all other impressions.

  ANALOG WAVES AND DIGITAL BITS

  The brain uses both analog and digital processing. Digital signals communicate what to pay attention to, and analog processing comes in to suppress some signals and amplify others in order to allow us to focus our attention on the object of choice. This is how, for example, we can catch the voice of the person we are looking for in a noisy room. One process tells the brain where our friend’s voice is coming from, and another part dials down the volume of other voices and dials up the volume of our friend.

  Analog circuits process continuous variations in intensity; digital circuits operate in an on-off/yes-no mode. So far, we have not been able to devise machines that use both analog and digital signal processing with anything like the specificity and flexibility that our hybrid brains do. For most of our history, we have relied on analog processing—printing ink on paper, painting pigments on canvas, inscribing sound waves onto wax cylinders or plastic discs. We are long familiar with the strengths and weaknesses of analog formats. They can be slow and imprecise, but infinitely rich in subtlety and nuance of perception.

  By its inherent nature, digital recording samples information, cuts it up, disambiguates it into 0s and 1s, and packages these bits in a way that makes them very easy to manipulate, reorder, and, most powerfully, send over long distances with minimal loss of information. This creates a paradox: that analog information has a greater integrity—of the literal kind—and more accurately mirrors the embodied mind in time and space. But it is less flexible and faithful as a means of analysis, comparison, and communication over long distances. The constraints of a book, a photograph, a paper map are precisely those of the human body—unable to leap tall buildings in a single bound or teleport itself over long distances in seconds. As man-made physical objects, all these artifacts of recorded knowledge—maps and photos, books and magazines—exist on the same scale as the humans who created them. The digital does not.

  With digital encoding comes a fateful dependence on machines for reading and playback equipment. A digital file is machine readable, not eye legible. We can make little or no sense of it unless we have the right hardware to run the right software to decipher it. A dependence on playback equipment is not new to digital information. It began with sound recordings. It takes only ambient light to read a musical score but any and all media that carry sound waves, from wax cylinders and wire to lacquer discs and cassette tapes, demand machines to convert grooves and magnetic signals into sound waves. The introduction of recording technologies that depend on machines to read them, dating back to Edison’s invention of the phonograph in 1877, marked the beginning of our unconditional dependence on secure and reliable sources of power to maintain our knowledge and memory banks.

  Digital moves us in great leaps and bounds to new efficiency and flexibility of the types of information we can record. And it makes manipulation of what is recorded easy, because everything is already cut up into little bits that can be shuffled and rearranged, tweaked for saturation of color or audio wavelength. But it does so with corresponding losses in our ability to distinguish amplitudes, continuities, and intensities. What the demotion of the analog capacity for detecting and capturing degree and continuity means for our culture, for ourselves, is unknown. But we know the effect of accelerated processing time and of binary thinking in our everyday lives: We have simultaneously more information and fewer means to sort its value.

  Digital recording technologies change the way a musician approaches performance; technical proficiency (or deficiency) is more audible in digital recording than degrees of expressiveness. The difference between capturing light on photochemical film and on digital memory chip has similar effect—greater sharpness, less subtle gradation of palette. At the extreme, digital can pixilate and reveal its complete inability to represent the nature of reality as we experience it—continuous
and deeply connected. As the cognitive scientist Randall O’Reilly says, “It is clear that the brain is much more like a social network than a digital computer.” Memory and learning are investigated now as products of “the graded, analog, distributed character of the brain.” It turns out that the computer is not an accurate metaphor for the brain.

  VALUE AND EMOTION

  Emotion is the body’s internal representation of value. Emotions such as fear and joy register the meaning of something to us. Their explicit and conscious counterparts—anxiety and happiness—are what we say we feel. Are we gripped with fear when weighing our chances of doing well on a crucial test we do not feel prepared for? Then thinking of it makes us anxious. Are we suffused with joy when we spend time with our children after a long absence? Then thinking about them makes us happy. Emotional value is processed preconsciously and tagged by chemical markers. The stronger the emotion, good or bad, the greater the value for better or for worse. Sometimes the emotions are passed to the conscious mind and we become aware of them, sometimes not. But whether conscious or not, they are associated with our senses through visual, olfactory, or aural cues. As the neuroscientist Susan Greenfield writes, “Pure emotion can be viewed as the core of our mental states … as when we are infants, feeling is not greatly tempered with individual memories, with cultural or private meaning, or, most important of all, with the self. Feelings just are.” No matter how long we live and how polished our manners, emotions can never be civilized. If they were, they would lose their value to us. They are meant to surprise us.