The Memory Illusion

by Dr Julia Shaw

  In a seminal paper published in 1975,26 John Flavell and Henry Wellman at Minnesota University coined the term metamemory to refer to an important faculty we possess that likely plays a part in this kind of memory self-correction. Metamemory is an individual’s awareness and knowledge of their own memory. It includes our knowledge of our own memory capacity, as well as an understanding of strategies that can improve our memories. And it also includes our ability to monitor what we can accurately remember, and to analyse our memories to confirm their plausibility.

  Thus, when we catch ourselves and realise that a memory we have is actually false, we are utilising our metamemory. It is also metamemory that generally allows us to distinguish between things we have imagined, things we have observed and things we have actually participated in – although as we have seen, this ability can be hijacked and misled to generate memory illusions. Without metamemory helping us identify the strength of our memories, assessing our memory plausibility and generally checking in on our memory abilities, we would likely be floating somewhere between reality and imagination at all times. It is why healthy adults don’t always think that what they imagined was real, and generally have a good grip on what did and did not happen to them.

  Here is another example from the False Memory Archive, of a woman who thought she had a real memory until her metamemory kicked in: ‘I was in an apartment. Four women were playing cards. The sky outside the window was dark. The curtains were an orangey plaid. The women were smoking; I remember the bluish smoke curling in tendrils toward the light over the table at which they played. One of the women said, “I think the baby’s coming!” Subsequently, she was rushed to the hospital.’

  Details about the weather, the arguably inconsequential curtains, and the incredibly vivid description of the cigarette smoke make this a compelling account. If we were told this story, it is highly likely we would take it at face value, perhaps even thinking that the teller had a fantastic memory since she was able to recall so many little details. It sounds impressive, until she goes on to state: ‘Now, the way I know that this memory is false (even though it is as clear today as when I was a child) is that the baby to whom she gave birth an hour later was me.’ The rememberer does not explain where this fabrication came from, but we can think of many possible sources, from her mother’s telling of the story, to pure imagination.

  If you are questioning the notion that something from before birth could even be called a memory, you need only talk to people who believe in the supernatural or past lives. To a memory researcher, what matters is that the person themselves classifies a recollection as a memory. They feel like it happened, and they feel like they remember it, even if it couldn’t be true. If we look at our adventure through the world of research on false memories so far, be they false memories of meeting Bugs Bunny at Disneyland, remembering a mobile seen the day after birth, or remembering birth itself, all of these events share the commonality that they are impossible but feel real.

  Metamemory is by no means flawless, however, and in trying to make sense of false memories, we can spin further narratives and come up with excuses so as to make things fit, as in this example, once again taken from the False Memory Archive:

  I went to study art history as an undergraduate for four years, carrying the pleasant memory of having [already] experienced a Michelangelo in all its glory, sublimity and sculptural grandeur [from having seen David on a visit to Florence as a child]. I found out that the V&A in London had a replica of the sculpture. When I went to see it, I was surprised to see how anorexic it looked compared to my memory. The one I thought I saw was much grander – [I thought] perhaps it was because I remembered looking at it from a child’s point of view when I was fascinated by its strength, immaculate stone quality and dominating presence. I called my dad and told him of my disappointment and to my dismay (that I feel till today) I found out that we never went to Florence and that I never saw Michelangelo’s David.

  Sometimes the realisation that something cannot have occurred only comes in the light of new evidence that contradicts our previous beliefs. Most of us are not constantly critical of our memories; our metamemory may not be on guard and so lets pieces of fiction slip past it. In these cases, it is only when we actively engage our metamemory again, usually when we see that a memory is unlikely or even impossible, that we can hope to rid ourselves of the false memories that have slipped in undetected. Metamemory is a beautiful thing that can help us decipher fact from fiction, but it too has its flaws.

  The formative forgotten years

  Before we leave childhood memories behind, I want to make one thing clear. This research absolutely does not suggest that just because we cannot remember them, early childhood events are unimportant. Our earliest years are, of course, fundamental for brain, personality and general cognitive development. According to a 2012 review of the long-term repercussions of adversity experienced in early life, by medical doctor Jack Shonkoff and his colleagues,27 experiencing adversity, even at an age before we can consciously remember it as adults, can have lasting effects. As they put it, ‘Early experiences and environmental influences can leave a lasting signature on the genetic predispositions that affect emerging brain architecture and long-term health.’ It is amazing and strange to think that the years which are possibly our most formative are also those which we remember the least.

  2. DIRTY MEMORIES

  #thedress, time travellers,

  and the good old days

  Why to remember is to perceive

  I WAS RECENTLY in San Diego, California, and found myself wandering around the absolutely stunning Balboa Park, lined with huge palm trees and dramatic cliffs. As a reprieve from the sweltering heat, I decided to go inside their science centre. There, I stumbled upon a magic show that promised to combine perceptual science, physics and psychology. I sat down in an audience of excitable children and tired-looking parents. To be honest, I was a bit worried about what I had got myself into. Then the magician, Jason Latimer, walked out on stage and began his show.

  I saw him walk through a solid mirror. I saw him form water into spheres with his hands. He put boxes that were exactly the same size into one another. He hung his coat on a beam of light. And to this day I have absolutely no idea how he managed any of this. During the performance, he consistently explained that there was nothing magical about what he was doing, rather he was using his understanding of science to fool our perceptual expectations. Latimer wanted to make us question our understanding of reality, to appreciate the capacity of science to make the seemingly impossible possible. Despite my initial concerns, I could see why they had let a magician into a science centre.

  Even when we knew exactly what trick he was about to perform, when he actively gave us a heads-up, I am confident that none of us could figure out how the illusions worked. This is important. Even when we know that they are false and are even expecting them to happen, visual illusions often remain incredibly compelling. And even though I know that my perception influences my experience of reality, and that my memory can be very flawed as a result, it usually still feels incredibly real.

  The creation of new memories relies on the raw data of our perceptions, and while our perceptions may always feel accurate, we know that in truth they are not. Even if I know that what I am seeing on stage is an illusion, and am able to explain what I have seen to myself in those terms, that illusion still has all the appearance of reality. And in other situations where our perceptions have been led astray we may not have the advantage of knowing that this is the case, meaning that we accept what we perceive at face value. In this chapter we’ll look at some of the ways in which perception and memory interact, and the possible flaws that can therefore be built into a memory right from the moment of its creation.

  #thedress

  ‘White and gold.’

  ‘No! Black and blue!’

  ‘No, it’s obviously white and gold!’

  In early 2015 the media was ablaze with th
is debate. There was a photo of a dress, and some people saw it as being blue with black lace while others saw it as being white with gold lace. Team black and blue and team white and gold dominated many of our Facebook feeds, with friends often becoming quite vocal, convinced that those on the opposing team were colour blind, stupid, or simply lying. Celebrities overloaded the Twittersphere, on teams #whiteandgold or #blackandblue. Dressgate had gone viral.

  Aside from being a great way to waste ten minutes squabbling with incredulous colleagues, that photo of #thedress actually says a lot about how our perception works, and how it can mislead us. Such a dramatic difference in perception seems impossible, as though it is some kind of trick. How could different people possibly see almost opposite colours in exactly the same photo at exactly the same time?

  Apparently I wasn’t the only one who wanted to know more about what was going on here. Three separate papers were published in 2015 on ‘dressgate’. One of the scientists fascinated by the phenomenon was Bevil Conway,1 2 Associate Professor of Neuroscience at Wellesley College, who claimed: ‘This is one of the first [documented] instances, if not the first, of people looking at exactly the same physical thing and seeing very different colours.’ He went on to say, ‘These three papers are the tip of the iceberg. The dress is going to continue to be a very important probe for understanding the fundamental problem of how the brain turns data into perception and cognition: how do you turn the stuff that hits your senses into something actionable, a perception or thought?’

  What the papers found was astonishing to some, but might seem quite obvious to others. In the first study, conducted by Karl Gegenfurtner and his colleagues from Giessen University in Germany,3 the question was simply how most people interpret the colour of the dress. They studied and recorded exactly how a group of people saw the dress, in order to discover which interpretation was most common. They found that rather than having only two possible combinations, participants saw the images on more of a colour continuum, with some seeing the main combinations (black and blue / white and gold) as darker versions of those colours and others seeing more pastel shades. This suggested that there could potentially be other more nuanced teams – team #blackandpastelblue or team #beigeandgold – but it did little to explain why these differences existed in the first place.

  Addressing the why at the heart of the issue, a team from the University of Nevada, spearheaded by Alissa Winkler,4 conducted a study to examine whether perhaps the phenomenon known as colour constancy was one explanatory mechanism here. Colour constancy refers to the way our vision compensates for differences in lighting to estimate what colour things ‘really’ are. It is what allows us to know what colour something is when it is outside exposed to bright sunlight or inside a dimly lit room, and to constantly interpret that colour as the same throughout the day – although the light wavelengths hitting our retinas may change considerably.

  Through their research, Winkler and her team actually found an entirely new way of conceptualising colour perception. They discovered a blue–yellow asymmetry, which means that surfaces are far more likely to be perceived as grey or white when the true colour of the surface is actually bluish than when it has a yellow, red or green tint. They argue that this is because there is a tendency to interpret blue tones as originating from light sources, such as the sky. In the case of the dress, its colours could be interpreted either as a product of lighting or as the actual colours of the fabric.

  What does all this have to do with memory? It’s simple, really. The reason we can have perceptual abilities such as colour constancy is not just because of our amazing physiology, but because we possess fundamental memories informing us about how the world works. We implicitly know that blue tones originate from sources such as the sky because we experience it on an almost daily basis. We have a tremendous wealth of memories that inform us what things look like and what they should look like given certain contextual cues. And we use these memories of our experiences to help us understand the information coming through all of our senses.

  This means that team #whiteandgold interpreted the dress as being dimly lit, meaning the blue tones were interpreted as shade, while team #blackandblue interpreted the dress as being better lit, correctly interpreting its colours as a result. Both teams were using a combination of ocular information and their internal memory-based models of the world.

  If you, like me, found that you saw the dress both ways at different times, you can rest assured that research also supported the idea that the image is multistable – it can be perceived differently by the same person when viewed on separate occasions. Summarising all the work done by different scientists on the phenomenon, Bevil Conway concluded: ‘The dress is a very powerful tool for understanding how the brain resolves ambiguity … there’s a lot of [scientific] interest in how internal models shape how you experience the patterns of life. We really thought everybody had the same internal model.’

  And for all those still wondering, the dress was actually black and blue.

  The dress started a conversation about how we all see the world differently, even through perceptual systems that often seem universal. Of course, this is happening simultaneously with other types of perception, not only sight. Indeed, there are more senses than we traditionally talk about; the myth that we have only five has long persisted, but this does little to give our bodies the credit they deserve.

  In addition to seeing, hearing, feeling, tasting and smelling, we are also constantly processing information about gravity, outside temperature, humidity, internal temperature, the location of our limbs in relation to the rest of us, tiredness, how our internal organs are doing, and muscle tension, to name but a few. And as with the other senses, if something is interpreted incorrectly during any of these many simultaneous processes – each of which is imperfect – it has the potential to introduce error into our memories at their inception.

  One view of the way that we perceive the world is laid out by a model called data-driven or bottom-up processing. It posits that our representation of what the world around us is like is driven almost entirely by the information we collect from our primary senses about our environment, and is minimally shaped by our expectations. This is generally a good model of how our perception works, as most of our experiences must accurately reflect the world around us – otherwise we would find it impossible to navigate our environment.

  As psychologists James and Eleanor Gibson, writing in 1955,5 put it, ‘the stimulus input contains within it everything that the percept has. … Perhaps all knowledge comes through the senses’. The ‘stimulus input’ is the information going into our brains through our senses, and a ‘percept’ is simply a mental concept that is developed as a consequence of the process of perception. For example, if you are looking at a flower, it is a stimulus input because it is stimulating your brain through your eyes. If you are looking at and paying attention to the flower, you perceive the flower, making it the percept.

  In their seminal paper ‘Perceptual learning: differentiation or enrichment?’, the Gibsons try to show that our interpretation of our senses does not necessarily rely on past experience. In their own words, ‘there is no proof that it incorporates memories’. This means that they claim that we can simply experience things, without bringing memory into it. We see a flower as a flower, whether or not we know anything about flowers. We might not call it ‘flower’, but it enters all of our brains with petals, a stem, and leaves. The bottom-up model mostly represents how we may intuitively feel perception should be – an accurate representation of the real world, built from the information our senses give to us about our environment.

  I like it on top

  Most of the time, however, we will be encountering things within a particular context. And we all have a host of complex memories and schemas – intuitive ideas regarding how the world works. We almost never just interpret an object in isolation, but instead bring memory into our interpretation of the world. When we look at a flowe
r we don’t just see colours and shapes, we also know that we are seeing part of a plant, that it is part of a plant known as a ‘flower’, and that we probably should not eat it. We also know that it exists in space and that it needs to follow the rules of gravity. This ability to interpret relatively basic information and make sense of it is surprisingly complex and memory-reliant.

  Here is another example of just how amazing our ability to extrapolate information from very limited input is. Imagine a drawing of a cube. A few lines on a page can represent such an object – the lines being interpreted to represent three dimensions. Although this is our experience of the drawing, in reality there are an almost infinite number of other figures that could be represented by the same set of lines when collapsed but we simply do not think about these infinite alternatives. The reason we can take this simple input and see ‘cube’ is because we have more than just the lines on a page to go by. Our visual systems have evolved to interpret the stimuli around us. They have been honed by our experience with the natural world which has shaped our understanding and expectations. Therefore, when we see a collection of lines that we would ordinarily associate with a three-dimensional cube we use our top-down system to interpret them correctly, even though from a bottom-up perspective they are simply a collection of lines on paper.