Roberts, D.F. and Foehr, U.G. (2004) Kids and Media in America. Cambridge University Press, Cambridge.
Simonton, D.K. (2011) Great Flicks: Scientific Studies of Cinematic Creativity and Aesthetics. Oxford University Press, New York, NY.
Chapter 6
The Cinematic Moment—Emotions and the Comprehension of Movies
When Eternal Sunshine of the Spotless Mind opens, it is Valentine's Day. Joel (Jim Carrey) impulsively skips work to take a train from New York City to Montauk Beach on Long Island. Coming back, he meets Clementine (Kate Winslet). Joel is mopey and withdrawn, while Clementine is outgoing and outrageous. Quirky, likable oddballs, the couple hit it off.
Up to this point, around 10 minutes in, a first-time viewer would have no difficulty following the plot. In fact, most viewers would have a strong intuition for where things are heading: the characters’ lighthearted, slightly neurotic chemistry would indicate we are in the land of romantic comedy. Joel and Clementine will fall in love; there will be complications due to their personality quirks; but in the end, they will wind up happily ever after.
And while the predicted story arc (boy finds girl, boy loses girl, boy regains girl) is fairly accurate, the viewer's journey through this strange film is anything but typical. The first 10 minutes actually represents the second time that Joel and Clementine meet on the train from Montauk. In between the two train trips, they have lived together, broken up, and had their memories erased. The movie flashes back to the fateful night that Joel's memories were deleted using a computer-assisted neurophysiological gadget.
Within this larger flashback, we get other flashbacks of Joel's relationship with Clementine, but not in sequential order. Many of the memories feature physical environments that are literally falling apart, a visual representation of how memories can be destroyed. To make things more difficult, some of the memories of Joel's childhood are transformed by Clementine's intrusion.
Eternal Sunshine of the Spotless Mind is a challenging intellectual puzzle. It takes effort to figure out what is happening. Yet, for many people, it also evokes strong emotions of longing, regret and emotional courage. To watch it, one has to think and feel, usually at the same time. This is true of any movie, but in an unusual film like Eternal Sunshine, we become more aware of what is going on in our heads and our hearts. This chapter isolates the “cinematic moment” when cognitive and emotional processes occur as viewers sit in their seats, gaze at the screen, and try to make sense out of their immediate experience.
Cognitive Psychology and the Movies
Our minds are very active when we watch movies. On the most fundamental level, we perceive the sights and sounds of the movie. The characters in Eternal Sunshine are really just two-dimensional projections of alternating light patterns, yet our visual system makes them appear to be moving human bodies. Furthermore, our hearing system contributes additional information by recognizing voices and separating out background noise.
Along with perception, we must be able to comprehend the story we are watching. We are able to identify the distinct characters of Joel and Clementine. We develop opinions about their personalities. We realize that they are on a train and that they are attracted to each other.
The interrelated mental activities of perception and comprehension are represented in Figure 6.1.1
Figure 6.1 Symbolic activity in film viewing: Comprehension, emotion and perception.
The process of perceiving and comprehending movies is so familiar it is sometimes not even obvious when it is occurring. This is especially true for viewers who have grown up with movies. Classical Hollywood production (clear establishing shots, seamless edits, smooth camera movements, etc.) intentionally tries to make storytelling invisible.2 From this stylistic perspective, a good movie flows so smoothly that we forget we are watching a movie, and it feels effortless. The answers to questions like “Who is who?” and “What is what?” are so evident, it never occurs to most people to ask them.
A closer consideration of the way that movies work reveals a different story. Watching the first 10 minutes of Eternal Sunshine, we must focus on the screen and not the person in front of us. Also we must remember that when Joel starts talking to Clementine, he came to the beach alone. We have to be aware that Joel and Clementine are on the same train, even though the camera only shows one character's face at a time. When the movie moves into its elaborate flashback structure, audience members realize they must piece together the chronology of Joel and Clementine's relationship to make sense of the story.
Perception and comprehension are important topics in the domains of cognitive psychology, cognitive science and neuroscience.3 These fields study various processes that constitute human thinking including sensation, perception, attention, memory, organization, problem solving, and so on. In recent years, cognitive science has had a significant impact on aesthetic criticism. In film studies, David Bordwell has led a movement to extend the precision of cognitive concepts and apply these concepts to narrative comprehension.4 Recently it has become common for film scholars to use vocabulary like “schema,” “long-term memory,” “emotions” and “associative networks” when discussing film techniques or genres. Cognitive concepts such as these have been used to construct new theories of film viewing, narrative comprehension and emotional experience.5 The confluence of film studies and cognitive science is part of an exciting intellectual trend combining scientific methods (experimental and laboratory observations) with the humanities (textual analysis) to help understand not only the perception and comprehension of film, but the human mind itself.
The Perception of Movies
To understand a movie, you must first see it and hear it. Everything we know about visual perception (color, depth, movement) and auditory perception (loudness, pitch, sound localization) is conceivably relevant to experiencing motion pictures. While a thorough explication of the The Godfather in terms of its perceptual components is beyond modern science, film technology is an area where film studies and perception research have historically overlapped.6
One problem in understanding perception and motion pictures is that film images don't really move. A series of 24 successive still images a second are captured by a movie camera and then, at the same rate, projected on a screen by an ultra-bright projector. Each image is briefly frozen before the film advances to the next frame. The actual movement of the film through the camera has to be disguised by the light temporarily flickering between frames (otherwise, the moving image would appear as a blur). Therefore, the viewer sees no real movement while watching a movie—this experienced motion is referred to as “apparent motion.” This technology was figured out by experimental perception researchers and commercial scientists (cameramen and projectionists) in the first decades after movies were invented. Through trial and error (the basis for the scientific method), these scientists and technicians discovered the speed at which film had to move through the camera in order to approximate the perception of real movement.
Later, academics studying visual perception were able to explain the phenomenon of apparent motion in psychological terms; apparent motion was stimulating the same cognitive and physiological pathways as real motion. When the static, pictorial differences between two successive frames of film are so small, the mind can't tell that they're different. Instead, it is fooled into thinking that there is physical continuity (like the way an object moves in the real world) even though such continuity is only an optical illusion.7
Over the years, film directors and cinematographers have developed rules of thumb for editing films in a manner that will orient viewers. Researchers have confirmed that these rules are grounded in the basic realities of human perception. One example is match-action editing, where one shot of a character engaged in some activity is followed by with another shot of the same character from a slightly different perspective. If these cuts are done the wrong way, they disorient the viewers (these are called jump cuts). However, when edits are done right, the audience never no
tices them.
Imagine a character stooping to pick up a diamond necklace that has been thrown to the ground. The audience first sees this action from approximately 20 feet. After a sudden edit, we see the action from the same angle, at a distance of five feet. It appears the character has jumped toward us, which will strike most viewers as unrealistic. Rules of continuity editing suggest that if a director is going to match action, the second shot must be from an angle at least 30° from the first shot. (A sudden edit from 20 to five feet at a significantly different angle will appear natural to the audience and will not be disorienting.)
When we know that we are moving through space (a phenomenon called proprioception, a perception of stimuli coming from the body itself), our visual system integrates multiple perspectives. In crossing a room, we see different objects from many different angles; this is not jarring because we know we are moving. After a 30°+ cut, the focal object appears sufficiently different that we assume we must have moved. However, in a jump cut where there is no discernible difference in the angle of the shots, the only aspect that changes is an object's size. In these situations our perceptual system does not receive the expected cues that we have moved through space, and we therefore assume that the object must have jumped. Since we rationally know this is impossible, the perception is disconcerting.8
Perceptual mechanisms are essential to everyday movie-going enjoyment. Most viewers know little about motion detection, but everyone knows the jarring sensation caused by choppy editing (or avante garde filmmaking such as in Goddard's Breathless). The science of perception is also important to the technological advancements which have made modern fantasy/sci-fi films possible. Although the Oscars for technical achievement are often ignored by the average viewer, such innovations as a new type of lens can alter the audience experience for billions of people.
Julian Hochberg, a vision researcher who has studied film perception, claims that perceptual processes should get more attention from film scholars.9 He believes that the foundations of perceptual processing are hardwired into human biology; they are universal and not determined by cultural variations (such as whether the film is a product of a capitalist society). While Hochberg does not consider culture irrelevant to the production, interpretation and reception of movies, he argues that certain aspects of the film experience do not vary significantly from culture to culture or from person to person.10 Perceptual processes can place certain parameters on how viewers understand a film. The philosophical meaning of Citizen Kane is open to debate, but what is unequivocal is the visual starkness of the black and white photography which allows a greater range of shadings than color film.
The Narrative Comprehension of Movies
Movies tell stories. The perceptual details of a film combine to create an overall structure, and in most films, that structure is narrative. Thus, when viewers try to understand a film, they must look at how the pieces of the story fit together. Bordwell argues that narrative comprehension is an ideal focus for film scholars because it is accessible to concepts of cognitive science and therefore conducive to a rigorous film studies.11
“Story” and “plot” have often been distinguished in narrative analysis.12 Story represents the causal, temporal, and spatial relationships between narrative events (what happens in time and space). The plot refers to what information is presented to the audience in what way (how the story is told). Pulp Fiction has several linear storylines; in each, one event is causally linked to another. The plot of Pulp Fiction, however, is nonlinear, where sequences of different stories are mixed together and presented in nonchronological order. Halfway through the film we see Vincent (John Travolta) get killed by Butch (Bruce Willis), even though Vincent appears in the final sequence, the shootout in the diner that bookends the action. It would be possible to change the plotting of the film to make it more chronological while preserving the same stories. The story is “imaginary”13 (“mental” or “cognitive”) in the sense that it emerges internally as the result of how the viewer processes the plot. Thus, we can say that the plot belongs to the film, while the story belongs to the viewer. Drawing on cognitive science, Bordwell believes that in order to bridge the gap between plot and story, viewers must use a variety of schemas (mental structures for organizing knowledge).14
People tend to use cognitive schemas to establish the consistent identity of things previously encountered. Humans are particularly sophisticated when it comes to differentiating the different people in our lives (friends, family, coworkers, waiters), but we are also able to distinguish significant objects (cars, wallets, clothing). There are individual differences between the quantity of discrete identifications we can make (some people have a better memory for faces or names), but the ability to recognize familiar objects is crucial in daily life, as well as for understanding what's going on in a film. Several types of schemas particularly apply to comprehending film. These are exemplified using Titanic.15
In Titanic we identify the heroes, Jack (Leonardo DiCaprio) and Rose (Kate Winslet), from the villain, Cal (Billy Zane). While this may seem simple enough, we are presented with characters from numerous camera angles, wearing different hairstyles and costumes. Viewers are able to keep track of them because the schema for each character remains constant yet allows for minor variations.
People also rely on schemas to make causal sense of the world. We assume events follow each other and that one event may cause another. As we piece together the events in our lives (some of which we may have witnessed and others we have been told about), we are constantly making adjustments to our schemas to render them plausible. While many Hollywood movies are plotted in sequential order, there are exceptions (including more mainstream films than Pulp Fiction). Titanic’s plot shifts between the modern day exploration of the ocean liner's sunken hull and its actual sailing and sinking in 1912. In order to make these time jumps easier for the audience, the film uses certain associative cues. When we first see Titanic sailing, the photography uses sepia tones recalling photography from the early twentieth century, allowing us to keep past and present separate while remaining aware of a relationship between time periods.
Other schemas are used to maintain an awareness of the physical relationship between spaces. We must be familiar with the organization of a room to locate objects in it (a pencil, the TV remote, or a chair). We need to have a schema for the path from our living room to the kitchen so we can get from one to the other when we get hungry. There are also places outside of our awareness that we know can be accessed using other schemas (e.g., the ability to read a map).
All of these spatial relationships apply to watching a movie, with one exception: we are spatially limited by the camera, not by the movement of our bodies. Therefore, our spatial knowledge is generally not as precise when it comes to movies as it is in real life. In order to make clear sense, Titanic provides specific information about spatial relations. Early in the film we see that the working class passengers are located in the steerage (lower decks) while the affluent passengers are on the upper decks. This relationship is more than just a socioeconomic metaphor but is a feature of the plot when the ship starts to sink. Jack and Rose are trapped among the passengers in steerage; they must keep moving upwards (literally toward the top of the screen) to escape death. Even so, while the audience knows that Jack and Rose are moving toward the upper decks, the rapid editing and the general chaos of the situation are too disorienting to allow for a accurate precise mental map of the ship.
The Emotional Comprehension of Movies
There is a tendency to think of feelings and thoughts as separate—the heart is hot and impulsive, while the head is cold and rational. Historically, psychology treated cognition and emotion as distinct areas of study, but most modern theories of emotion argue that the two cannot be easily separated. Cognitive processes occur virtually simultaneously with emotional processes, and emotional reactions have physiological connections to thought patterns.16 Consequently, recent contributions
to narrative comprehension in film emphasize the emotional dimension, making puns about the “moving” component of moving pictures17 and referring to film as an “emotion machine.”18
Emotional Arousal
When they watch a movie, people experience a range of emotions. The next time you are in a movie theater, watch the people around you. Laughter, tears, expressions of terror, and jolts of surprise can all be witnessed in the audience's physical reactions. Psychological researchers have studied virtually every emotion; in many cases, these studies use movies to evoke emotion. Films are so effective at doing this that they have become a crucial element in emotion research. Typically a movie (or a clip) is shown to participants who are then measured for a particular emotion and compared to other psychological variables such as gender, memory or risk-taking.
Researchers are often clever in devising a range of behavioral, subjective and physiological methods for detecting and measuring emotion.19 Behavioral techniques include videotaping people while they watch a film, then coding their expressions based on the musculature of the face.20 Subjective methods include straightforward questionnaires as well as a more sophisticated technique in which participants control a dial indicating degree of emotion over the course of a movie.21 Physiological methods include measuring skin moisture (galvanic skin response: GSR); brain waves (electroencephalogram: EEG);22 cortisol levels in saliva;23 and genital blood flow24 while subjects watch a film.
Based on such measures, researchers have developed a recommended list of film clips for evoking specified emotions: amusement, anger, disgust, fear, sadness, and surprise.25 The scene from Silence of the Lambs where FBI agent Clarice Starling (Jodie Foster) is trying to find the serial killer hiding in a dark basement is used to evoke fear, while a clip from The Champ where a young boy (Ricky Schroeder) cries over his father's lifeless body is used to evoke sadness. Such clips do not guarantee all audience members will respond the way the researchers expect them to,26 but they have been shown to produce high levels of arousal for specific emotions in many participants under controlled conditions. (See Appendix C for a full list of films and target emotions.)
Psychology at the Movies Page 12