JERRY LETTVIN AND THE BIRTH OF MOTHER AND GRANDMOTHER CELLS
Jerry Lettvin originated the term “grandmother cell” around 1969 in his MIT course on “Biological Foundations for Perception and Knowledge.” When discussing the problem of how neurons can represent individual objects, he told a (tall) tale of how the neurosurgeon A. Akakhievitch had located a group of brain cells that “responded uniquely only to a mother . . . whether animate or stuffed, seen from before or behind, upside down or on a diagonal or offered by caricature, photograph or abstraction.”3 At this point the mother-obsessed character from Philip Roth’s novel Portnoy’s Complaint4 appeared and Akakhievitch ablated all of the mother cells in Portnoy’s brain. As a result, Portnoy completely lost the concept of his mother. (See box 12.1.) Akakhievitch then went on to the study of “grandmother cells.”
From this origin, the term grandmother cell seems to have spread so quickly that Horace Barlow, in his 1972 paper “Single units and sensation: A neuron doctrine for perceptual psychology,”5 didn’t even explicitly define the term in criticizing the idea, and in 1973 Colin Blakemore could write of the “great debate [that] has become known as the question of the ‘grandmother cell.’ Do you really have a certain nerve cell for recognizing the concatenation of features representing your grandmother?”6
Box 12.1
Lettvin’s Story about Mother and Grandmother Cells (ca. 1969)
In the distant Ural mountains lives my second cousin, Akakhi Akakhievitch, a great if unknown neurosurgeon. Convinced that ideas are contained in specific cells, he had decided to find those concerned with a most primitive and ubiquitous substance—mother. . . . And he located some 18,000 neurons that responded uniquely only to a mother, however displayed, whether animate or stuffed, seen from before or behind, upside down or on a diagonal, or offered by caricature, photograph, or abstraction.
He had put the mass of data together and was preparing his paper, anticipating a Nobel prize, when into his office staggered Portnoy, world-renowned for his Complaint (Roth, 1969). On hearing Portnoy’s story, he rubbed his hands with delight and led Portnoy to the operating table, assuring the mother-ridden schlep that shortly he would be rid of his problem.
With great precision he ablated every one of the several thousand separate neurons and waited for Portnoy to recover. We must now conceive the interview in the recovery room.
“Portnoy?”
“Yeah.”
“You remember your mother?”
“Huh?”
(Akakhi Akakhievitch can scarcely restrain himself. Dare he take Portnoy with him to Stockholm?)
“You remember your father?”
“Oh, sure.”
“Who was your father married to?”
(Portnoy looks blank)
“You remember a red dress that walked around the house with slippers under it?”
“Oh, certainly.”
“So who wore it?”
(Blank)
“You remember the blintzes you loved to eat every Thursday night?”
“They were wonderful.”
“So who cooked them?”
(Blank)
“You remember being screamed at for dallying with shikses?”
“God, that was awful.”
“So who did the screaming?”
(Blank)
And so it went. . . . It made no difference—Portnoy had no mother. “Mother” he could conceive—it was generic. “My mother” he could not—it was specific. . . .
Akakhievitch then . . . went back to . . . “grandmother cells.”
This parable is abridged from a letter Lettvin sent Horace Barlow in 1995 (described in Barlow, 1995). Much earlier, Barlow (1953) had described cells in the frog’s retina as “bug detectors,” but little notice had been taken. In the late 1950s Lettvin and his colleagues at MIT were studying these and other complex cells in the frog, but again the mainstream neuroscience community had ignored their work. Thus at a 1959 meeting on sensory communication at MIT, Barlow (respectable for other reasons by then) but not Lettvin had been invited; Barlow arranged for some of the participants to see experiments in Lettvin’s lab. Subsequently a paper by Lettvin and his colleagues was added to the end of the meeting’s Proceedings (Lettvin et al., 1961) and eventually “What the frog’s eye tells the frog’s brain” (Lettvin et al., 1959) became well known. Presumably, Lettvin’s previous research on how the frog’s retina codes complex stimuli was related to his story about mother and grandmother cells.
JERZY KONORSKI’S GNOSTIC UNITS
Although unknown to Lettvin, the grandmother cell idea had actually been set out in detail as a serious scientific proposal a few years earlier by the Polish neurophysiologist and neuropsychologist Jerzy Konorski in his Integrative Activity of the Brain,7 a wide-ranging set of speculations on the neurophysiology of perception and learning (see figure 12.1 and box 12.2). His ideas on the organization of the cerebral cortex in perception anticipated subsequent discoveries to an amazing degree. Konorski predicted the existence of single neurons sensitive to complex stimuli such as faces, hands, emotional expressions, animate objects, locations, and so on (see figure 12.2). He called them “gnostic” neurons, and they were virtually identical to what later were called “grandmother cells.” He suggested that the gnostic neurons were organized into specific areas of the cerebral cortex he termed “gnostic fields.” That is, he predicted (correctly in many cases) the existence of areas of the cortex devoted to the representations of such things as faces, emotional expressions, places, and spatial relations. Destruction of a gnostic field would lead, he predicted, to what were later described as category-specific agnosias. Furthermore, he localized many of these gnostic fields, such as the face field in ventral temporal cortex and the space field in posterior parietal cortex (figure 12.3). Overall, these gnostic fields and their locations are remarkably similar to contemporary views of the putative functions of extrastriate cortex based on monkey single-neuron studies and human imaging experiments.8
Figure 12.1
Jerzy Konorski in front of the Nencki Institute of Experimental Biology in Warsaw. Photograph 1961 by the author.
Box 12.2
Jerzy Konorski (1903–1973)
Konorski’s speculations about gnostic cells came at the end of a long and distinguished career studying the brain and behavior (Konorski, 1967, 1974; Fonberg, 1974). As medical students in Warsaw he and Stefan Miller discovered that there was another type of conditioned reflex other than the one discovered by Pavlov, namely one under the control of reward. They called it Type II to distinguish it from Pavlov’s, which they called Type I. Subsequently and independently, Skinner made this same distinction and Konorski and Miller’s Type II conditioning became known as operant or instrumental conditioning.
After a few years as a psychiatrist, Konorski spent two years in Pavlov’s laboratory in Leningrad but never convinced the master that there really were two types of conditioned reflexes. Konorski then returned to Warsaw and set up a conditioning laboratory in the Nencki Institute of Experimental Biology. He also married and collaborated with Dr. Liliana Lubinska, who had studied neurophysiology in Paris, and through her became familiar with Western and particularly Sherringtonian neurophysiology. When the war started, Konorski was extraordinarily fortunate to be able to escape Poland. (His colleague Miller committed suicide when the Nazis arrived.) His Russian friends got him appointed head of the famous primate laboratory at Sukhumi on the Black Sea. The laboratory eventually moved to Tbilisi as the Germans approached. It was still near the front, and Konorski had a great deal of experience treating head wounds in a nearby army hospital. At the end of the war he returned to Poland and played a major role in reconstructing Polish neuroscience as head of the Department of Neurophysiology at the Nencki Institute.
In 1948 Cambridge University Press published his Conditioned Reflexes and Neuron Organization, which was an attempt to bring Pavlovian reflexology in line with Sherringtonian neurophysiology. In
the 1960s there were close collaborative relations between Konorski’s laboratory and researchers at the Laboratory of Neuropsychology at the National Institutes of Health (Hal Rosvold, Mort Mishkin, and Patricia Goldman). In addition to the concepts of gnostic units and gnostic fields, Konorski’s Integrative Activity of the Brain (1967) contains many important and influential ideas about learning and memory.
Figure 12.2
This illustration is taken from Konorski, 1967, figure III-1, “Particular categories of visual stimulus-objects probably represented in different gnostic fields.” (a) Small manipulable objects; (b) larger partially manipulable objects; (c) nonmanipulable objects; (d) human faces; (e) emotional facial expressions; (f ) animated objects; (g) signs; (h) handwriting; (i) positions of limbs. Used by permission of University of Chicago Press.
At the time of their publication, there was nothing in the literature anything like Konorski’s ideas of highly specialized perceptual neurons in mammals or of areas of the cortex devoted to the representations of particular classes of visual stimuli. In retrospect, however, it is possible to delineate the origins of Konorski’s speculations.
Konorski’s views of the neural organization of perception were a synthesis and extension of three lines of work in the decade before the publication of his book. The first was Hubel and Wiesel’s demonstration of the hierarchical processing of sensory information: how as one proceeds from center-surround to simple receptive fields and then to complex and then the (now revised) hypercomplex ones the selectivity of the cells increases and their ability to generalize across the retina increases.9 The possibility that this hierarchy of increasing stimulus specificity continues beyond visual areas V2 and V3 was made explicit in their 1965 paper, which is repeatedly cited by Konorski. That paper ends as follows:
How far such analysis can be carried is anyone’s guess, but it is clear that the transformations occurring in these three cortical areas [V1, V2 and V3] go only a short way toward accounting for the perception of shapes encountered in everyday life.10
Figure 12.3
From Konorski, 1967. “Conceptual map of the human cerebral cortex.” A, anterior; P, posterior; L, lateral; M, medial. Projective fields are hatched; gnostic fields are plain. The modality boundaries are thick lines. The arrows denote connections. The numbers in parentheses are tentative correspondences with Brodmann’s areas. The letters signify the gnostic fields shown in figure 12.2. V, visual analyzer: V-I (Brodmann’s area 17); V-II (18); V-III (19); V-Sn, sign visual field (7b); V-MO, field for small manipulable objects (7b); V-VO, field for large objects (39); V-Sp, field for spatial relations (39, right hemisphere); V-F, field for faces (37); V-AO, field for animated objects (37). A, auditory analyzer: A, projective auditory field (41, 42); A-W, audio-verbal field (22); A-Sd, field for various sounds (22, right hemisphere); A-VO, field for human voices (21). The legends for the symbols for the somesthetic (S) and kinesthetic (K) fields have been omitted. Ol, olfactory analyzer; E, emotional analyzer. Used by permission of University of Chicago Press.
A second line of inspiration for Konorski’s ideas was the research by Karl Pribram and his students, particularly Mort Mishkin, on the cognitive effects of lesions on what was then called “association cortex” in monkeys.11 From his close association with Hal Rosvold and Mishkin (both commented on earlier drafts of his book) Konorski was well aware that lesions of inferior temporal (IT) cortex produced specific impairments in visual cognition in monkeys and that similar areas of “association cortex” existed for audition and somesthesis. Today, at the annual meeting of the Society for Neuroscience, there are multiple sessions on IT cortex under the general rubric of “Vision.” However, at that time most visual neurophysiologists had never heard of this area and did not realize that it had visual functions, let alone that it sat at the top of a series of hierarchically arranged extrastriate visual areas. Indeed, although V2 and V3 had been described, no other extrastriate visual areas such as MT or V4 were known until 1971.12 Citing Pribram and Mishkin,13 Konorski wrote:
In monkeys the gnostic visual area seems to be localized in inferotemporal cortex, as judged from numerous experimental results in which ablations of this region produced impairment of visual discrimination.14
The third line of evidence for his theories of gnostic neurons and areas came from Konorski’s familiarity with the various agnosias that follow cortical lesions in humans from his own clinical experience, from the Western neuropsychological literature, and from Luria’s work in the Soviet Union. He was aware of both the symptomatic specificity of some cases of agnosia and their tendency to be localizable. Furthermore, unlike most contemporary neuropsychologists and neurophysiologists he was aware of the similarity of human agnosias to the effects of experimental lesions in monkeys. For example, he directly related prosopagnosia, or face agnosia, after ventral temporal lesions in humans to the visual learning deficits in monkeys after inferior temporal lesions.
In summary, Konorski’s prophetic ideas on gnostic neurons and gnostic fields came from a bold extension of Hubel and Weisel’s findings to account for specific cognitive effects of specific lesions in monkeys and humans.
Konorski’s book received a long and laudatory review in Science (by me).15 However, for at least the next decade virtually all of the many citations to the book were to the parts concerned with learning rather than perception; learning theory still dominated American psychology. As described in the next section, the ideas on gnostic neurons did influence one laboratory, namely mine, the laboratory that first reported (the predicted) neurons in IT cortex that selectively respond to faces and hands.
In the last decade gnostic cells have begun to be commonly mentioned in textbooks and in the vision and pattern-recognition literature, usually as synonyms for grandmother cells and usually in the context of inferior temporal cortex cells.
THE DISCOVERY OF FACE- AND HAND-SELECTIVE CELLS IN INFERIOR TEMPORAL CORTEX OF THE MONKEY
In the late 1960s my colleagues and I reported visual neurons in inferior temporal cortex of the monkey that fired selectively to hands and faces.16 These observations were probably primed by our familiarity with Konorski’s gnostic units as well as the propinquity of Lettvin’s work on detectors in the frog’s eye,17 Hubel and Wiesel’s discoveries on the hierarchical processing in cats and monkeys, and local talk about grandmother cells. Starting about twelve years later, these findings were replicated and extended in a number of laboratories18 and were often viewed as evidence for grandmother cells. Konorski himself saw them as confirming his ideas of gnostic cells.19 For some time these cells were the strongest evidence for the existence of grandmother/gnostic cells. However, there has been no good evidence for cells from monkeys that are selective for other visual objects important or common for monkeys such as fruit, tree branches, monkey genitalia, or other features in their natural environments. However, inferior temporal cells can be trained to show great specificity for arbitrary visual objects and these would seem to fit the requirements of gnostic/grandmother cells.20 Furthermore, there is now good evidence for cells in the human hippocampus that have highly selective responses to gnostic categories, including highly selective responses to individual human faces.21
However, most of the reported face-selective cells do not really fit a very strict criteria of grandmother/gnostic cells in representing a specific percept, that is, a cell narrowly selective for one face and only one face across transformations of size, orientation, and color.22 Even the most selective face cells usually also discharge, if more weakly, to a variety of individual faces. Furthermore, face-selective cells often vary in their responsiveness to different aspects of faces, suggesting that they form ensembles for the coarse or distributed coding of faces rather than detectors for specific faces. Thus, a specific grandmother may be represented by a specialized ensemble of grandmother or near-grandmother cells.
There are two reasons why the members of face-coding ensembles may appear more specialized than the members of other
stimulus-encoding ensembles, that is, why there are many more face cells than banana cells. First, it is more crucial for a monkey (or human) to differentiate among faces than among any other categories of stimuli such as bananas. Second, faces are more similar to each other in their overall organization and fine detail than any other stimuli that a monkey must discriminate among. If there had been strong selective pressure for a monkey to distinguish individual bananas, it would probably have ensembles for doing so that were made up of cells selective for banana in general, but which showed graded response to different characteristics of bananas.
LABELED LINES AND HIERARCHIES
Two central characteristics of grandmother/gnostic cells have a long history. The first is that they are examples of labeled line coding, and the second is that they are at the top of a hierarchy of increasing convergence.
Labeled line coding refers to activity in a neuron coding a particular stimulus property, such as red, or a grandmother. This specificity derives from the connections of the neuron, not from the pattern of the neuron’s firing as is the case for various temporal codes such as rate, latency, or phase locking. Perhaps the earliest notion of a labeled line was in Galen’s distinction between sensory and motor nerves in the second century23 (see chapter 2). On the basis of his experience as physician to the gladiatorial school in Pergamon he realized that section of some nerves resulted in a specific sensory loss and section of others resulted in motor loss. He thought the distinction derived from the connections of the nerves to specific regions of the brain.
The first modern labeled line theory of vision was Thomas Young’s trichromatic theory of color.24 Johannes Müller (1838) then generalized this idea to all the senses in his doctrine of specific nerve energies.25 In that doctrine, when a given nerve type (or nerve energy in his terms) is excited, the same type of experience is produced independent of the cause of the excitation. The first example of labeled line coding by single-neuron activity was probably Adrian and Matthews’ finding that action potentials in a given optic nerve fiber of the conger eel signaled the photic stimulation of a specific part of the eel’s retina.26
A Hole in the Head Page 22