Seeing Voices

by Oliver Sacks

  It has been thought for a century or more (since Hughlings-Jackson’s formulations in the 1870s) that the left hemisphere of the brain is specialized for analytic tasks, above all for the lexical and grammatical analysis that makes the understanding of spoken language possible. The right hemisphere has been seen as complementary in function, dealing in wholes rather than parts, with synchronous perceptions rather than sequential analyses, and, above all, with the visual and spatial world. Sign languages clearly cut across these neat boundaries—for on the one hand, they have lexical and grammatical structure, but on the other, this structure is synchronous and spatial. Thus it was quite uncertain even a decade ago, given these peculiarities, whether sign language would be represented in the brain unilaterally (like speech) or bilaterally; which side, if unilateral, it would be represented on; whether, in the event of a sign aphasia, syntax might be disturbed independently of lexicon; and, most intriguingly, given the interweaving of grammatical and spatial relations in Sign, whether spatial processing, overall spatial sense, might have a different (and conceivably stronger) neural basis in deaf signers.50

  These were some of the questions faced by Bellugi and her colleagues when they launched their research. At the time, actual reports on the effects of strokes and other brain lesions on signing were rare, unclear, and often inadequately studied—in part because there was little differentiation between finger spelling and Sign. Indeed, Bellugi’s first and central finding was that the left hemisphere of the brain is essential for Sign, as it is for speech, that Sign uses some of the same neural pathways as are needed for the processing of grammatical speech—but in addition, some pathways normally associated with visual processing.

  That signing uses the left hemisphere predominantly has also been shown by Helen Neville, who has demonstrated that Sign is “read” more rapidly and accurately by signers when it is presented in the right visual field (information from each side of the visual field is always processed in the opposite hemisphere). This may also be shown, in the most dramatic way, by observing the effects of lesions (from strokes, etc.) in certain areas of the left hemisphere. Such lesions may cause an aphasia for Sign—a breakdown in the understanding or use of Sign analogous to the aphasias of speech. Such sign aphasias can affect either the lexicon or the grammar (including the spatially organized syntax) of Sign differentially, as well as impairing the general power to “propositionize” which Hughlings-Jackson saw as central to language.51 But aphasic signers are not impaired in other, nonlinguistic visual-spatial abilities. (Gesture, for example—the nongrammatical expressive movements we all make [shrugging the shoulders, waving good-bye, brandishing a fist, etc.]—is preserved in aphasia, even though Sign is lost, emphasizing the absolute distinction between the two. Patients with aphasia, indeed, can be taught to use “Amerindian Gestural Code,” but cannot use Sign, any more than they can use speech.)52 Signers with right hemisphere strokes, in contrast, may have severe spatial disorganization, an inability to appreciate perspective, and sometimes neglect of the left side of space—but are not aphasic and retain perfect signing ability despite their severe visual-spatial deficits. Thus signers show the same cerebral lateralization as speakers, even though their language is entirely visuospatial in nature (and as such might be expected to be processed in the right hemisphere).

  This finding, when one considers it, is both startling and obvious and leads to two conclusions. It confirms, at a neurological level, that Sign is a language and is treated as such by the brain, even though it is visual rather than auditory, and spatially rather than sequentially organized. And as a language, it is processed by the left hemisphere of the brain, which is biologically specialized for just this function.

  The fact that Sign is based here in the left hemisphere, despite its spatial organization, suggests that there is a representation of “linguistic” space in the brain completely different from that of ordinary, “topographic” space. Bellugi provides a remarkable and startling confirmation of this. One of her subjects, Brenda I., with a massive right hemisphere lesion, showed a profound neglect of the left side of space, so that when she described her room, she put everything, higgledy-piggledy, on the right side, leaving the left side entirely vacant. The left side of space—of topographic space—no longer existed for her (fig. 3a–b). But in the actual signing, she established spatial loci, and signed freely, throughout the signing space, including the left side (fig. 3c). Thus her perceptual space, her topographic space, a right hemisphere function, was profoundly defective; but her linguistic space, her syntax space, a left hemisphere function, was completely intact.

  Thus there develops in signers a new and extraordinarily sophisticated way of representing space; a new sort of space, a formal space, which has no analogue in those of us who do not sign.53 This reflects a wholly novel neurological development. It is as if the left hemisphere in signers “takes over” a realm of visual-spatial perception, modifies it, sharpens it, in an unprecedented way, giving it a new, highly analytical and abstract character, making a visual language and visual conception possible.54

  Figure 3. A massive lesion in the right cerebral hemisphere destroys Brenda I.’s ability to “map” on the left side, but not her ability to use syntax. Figure (a) shows the actual layout of Brenda’s room, as it would be correctly signed. Figure (b): In describing her room, Brenda leaves the left side bare, and (mentally) piles all the furniture in the right side of the room. She can no longer even imagine “leftness.” Figure (c): But in her actual signing, Brenda uses a full space, including the left side, to represent syntactic relations. (Reprinted by permission from What the Hands Reveal About the Brain, H. Poizner, E. S. Klima, & U. Bellugi. The MIT Press/Bradford Books, 1987.)

  One must wonder whether this linguistic-spatial power is the only special development in signers. Do they develop other, non-linguistic, visual-spatial powers? Does a new form of visual intelligence become possible? This question has led Bellugi and her colleagues to launch a fascinating study of visual cognition in deaf signers. They compared the performance of deaf, natively signing children with that of hearing, nonsigning children on a battery of visual-spatial tests. In tests of spatial construction, the deaf children did much better than the hearing ones, and indeed, much better than “normal.” There were similar findings with tests of spatial organization—the ability to perceive a whole from disorganized parts, the ability to perceive (or conceive) an object. Here again, deaf four-year-olds did extraordinarily well, getting scores that some hearing high school students could not match. With a test of facial recognition—the Benton test, which measures both facial recognition and spatial transformation—the deaf children were again markedly ahead of the hearing children, and far in advance of their chronological norms.

  Perhaps the most dramatic test results have come from deaf and hearing children in Hong Kong, where Bellugi has investigated their ability to perceive and remember meaningless Chinese “pseudo-characters” presented as swift patterns of light. Here the deaf, signing children did startlingly well—and the hearing children were almost unable to do the task at all (see fig. 4). The deaf children, it seems, were able to “parse” these pseudo-characters, to achieve a very complex spatial analysis, and this enormously facilitated their powers of visual perception, enabling them to “see” the pseudo-characters at a glance. Even when the experiment was repeated with deaf and hearing American adults who had no knowledge of Chinese characters, the deaf signers did notably better.

  Figure 4. Asked to reproduce a Chinese pseudo-character (presented as a moving point light display), deaf Chinese children do extremely well, and hearing Chinese children extremely badly. (Reprinted by permission from “Dyslexia: Perspectives from Sign and Script,” U. Bellugi, Q. Tzeng, E. S. Klima, and A. Fok. In From Reading to Neurons, A. Galaburda, ed., The MIT Press/Bradford Press, 1989.)

  These tests, in which signing children perform far above normal levels (a superiority that is especially marked in the first few years of life
), all emphasize the special visual skills learned in acquiring Sign. As Bellugi notes, the spatial organization test involves not only the recognition and naming of objects, but also mental rotation, form perception, and spatial organization, all of which are relevant to the spatial underpinnings of Sign syntax. The ability to discriminate faces, and recognize subtle variations of facial expression, also carries extreme importance to the signer, since facial expression plays an important role in ASL grammar.55

  The ability to separate discrete configurations, or “frames,” from a continuous stream of movement (as was done with the Chinese pseudo-characters) brings out another important ability of signers—their enhanced power of “movement parsing.” This is seen as being analogous to the ability to break down and analyze speech from a continuous and ever-changing pattern of sound waves. All of us have this ability in the auditory sphere—but only signers have it so dramatically in the visual sphere. And this too, of course, is essential to the comprehension of a visual language, which is extended in time as well as in space.

  Is it possible to detect a cerebral basis for such enhancement of spatial cognition? Neville has studied the physiological correlates for such perceptual changes, by measuring changes in the brain’s electrical responses (evoked potentials) to visual stimuli in particular, movements in the peripheral visual field. (Enhanced perception of such stimuli is crucial in Sign communication, for the signer’s eyes are generally fixed on the other’s face, and signing movements of the hands therefore lie in the periphery of the visual field.) She has compared these responses in three groups of subjects: deaf native signers, hearing non-signers, and hearing native signers (usually born of deaf parents).

  Deaf signers show greater speed of reaction to these stimuli—and this goes with an increase of evoked potentials in the occipital lobes of the brain, the primary reception areas for vision. Such increases of speed and occipital potentials were not observed in any of the hearing subjects, and seem to reflect a compensatory phenomenon—the enhancement of one sense in place of another (greater auditory sensitivities, similarly, may occur in the blind).56

  But there were also enhancements at higher levels: the deaf subjects showed greater accuracy in detecting the direction of motion, especially when the movement was in the right visual field, and coincident with this was an increase in evoked potentials in the parietal regions of the left hemisphere. These enhancements were also observed in the hearing children of deaf parents and have therefore to be seen not as an effect of deafness, but as an effect of the early acquisition of Sign (which demands very superior perception of visual stimuli). It is not only detection of motion in the peripheral field that is shifted, in signers, from being a right hemisphere to a left hemisphere function. Neville and Bellugi obtained evidence—indeed, quite early on—for a similar left hemisphere specialization (and shift from the “normal” right hemisphere specialization) in deaf signers for picture identification, dot localization, and the recognition of faces.57

  But the very greatest enhancements were observed in deaf signers—and in these, intriguingly, the enhancement of evoked potentials spread forward into the left temporal lobe, which is normally regarded as purely auditory in function. This is a very remarkable and, one suspects, fundamental finding, for it suggests that what are normally auditory areas are being reallocated, in deaf signers, for visual processing. It constitutes one of the most astonishing demonstrations of the plasticity of the nervous system, and the extent of its adaptability to a different sensory mode.58

  Such a finding also raises fundamental questions as to the extent to which the nervous system, or at least the cerebral cortex, is fixed by inborn genetic constraints (with fixed centers and fixed localization—areas “hardwired,” “preprogrammed,” or “prededicated” for specific functions) and to what extent it is plastic and may be modified by the particularities of sensory experience. The famous experiments of Hubel and Wiesel have shown how greatly the visual cortex may be modified by visual stimuli, but leave unclear how much input merely kindles built-in potentials, and how much it actually shapes and molds them. The experiments of Neville suggest a shaping of function to experience—for it can hardly be supposed that the auditory cortex has been “waiting” for deafness, or visual stimulation, to become visual and change its character. It is very difficult to explain such findings except by a radically different sort of theory, one that does not see the nervous system as a universal machine, hardwired and preprogrammed for (potentially) everything, but sees it as becoming different, as free to take on completely different forms, within the constraints of what is genetically possible.

  To comprehend the significance of these findings one also needs a different way of looking at the cerebral hemispheres and their differences and their dynamic roles in dealing with cognitive tasks. Such a way has been provided by Elkhonon Goldberg and his colleagues in a series of experimental and theoretical papers.

  Classically the two cerebral hemispheres are seen as having fixed (or “committed”) and mutually exclusive functions: linguistic/nonlinguistic, sequential/simultaneous, and analytic/gestalt are among the dichotomies suggested. This view runs into obvious difficulties when one confronts a visuospatial language.

  Goldberg would first enlarge the domain of “language” to one of “descriptive systems” in general. Such descriptive systems, in his formulation, constitute superstructures imposed on elementary “feature detection” systems (for example, those of the visual cortex), a variety of such systems (or “codes”) being operative in normal cognition. One such system is, of course, natural language; but there may be many others—such as formal mathematical languages, musical notation, games, etc. (insofar as these are encoded by special notations). It is characteristic of all of these that they are first approached in a tentative, groping way but later acquire an automatic perfection. Thus there may be with these, and with all cognitive tasks, two ways of approach, two cerebral “strategies,” and a shift (with the acquisition of skill) from one to the other. The right hemisphere’s role, as thus conceived, is critical for dealing with novel situations, for which there does not yet exist any established descriptive system or code—and it is also seen as playing a part in assembling such codes. Once such a code has been assembled, or emerged, there is a transfer of function from right to left hemisphere, for the latter controls all processes that are organized in terms of such grammars or codes. (Thus a novel linguistic task, even though it is linguistic, will initially be processed predominantly by the right hemisphere, and only subsequently become routinized as a left hemisphere function. And a visuospatial task, conversely, even though it is visuospatial, will, if it can be embedded in a notation or code, come to show a left hemisphere superiority.)59

  With such an approach—so different from the classical doctrines of fixed hemispheral specificities—one can understand the role of the individual’s experience and his development, as he moves from his first gropings (in linguistic or other cognitive tasks) toward expertise and perfection.60 (Neither hemisphere is “more advanced” or “better” than the other; they are merely suited for different dimensions and stages of processing. Both are complementary, interacting; and between them they allow the mastery of new tasks.) Such a view makes clear, without paradox, how Sign (though visuospatial) can become a left hemisphere function, and how many other sorts of visual ability—from perception of movement to perception of patterns, perception of spatial relation to perception of facial expressions—by having become part of signing, will be swept along with it, as it develops, into becoming left hemisphere functions too. We can understand why the signer becomes a sort of visual “expert” in many ways, in certain nonlinguistic as well as linguistic tasks—how there may develop not just visual language but a special visual sensibility and intelligence as well.

  We need more hard evidence about the development of a “higher” visuality, a visual style, comparable to the evidence Bellugi and Neville have provided about the enhancement of “lower�
�� visual-cognitive functions in the deaf.61 As yet we have mostly anecdotes and accounts; but the accounts are extraordinary and demand close attention. Even Bellugi and her colleagues, who rarely depart from rigorously scientific description, are moved to include the following brief account, in passing, in their book, What the Hands Reveal About the Brain:62

  We first saw this mapping aspect of signing in its full form when a visiting deaf friend was telling us about his recent move to new quarters. For five minutes or so, he described the garden cottage in which he now lived—rooms, layout, furniture, windows, landscaping and so forth. He described it in exquisite detail, and with such explicit signing that we felt he had sculpted the entire cottage, garden, hills, trees and all in front of us.

  What is related here is difficult (for the rest of us) to imagine—it has to be seen. It is very similar to what Charlotte’s parents say of her—her ability to create a real (or fictional) landscape with such precision, such fullness, such aliveness, as to transport the beholder. The use of such picturing, pictorial power, goes with the use of Sign—even though Sign is not in the least a “picture-language” itself.

  The other side of this linguistic expertise, and visual expertise generally, is the tragically poor linguistic and intellectual function that may afflict a large number of deaf children. It is clear that the high linguistic and visual competences of the well-functioning deaf lead to the establishment of strong cerebral lateralization, with a shift of language functions (and also visual-cognitive functions generally) to a well-developed left hemisphere. But what, we have to wonder, is the situation, neurologically, with the poorly functioning deaf?

  Rapin was struck by “a remarkable linguistic deficiency” in many of the deaf children she works with—specifically, an inability to understand question forms, to understand the structure of sentences—an inability to manipulate the language-code. Schlesinger shows us other dimensions of this deficiency, dimensions that expand it from the linguistic to the intellectual: the low-functioning deaf, in her description, not only have difficulty in the understanding of questions, but refer only to objects in the immediate environment, do not conceive remoteness or contingencies, do not formulate hypotheses, do not rise to superordinate categories, and are in general confined to a preconceptual, perceptual world. She feels that their utterances are somewhat deficient syntactically and semantically, but clearly are also deficient in a much deeper sense.