I decided that my main long-term interest was in the problem of consciousness, though I realized that it would be foolish to start with this. Consciousness is most apparent in man—at least I know I am conscious and I have good reasons to suspect that you are too. Whether a fruit fly is conscious is an open question. There are, however, grave experimental handicaps to working on human beings, since so many experiments are impossible for ethical reasons. It seemed reasonable, therefore, to concentrate on animals close to man in evolution; that is, the mammals and in particular the primates—the monkeys and apes.
My next problem was to choose some particular aspect of the mammalian brain. As I knew very little I decided to make the obvious choice and concentrate on the visual system. Man is a very visual animal (as are monkeys), and much work had already been done on many aspects of vision.
How can one study vision in man by working on monkeys? The obvious approach is to do what one can on man, and, in parallel, study the same system in a monkey or other mammal. In work on perception, it is now becoming standard practice to use arguments from detailed psychophysical studies on man (plus rather cruder psychophysical studies on a monkey) combined with all the neuroanatomical and neurophysiological knowledge available on the relevant part of a monkey’s brain. Occasionally other data from man can be used, such as evoked potentials (a type of brain wave), or various rather expensive scans, but as yet these have a much lower resolution, either in space and/or time, and thus usually give us much less information.
This is why, to someone like myself, the visual system is attractive since, as far as we can tell, a macaque monkey sees in much the same way as we do. There are, of course, few subjects more important to us than language, since it is one of the main differences between man and all lower animals. Unfortunately, for this very reason, there is no suitable animal for such studies. This is why I believe that modern linguistics, sophisticated though it is, will run into a brick wall unless much more can be found out about what happens inside our heads when we talk, listen to speech, and read. If language is anything like as complex as vision (which seems more than likely), the chance of unscrambling the way it really works without this extra knowledge seems to me to be rather small. Linguists, not surprisingly, usually find this argument unacceptable.
I also decided that, at least at first, I would not attempt to do experiments. Apart from the fact that, technically, they are often very difficult, I thought I could contribute more from a theoretical viewpoint. It seemed to me that I might perform a useful function by studying the problem of vision from as many points of view as possible. I hoped that I might help to build bridges between the various scientific disciplines, all of which studied the brain from one point of view or another. I had rather little expectation of producing any radically new theoretical ideas at such an advanced age, but I thought I might interact fruitfully with younger scientists. In any case I expected that the subject would prove endlessly interesting and that at my time of life I had a right to do things for my own amusement, provided I could make an occasional useful contribution.
Having decided that I could learn about the mammalian visual system, my next problem was to select which aspect to study first. I had never had a medical education, so my knowledge of neuroanatomy was almost zero. I decided to tackle that first, as I expected it to be the dullest part of the subject. It would be as well, I thought, to get it out of the way before going on to other, more interesting, topics.
To my surprise I soon discovered that there had been a minor revolution in dry-as-dust neuroanatomy. Thanks mainly to the introduction of various rather simple biochemical techniques, it was now possible to discover how the various regions of the brain were connected together. Moreover, the techniques were not only powerful but considerably more reliable than most of the older methods. Unfortunately most of them cannot be used on humans (one cannot, at the end of an experiment, “sacrifice” the graduate student who has been acting as the subject, as one can do with animals, for obvious ethical reasons). We thus have the curious situation that more is known about neural connections in the brain of the macaque monkey than about those in the human brain. In fact, we shall soon know so much about the broad pattern of connections in the macaque, and about the location in the brain of various chemical transmitters and the receptors for them, that the only way to cope with all this new information will be to store it in computers, in such a way that it can be displayed in some vivid graphic form for easy comprehension.
I first started by reading experimental papers and reviews. I found it was not difficult to approach experimentalists provided one was genuinely interested in what they were doing and had first made some effort to discover from their publications what they were up to. In this way I made many new friends, far too many to list here. I was lucky in finding in La Jolla several people interested in vision or in theory. A group at the Psychology Department at the University of California, San Diego (UCSD), under the leadership of Bob Boyton, studied mainly the psychophysics of vision. Other psychophysicists I got to know were Don MacLeod and V. S. (“Rama”) Ramachandran when he came to San Diego from Irvine. I also interacted with another group in the same department, led then by David Rumelhart and Jay McClelland, that did theoretical work. After a while the department appointed me an adjunct professor of psychology, in spite of my very flimsy knowledge of the subject.
In 1980 Max Cowan came to the Salk, setting up a large group of neuroscientists there. Some of these people, such as Richard Andersen (now at M.I.T.) and Simon LeVay do experimental work on the visual system. Although Max left in 1986, the Salk still has a strong interest in neuroscience and has recently recruited Tom Albright, an experimentalist from Princeton.
Another blessing was the arrival, in 1984, of the Canadian philosophers Paul and Pat Churchland, to take up chairs in the philosophy department at UCSD. It is unusual to find philosophers who are even remotely concerned about the brain, so it is a great help to have the advice of two people who do take a keen interest in it. Both had written very well on reductionism (a dirty word to some, especially to those who regard me as an archreductionist). More recently Pat has written a large book, called Neurophilosophy, published by the Bradford Book section of the M.I.T. Press, setting out the philosophical, theoretical, and experimental aspects of their new point of view. Its subtitle is “Towards a Unified Science of the Mind-Brain.”
Ramachandran and Gordon Shaw (a physicist at U.C. Irvine) were the co-founders of the Helmholtz Club, named after the nineteenth-century German physicist who pioneered the scientific study of perception. The members meet about once a month, starting with lunch and ending with dinner. In between we have talks by two speakers, on topics mostly connected with the visual system. This schedule allows plenty of time for discussion. The meetings are held at Irvine, which is midway between Los Angeles and San Diego, so that members and guests from the other university campuses can attend.
This is not the place for me to attempt to outline what we now know about the visual system—that would take at least another whole book—let alone the rest of the brain. I will restrict myself to rather general comments. In the first place, it is not obvious to most people why we need to study vision. Since we see so clearly, without any apparent effort, what is the problem? It comes as a surprise to learn that in order to construct our vivid mental representation of the outside world, the brain has to engage in many complex activities (sometimes called computations) of which one is almost completely unaware.
We succumb all too easily to the Fallacy of the Homunculus—that somewhere attached to our brain there is a little man who is watching everything that is going on. Most neuroscientists don’t believe this (Sir John Eccles is an exception) and think that our picture of the world and of ourselves is due solely to neurons firing and other chemical or electrochemical processes inside one’s body. Exactly how these activities give us our vivid picture of the world and of ourselves and also allow us to act is what we want to discover.
The main function of the visual system is to build a representation inside our head of objects in the world outside us. It has to do this from the complex signals reaching the retinas of our eyes. Though these signals have much information implicit in them, the brain needs to process this information to obtain explicit representations of what interests it. Thus the photoreceptors in our eyes respond to the wavelength of the impinging light coming from an object. But what the brain is mainly interested in is the reflectivity (the color) of an object, and it can extract this information even under quite different conditions of illumination of that object.
The visual system has been evolved to detect those many aspects of the real world that, in evolution, have been important for survival, such as the recognition of food, predators, and possible mates. It is especially interested in moving objects. Evolution will latch onto any features that will give useful information. In many cases the brain has to perform its operations as quickly as possible. The neurons themselves are inherently rather slow (compared to transistors in a digital computer) and so the brain has to be organized to carry out many of its “computations” as quickly as possible. Exactly how it does this we do not yet understand.
It is very easy to convince someone that however he may think his brain works, it certainly doesn’t work like that. That misunderstanding can be demonstrated from the effects of human brain damage, or by psychophysical experiments on undamaged humans, or by outlining what we know about monkey brains. What seems a uniform and simple process is in fact the result of elaborate interactions between systems, subsystems, and sub-subsystems. For example, one system determines how we see color, another how we see in three dimensions, (although we receive only two-dimensional information from each of our two eyes), and so on. One of the subsystems of the latter depends on the difference between the images in our two eyes; this is called stereopsis. Another deals with perspective. Another uses the fact that objects at a distance subtend a smaller angle than when they are nearer to us. Others deal with occlusion (one object occluding part of an object behind it), shape-from-shading, and so on. Each of these subsystems may well need sub-subsystems to make it work.
Normally all the systems produce roughly the same answer, but by using tricks, such as constructing rather artificial visual scenes, we can pit them against one another and so produce a visual illusion. If a person looks, with one eye through a small hole, into a room built with false perspectives, an object on one side of the room can be made to appear smaller than the same object on the other side. Such a full-scale room, called an Ames room, exists at the Exploratorium in San Francisco. When I was looking into it some children appeared to be running from side to side. They appeared to grow taller as they ran to one side and to get shorter again as they ran back to the other side. Of course, I know full well that children never change height in this way, but the illusion was nevertheless completely compelling.
The conception of the visual system as a bag of tricks has been put forward by Rama Ramachandran, mainly as a result of his elegant and ingenious psychophysical studies. He calls his point of view the utilitarian theory of perception, writing:
It may not be too farfetched to suggest that the visual system uses a bewildering array of special-purpose tailor-made tricks and rules-of-thumb to solve its problems. If this pessimistic view of perception is correct, then the task of vision researchers ought to be to uncover these rules rather than to attribute to the system a degree of sophistication that it simply doesn’t possess. Seeking overarching principles may be an exercise in futility.
This approach is at least compatible with what we know of the organization of the cortex in monkeys and with François Jacob’s idea that evolution is a tinkerer. It is, of course, possible that underlying all the various tricks there are just a few basic learning algorithms that, building on the crude structures produced by genetics, produce this complicated variety of mechanisms.
Another thing I discovered was that although much is known about the behavior of neurons in many parts of the visual system (at least in monkeys), nobody really has any clear idea how we actually see anything at all. This unhappy state of affairs is usually never mentioned to students of the subject. Neurophysiologists have some glimpses into how the brain takes the picture apart, how somewhat separate areas of our cerebral cortex process motion, color, shape, position in space, and so on. What is not yet understood is how the brain puts all this together to give us our vivid unitary picture of the world.
I also discovered that there was another aspect of the subject one was not supposed to mention. This was consciousness. Indeed an interest in the topic was usually taken as a sign of approaching senility. This taboo surprised me very much. Of course, I knew that until recently most of the experiments on the visual system of animals were done when the animals were unconscious under an anesthetic so that, strictly speaking, they could not see anything at all. For many years this did not unduly disturb the experimentalists, since they found that the neurons in the brain, even under these restrictive conditions, behaved in such interesting ways. Recently more work has been done on alert animals. Although these animals are technically rather more difficult to study, there are compensations, since the animals are returned to their cages after a normal day’s work and the experimenter can go home to supper. Such animals are usually studied for many months before being sacrificed. (Experiments on anesthetized animals can be much more demanding since they usually last for many many hours at a stretch, after which the animal is sacrificed straight away.) Curiously enough, hardly any experiments have yet been done on the same sort of neurons, in the same animal, first when it is alert and then when it is under an anesthetic.
It was not only neurophysiologists who disliked talking about consciousness. The same was true of psychophysicists and cognitive scientists. A year or so ago the psychologist George Mandler did organize a course of seminars at the psychology department at UCSD. The seminars showed that there was hardly any consensus as to what the problem was, let alone how to solve it. Most of the speakers seemed to think that no solution was possible in the near future and merely talked around the subject. Only David Zipser (another ex-molecular biologist, now at UCSD) thought as I did, namely that consciousness was likely to involve a special neural mechanism of some sort, probably distributed over the hippocampus and over many areas of the cortex, and that it was not impossible to discover by experiment at least the general nature of the mechanism.
Curiously enough, in biology it is sometimes those basic problems that look impossibly difficult to solve which yield the most easily. This is because there may be so few even remotely possible solutions that eventually one is led inexorably to the correct answer. (An example of such a problem is discussed toward the end of chapter 3.) The biological problems that are really difficult to unscramble are those where there is almost an infinity of plausible answers and one has painstakingly to attempt to distinguish between them.
One main handicap to the experimental study of consciousness is that while people can tell us what they are conscious of (whether they have suddenly lost their color vision, for example, and now only see everything in shades of gray), it is more difficult to obtain this information from monkeys. True, monkeys can be laboriously trained to press one key if they see a vertical line and another if they are shown a horizontal one. But we can ask people to imagine color, or to imagine they are waggling their fingers. It is difficult to instruct monkeys to do this. And yet we can look inside a monkey’s head in much more detail than we can look inside a person’s head. It is therefore not unimportant to have some theory of consciousness, however tentative, to guide experiments on both humans and monkeys. I suspect that consciousness may be able to do without a fully working long-term memory system but that very short-term memory is indispensable to it. This suggests straight away that one should look into the molecular and cellular basis of very short-term memory—a rather neglected subject—and this can be done on animals, even on
a cheap and relatively simple animal like a mouse.
And what of theory? It is easy to see that theory of some sort is essential, since any explanation of the brain is going to involve large numbers of neurons interacting in complicated ways. Moreover, the system is highly nonlinear, and it is not easy to guess exactly how any complex model will behave.
I soon found that much theoretical work was going on. It tended to fall into a number of somewhat separate schools, each of which was rather reluctant to quote the work of the others. This is usually characteristic of a subject that is not producing any definite conclusions. (Philosophy and theology might be good examples.) I renewed acquaintance with the theorist David Marr (whom I had originally met in Cambridge) when he came with another theorist, Tomaso (Tommy) Poggio, to the Salk for a month in April 1979 to talk about the visual system. Alas, David is now dead, at the early age of thirty-five, but Tommy (now at M.I.T.) is still alive and well, and has become a close friend. Eventually I met many of the theorists working on the brain (too numerous to list here), mainly by going to meetings. Some I got to know better from personal visits.
Much of this theoretical work was on neural nets—that is, on models in which groups of units (somewhat like neurons) interact in complicated ways to perform some function connected, often rather remotely, with some aspect of psychology. Much work was being done on how such nets could be made to learn, using simple rules—algorithms—devised by the theorists.
A recent two-volume book, entitled Parallel Distributed Processing (PDP), describes much of the work done by one school of theorists, the San Diego group and their friends. It is edited by David Rumelhart (now at Stanford) and Jay McClelland (now at Carnegie-Mellon) and published by Bradford Books. For such a large, rather academic book it has proved to be a best-seller. So striking are the results that the PDP approach is having a dramatic impact both on psychologists and on workers in artificial intelligence (AI), especially those trying to produce a new generation of highly parallel computers. It seems likely to become the new wave in psychology.
What Mad Pursuit Page 19