IGMS - Issue 15

by IGMS

  VINGE: Alas, that's one ugly possibility. In part it depends on how much payoff there is for a few smart persons compared to what those same people could have being part of a much larger community of smart persons. If the first ones figure they are in a positive-sum game, then they would probably be inclusive. Personally, I think that -- besides the natural risks inherent in power tools -- the rise of technology has brought an overwhelming affirmation that we are in a positive-sum game. If it's perceived as a zero -- or a negative-sum game -- then things could be very bad. If the first to be smart are willing to use force to prevent other players from getting super-intelligence, then this seems much like the classic "AIs enslave us."

  I have friends who would prefer to have a pure AI rather than Intelligence Amplification of humans. They point out that we're carrying fifty million years of evolutionary bloody baggage in the back of our heads and so we just can't be trusted to the extent you might trust a machine that doesn't have that instinctual killer inclination. I have one friend who makes this argument and then taps his chest and says, "In fact, there is only one person I would trust to undertake this responsibility." So if one figures that the threat in your question is convincing, that the first enhanced humans are going to become gods and reduce us all to serfs -- then one is reasonably nervous about Intelligence Amplification.

  SCHWEITZER: It might not be a matter of active malevolence, but of a lot of people being left behind. The nation or social group which enhances itself first will inevitably win. If we can make ourselves ten times smarter, immune to most diseases, and much stronger, then the people who have not become supermen are much less employable. Let us throw into this the idea that the enhanced people can interact with machines in a way that unaltered people cannot. The science fiction precedent I see for this is Asimov's division, in his robot novels, of the human race into short-lived people and the long-lived spacers, who are almost two different species. Couldn't we see the human race divided into two species, those who are enhanced and those who are left behind?

  VINGE: Actually, I think that will happen if there is enhancement without coercion. A certain percentage of people will reject the technology. There are all sorts of reasons for a person to reject it. I think some of the reasons might be sound. And just from the viewpoint of hedging humanity's risks, I'd hope there would be folks who would not opt in.

  The mellow version of this could be the scenario where some people say, "Hell, no! I've got legal title to my property, and I've paid off my mortgage. My I.Q. is 130, and I'm satisfied. So you guys that are smarter, I don't want to buy into what you have. I'll just stay where I am, and you will have to find somewhere else to play your game."

  There are number of interesting variations here: One is where the stay-behinds are not allowed to opt-in later, and come to feel seriously downtrodden. Another might be where the stay-behinds are doing better than any prior civilization in human history, but from the viewpoint of post-humans, those stay-behinds are living in squalor.

  SCHWEITZER: So some people opt out, and they're still the equivalent of stone-age farmers plowing a field behind a mule. Everybody else goes high-tech. The inferiors just get ignored.

  VINGE: And "ignored" could mean several things. Of the benign variants: I think it's possible that the post-humans wouldn't consider Earth the most profitable real estate. If so, the stay-behinds might be literally so, perhaps protected as an emergency backup in case of some post-human catastrophe. (I'll bet sf readers can think of several stories of this sort! When it comes to scenarios, science-fiction has outposts scattered all over the map. That's a strength, but also the reason why the field should not be considered prophetic.) Another variant might be where the post-humans haven't all gone away. It might involve an intermediate level of entrepreneurs who would provide access to design studios that are run by smart people.

  SCHWEITZER: You realize, of course, that a good deal of the population of the world has no idea what you are talking about, because they have never seen a computer.

  VINGE: I think your main point is correct. Today. But the last ten years have changed the part about having computers -- cell phones! The fact that this worldwide diffusion could happen in just ten years is to me an astonishing fact, and gives some reason to believe that these issues are not going to remain the obscure obsession of a small minority.

  SCHWEITZER: They're also used with those previously mentioned killer instincts to set off remote-controlled bombs. Maybe nothing has really changed since the discovery of fire, or before.

  VINGE: Yes indeed. Will our creativity kill us before it saves us? I think we have a good chance at success, but we have only one example to look at.

  SCHWEITZER: Can't we argue that the first step in this future evolution of mankind is science fiction itself, in the sense that you and I are using a science-fictional method of thinking even to have this discussion?

  VINGE: Yes. It seems to me that science fiction plays for the body politic the role that dreaming does for the individual. Most dreams are nonsense. Some dreams clue you in to things that you should be worrying about, like, "My God, if I forget to pay that bill, there could be some really bad consequences." In the later Twentieth Century, the notion of thinking out scenarios, which we have been doing in science fiction since forever, began to be a serious bureaucratic planning tool. I think it is often superior to using forecasts and trend-lines. In fact, no one knows what the future is going to be like. Earlier in this interview you were talking about dystopian possibilities related to intelligence amplification. It's important for people to work through all sorts of scenarios like this. Science fiction is a relatively light-hearted way of doing this.

  When it's done more seriously (as in companies such as GBN) it can be part of an overall planning strategy: You come up with a scenario and then work backwards from that, thinking, "If that's really how things turned out, what would be the symptoms that we would see as we fall into that scenario?" Do that reasoning with a number of contrasting scenarios. Then you have these families of symptoms for different sorts of outcomes. To me, this is a far more effective way of dealing with uncertainty than simple forecasts. As time goes on, you can watch for symptoms on your various lists. You can say, well, if such-and-such happens, that makes it more likely that we are in Scenario A or Scenario C, in which case it would be good to spend some money on such-and-such. Of course, this also feeds back into further scenario generation.

  SCHWEITZER: Does this then give the science fiction writer a certain responsibility to be realistic?

  VINGE: I think that a story should be true to itself, but that's a much broader requirement than what most people mean by "realism."

  SCHWEITZER: You're talking about science fiction as a kind of dreaming. A lot of science fiction makes no attempt to be realistic. Think of van Vogt's The World of Null-A. It was supposed to be a big revelation, but there is nothing realistic in it at all.

  VINGE: I haven't read The World of Null-A in a long time, so I shouldn't try to comment on it specifically. I have noticed that there are so many different categories of quality that a story can be an abject failure in some ways and still be a treasure for what it does in others.

  SCHWEITZER: Aren't we describing only a specific type of science fiction, if the subject of realism comes up?

  VINGE: The dreaming metaphor probably covers most types of science fiction (and the superset of sf, fantasy). It's not reasonable to have a preset definition of serious science fiction. And taking science fiction writers as a whole, there is no reason why they have to take themselves seriously. Now, the people who read their stories get various different things out of them. I read some people's stories for reasons that may be fairly serious, but for many, the more crazy the better, if the author can handle it cleverly and/or with emotional impact.

  Now it's true that the kind of sf writer who is invited to planning meetings is normally of the more realistic variety. But their value may be as loose cannons, crazy enough to mix things up
without turning the boat over. And even the non-sf scenario-based planners can be affected by oddball stories. An emotionally evocative story can put your head in a different place, and suddenly facts have new connections.

  SCHWEITZER: I am thinking of the Balonium Factor. Balonium is that substance or field effect which suspends the laws of the universe as needed for the plot. Doesn't any given science fiction contain something which is probably impossible? If Wells had known how to build a time machine, he wouldn't have written a story about it. He would have done so, and collected his Nobel Prize. What he was actually doing was pretending that someone had discovered a new scientific principle which made this possible. So, in a science fiction story, how much extrapolative realism is required, and how much balonium is desirable.

  VINGE: [Laughs] There are different grades of balonium. At one extreme, there are stories describing inventions that may well be patentable. At the other extreme, there is fantasy relabeled. I think that it's unwise to say, "beyond this point, the balonium is pure." Sometimes you read something that seems like pure balonium, but if you look at the story in the right way, you say, "Oh, geez, this situation could be caused by X," where X has nothing to do with the balonium. The area of computers may be the most fertile source of such surprises, because there are many crazy and magical things you can do if you have a proper distributed system setup. For science fiction, it's good that we don't have any predefined proper balance between realism and balonium, that that remains a matter of taste and saleability.

  SCHWEITZER: The idea that what you can imagine might actually become possible must have itself only been possible when people became aware that the past was different from the present.

  VINGE: Yes! This and your point about Wells's time machine could be part of a framework describing much of human progress. Certainly, if we really knew how to make some bit of future super-science, we'd do it now and it wouldn't be future super-science anymore. In fact, even if we don't know the details, just knowing that something desirable is possible is enough to prod some humans into the invention. And then there is the most extreme case, namely just imagining that something is desirable being enough to get the invention.

  In the millennia of pre-history, I imagine that the idea of human progress was a rare notion -- and that was an important reason why change was so slow! The acceleration of progress through the Renaissance was partly due to "compound interest" growth of knowledge, but also to the notion that progress was possible. One might argue that part of the acceleration during the Industrial Revolution was due to smart people consciously focusing on the importance of invention.

  Somewhere I read that in the late 1940s, Soviet nuclear espionage did not get detailed engineering data that made a big difference. The most important thing the spies delivered were statements such as, "Yes, this approach works. You can do it this way." That by itself was enough to point their physicists to success. Now in the first part of the twenty-first century, humanity is toying with the more dubious extremes of this progression, wondering where is the balance between the wish and the fact.

  SCHWEITZER: But, for example, Lucian of Samosata could imagine going to the Moon. He made a joke out of it. He wrote something like a Douglas Adams story. But for thousands of years thereafter there was no feeling that merely because you could imagine going to the Moon that one day it would be possible. Maybe somewhere around the 18th century did this begin to change. People had imagined flying for a very long time, but only about then did more than a very few people realize that they actually could. There was Leonardo back around 1500, but he was way ahead of his time and pretty much alone.

  VINGE: Yes, the wish and the fact can be very far apart -- or near enough to be extremely frustrating. I believe Benjamin Franklin once speculated that within two hundred years or so, we would have prolongevity. Too bad for Ben. Maybe too bad for you and me.

  End of Part One. Part Two will appear in IGMS issue 16.

  For more from Orson Scott Card's InterGalactic Medicine Show

  go to http://www.InterGalacticMedicineShow.com

  Copyright © 2009 Hatrack River Enterprises