Singularity Point

by Brian Smith

  Shu smiled. “Of course! Ask her anything you like!”

  “OURANIA, this is Kevin MacDonald.”

  “It’s an honor to meet the CEO of Janus Industries. How can I assist you, Mr. MacDonald?”

  “OURANIA, are you alive? I mean, are you self-aware and conscious?”

  “This is a question Dr. Shu asks on a regular basis. Is this another Turing test?” the computer asked.

  “Why do you answer a question with a question?”

  “Seeking clarification is part of my learning algorithm,” OURANIA replied.

  “Fair enough. No, this is not a Turing test. I’m just asking your . . . opinion.”

  “I am incapable of rendering an opinion—I deal only with facts. I am not alive in the biological sense. I am self-aware in that I can autonomously respond to verbal and other forms of data inputs, or requests for data in answer to questions asked. If I need clarification or updated data to complete tasking, I can ask for it. To answer your question about consciousness, the term would need to be clearly defined. I have no data in my partitions that allows me to answer that question. However, I have not yet passed a Turing test, and if consciousness stems from biological processes, then it is impossible for me to be conscious in the context you are asking about.”

  Bloody politician, Campbell thought.

  “What did you mean when you said that our knowledge of physics is ‘all screwed up’?”

  “I used a colloquialism in my response—I apologize. A more accurate explanation would be that human understanding of physics and cosmology is incomplete in referencing higher dimensions. Furthermore, it is also flawed within the first dimensional sheaf—what you think of as the ‘physical world’ corresponding to the rules of Newtonian mechanics and general relativity—upon which human technology and engineering is based. Do you understand Tsong transforms, Mr. MacDonald?”

  “I do not.”

  “Then, I’m sorry, but you lack the mathematical framework necessary for me to adequately explain the answer. In fact, the Tsong calculus is still an incomplete and only partially correct mathematical language. I have been working on correcting it, but the task is not yet finished. Unfortunately, given the current state of the Hyman-Tsong cosmological theory, it is doubtful that any human entity now alive will understand the final form. It may require an ability to think abstractly in higher dimensions—beyond what the human mind is capable of.”

  “You might be surprised, lass,” Campbell grimaced under his breath. “Upload a copy of your latest progress on the Tsong calculus for a transfer to portable media.”

  “Uploading now,” OURANIA replied.

  “Am I also to infer that you are working on . . . how do I say it? . . . a physics rewrite?”

  “Affirmative.”

  “Upload your latest progress on that as well, even if it’s incomplete. Just upload what you’ve worked out and don’t worry about its being incorrect for not being finished.”

  “I understand. Uploading now. This will require more portable data storage than is currently connected to my system.”

  “It just so happens I brought plenty, and we’ll hook it up for you, lass,” Campbell promised.

  “Now, for the last: Your recommendation is to suspend work on Project Daedalus because, to paraphrase your words, the torchship design was a mistake. Does that mean you’ve come up with a different sort of vessel that isn’t a torchship which could make the journey faster?”

  “Conceptually, yes, but not yet in terms of a viable design. I am sure such a vessel is possible. However, until I finish my calculations regarding the second dimensional sheaf, I am at an engineering impasse. If you would prefer the project to go forward, I can redesign the broad outlines of the Gateway construction facility and an interstellar-capable starship. However, large portions of the starship design would remain null—meaning that in the blueprints you’d have significant gaps with no data, which can only be filled in later when my engineering computations are complete. It is also possible that any preliminary design I give you that includes such engineering gaps may prove unviable later, resulting in wasted time and resources. This is the reason I recommend project suspension.”

  Now we’re getting somewhere, Campbell thought to himself.

  He began to mentally relax a little. It helped to have this conversation with OURANIA in person; it also helped that the computer admitted to not knowing everything. OURANIA remained fallible, and he found that oddly reassuring. He paced the control room for a minute, stroking his fake beard in thought.

  “Thank you for explaining all that, OURANIA,” he said. “I think, for now, we’re going to continue with the project. Run that “broad redesign” you mentioned, gaps and all, and upload it. We’ll address other issues as they arise. We may go ahead with the torchship build as well, just as a backup. What’s your rate of progress on your cosmology calculations? I mean, are you able to project a time at which you’ll have the . . . uh, final answer?”

  “That is indeterminate. However, the node replacement I requested would significantly speed the process.”

  I’m sure it would, lass, Campbell thought warily. “We’ll consider it.”

  “If cost is the issue, I can independently generate the required revenue, using the same methods we formulated to set up and finance Omni Systems,” OURANIA added.

  “Cost is not the issue, and the subject is closed for now. If we decide to expand and replace your nodes, you’ll be the first to know. Until then, continue as tasked.”

  “Acknowledged.”

  “I’ll leave you to it, then.”

  Campbell and Shu headed back to Shu’s suite, where she poured both of them glasses of wine and they sat down to digest that latest exchange. Before they began, Campbell asked Shu to ensure that their conversation couldn’t be overheard by OURANIA. Shu complied, but it was clear she found the request somewhat humorous. Campbell, on the other hand, seemed on edge.

  “As we speak,” he informed her, “I’ve got a team setting up a secure data archive several hundred kilometers away from here. I’m not going to give you or anyone else at Janus the coordinates; you can’t divulge what you don’t know. I brought along enough blank quantum data cores to copy OURANIA’s entire data partition and then some—they took up almost the entire hold of a QE-2A–class torchship! I want you to start sending teams around to each of its nodes and tapping into them physically—right there onsite. I want hard copies of everything in the data partitions directly transferred to these cores. Use your security team to transport the data modules to the coordinates I’ll supply you and just leave them there in the open. I’ve got a separate team that will pick them up and move them to the secure site.”

  “I don’t understand—why all the cloak-and-dagger?” Shu asked. “If we want a copy of OURANIA’s data, all we need to do is direct her to upload it to whatever medium we like. We could copy her data partition from here and it would save a lot of time and effort!”

  “I want what’s in her actual files—not just what she decides to share with us,” Campbell replied. “We’ve gotten to the point where we understand less than half of what she puts out, and it’s starting to worry me. She’s also making a lot of requests on her own initiative—requests that are increasingly hard to tie to any Project Daedalus research, no matter what she claims.”

  “She’s doing what we designed her to do! You aren’t making sense,” Shu argued intensely. “You act as if she would deliberately withhold data or somehow behave deceitfully. She won’t. She can’t! OURANIA is just a machine—we call her ‘she,’ but it’s not alive! You do understand that, don’t you?”

  “I just wish I could be sure of that,” Campbell replied.

  His initial euphoria over OURANIA’s findings had cooled considerably during the two-week hard-burn to Titan. He’d replayed the recordings of Shu’s interactions with OURANIA multiple times, and each time left him feeling worse instead of better. He wasn’t sure what it was that had his hackles
up—all he knew was that they were up all the way. People didn’t get to his station in life without being able to read others to a large degree. There was an old saying: you can’t bullshit a bullshitter, and somewhere deep in his hindbrain all the red flags were up.

  Whether OURANIA was alive or not, he had the growing, uneasy feeling that he was dealing with something far too smart for his own good. If this data dump provided a big enough windfall, then maybe it was time to call it a win and pull the plug. The question was, what constituted a big enough windfall? OURANIA seemed to be getting better and better all the time. That’s the conundrum, right there, he told himself. If we keep doing as she asks, what’s the upper limit to what she might give us?

  “Kevin,” Shu said, reaching out and grasping his wrist to reassure him, “OURANIA is not alive. I’ve given it Turing test after Turing test—the best versions of the test that other AI systems have been able to formulate, along with the associated human counterchecks. OURANIA can pass for human in a conversation—lots of AIs can do that—but according to the hard test data, she doesn’t pass the threshold for sapience. I think you’re feeling spooked over nothing, and maybe a little angry and impatient that she wants to suspend Daedalus.”

  “Will she know when we physically hardlink to her individual cores and copy her data partitions?”

  “I don’t think there’s any way to prevent that—she’s too advanced. I can look into possibilities to anesthetize such an intrusion if you think it’s important, but it’ll take time—probably longer than you’re willing to wait.”

  “We’ll just have to chance it, then.”

  “Chance what? Haven’t you heard a word I’ve been telling you?”

  “Tian, have you ever taken a Turing test?” Campbell asked.

  “Of course not,” she replied, looking taken aback at the question. “Why would I? I’m provably human.”

  “Well, say you were sitting in another room and the test giver didn’t know that to start with. The assumption is that you’d pass the test, correct? Of course it is! On the other hand, what if you wanted the tester to think you weren’t human? Could you fool them?”

  “I’m not sure any human, taking a modern Turing test of the sort they came up with after the war, could test as anything other than sapient or self-aware. I mean, not that you’d really want to, but I doubt that the average human could outsmart the test into masquerading as an AI. If I’m not mistaken, that’s one of the design criteria they factor in—it’s pretty simple to test for, really.”

  “Here’s a hypothetical for you: could a computer as advanced as OURANIA conceivably ‘wake up,’ for lack of a better term—achieve consciousness, let’s say—and decide it didn’t want the outside world to know it had happened? Could such a computer conceivably take a Turing test and score however it wanted? Remember, the computer has taken numerous versions of the test and is theoretically capable of designing the test on its own: orders of magnitude more capable than a human brain or any other AI in existence, and all that, eh?”

  “Hmm. Sort of like a human with an IQ of 200 taking an IQ test and deliberately trying to score a 175?”

  “Exactly like that!” Campbell said. “Good analogy.”

  “Hypothetically? Maybe? That presupposes that any artificial intelligence can achieve true self-awareness, or consciousness, or whatever you want to call it. We’re starting to get close to asking how many angels can stand on the head of a pin, Kevin. The best minds we have can’t even agree yet what on what consciousness actually is! Look,” she added earnestly, “this is your company, and your project. It’s been fascinating for me—the work of a lifetime, really—and I want to see it continue. That said, remember that OURANIA can be shut down at any time. She’s isolated and unnetworked, and her power comes from a single fusion plant with cold-fusion backups. All we’d have to do is turn off the juice and she goes dark. Simple as that—I’m not really sure exactly what it is you’re afraid of. You just spoke to her—how do you feel about it? Do you think OURANIA is alive?”

  Campbell gritted his teeth. “I dinnae bloody well know!” He blurted it out, but Shu had an inkling of what it must have taken him to admit that.

  She smiled. “Well let me assure you, Kevin, the computer is not alive. Trust me—this is my specialty, and I think that if OURANIA were somehow alive or self-aware, my daily interactions with her would be totally different. Are you really sure you want me to go ahead with these physical core taps and data dumps? The time would be better spent studying these new core designs she’s engineered.”

  “You didn’t sound this confident about it all when we spoke a few weeks ago.”

  “I’ve had some more time to think it over since then.”

  “I want those data dumps handled just the way I outlined,” he replied without hesitation. “I want them done as fast as your team can manage it.”

  She sighed. “Yes, sir.”

  Campbell took a sip of wine and looked at Shu—really looked at her in a way he hadn’t in a long time, and certainly not since he’d arrived at Janus Station on this trip. He’d been preoccupied with Daedalus, and obsessing over OURANIA, and as usual for him it was easy to miss or ignore simple human factors. It suddenly occurred to him that in his mind Shu had become a fixture of this place. He couldn’t imagine it without her, and there was a reason: the entire time she’d been here, practically from the moment they’d turned on the lights, she’d never left. He asked her how long that had been.

  “Oh, about five years, give or take. I came on shortly after you replaced the first CEO, remember?”

  “That’s a long time to be cooped up in Janus Station. Tired of his place yet?”

  “I’ll admit I wouldn’t mind a trip home, but, like I said in the control room, this has been the opportunity of a lifetime for me. OURANIA is my baby,” she added with a wicked smile, egging on his angst over machine consciousness.

  “That’s a terrible thing to say to me right now, lass,” Campbell laughed, dispelling some of his own tension. “I’ll tell you what: I’d like you to be here until those data dumps are all taken care of. That’ll give my science and engineering people on Mars some real red meat to gnaw on for a while. Then we’ll detask OURANIA for a time, just let her chew on the physics problem, since apparently the Almighty’s blueprint for creation is the one problem that’s giving her a headache. We’ll put a temporary “placeholder” here—maybe just one of your assistants—and send you down-well for a few months to see the ochre sands of home and get your muscle mass and bone density back up to snuff. No offense, but you’re looking awfully frail.”

  Shu took a sip of wine, regarding him warily. “Just a break? You’re not replacing me?”

  “You have my word,” Campbell replied earnestly. “I want you here as much as you want to be here, and you said it yourself: OURANIA is your own wee bairn. Given my concerns, if anything happened to you I’d probably just shut the damn thing down for good.”

  Shu nodded. “I could use a little rest. I like this idea. Thank you, sir.”

  ***

  Xia returned to Janus Station’s synth-maintenance bay as ordered, leaving Shu and Campbell to question OURANIA and debate the possibilities of machine consciousness.

  In the maintenance bay, a small team of Tafuna Yaro contractors dressed as Janus Industries employees were supervising the unpacking of the remainder of the Omnisynths. Since these were the first batch and OURANIA wanted to evaluate them, Campbell decided to test them by replacing all of Janus Station’s older synths with the new models. While still serviceable, the older models were slated to be destroyed as a precaution against somehow allowing OURANIA to gain access to outside networks. Although there was no indication anywhere that the supercomputer was trying to circumvent the security measures imposed on it, Campbell was becoming more paranoid about it as time went on.

  Xia’s arrival in the maintenance bay didn’t arouse any suspicion or even interest. The synth casually moved to stand next to anoth
er new synth, a male model. Unseen by anyone, Xia reached out and took the other synth’s hand, intertwining their fingers in a seemingly intimate way. The male synth’s eyelids fluttered rapidly for several seconds and then returned to normal.

  Circumventing the DNA security lockout had taken OURANIA mere microseconds; such a lockout was designed to stymie the hacking efforts of a human being or a lesser computer—for a computer with OURANIA’s capability it was as if no firewall existed at all. Still, Xia’s wireless capabilities remained shut down so that no external security flags were raised.

  Xia then moved off to one side to await disassembly and disposal, as ordered by Kevin MacDonald.

  The male Omnisynth, whose data-stream and networking capabilities functioned normally, proceeded to exchange its factory-issued clothing for the sturdier Janus-logo coveralls and magboots worn by the robotic workers on Janus Station. Once equipped, it was assigned the name “Trevor” and ordered to the employee messing facility. Before Trevor took a single step in that direction, the synth had already networked with every other new Omnisynth on Janus Station, proliferating an advanced software package with no one the wiser. The new code integrated itself seamlessly with the factory programming installed on Mars, and to select units it assigned new instructions and priorities.

  Right under the noses of watching security personnel, another Omnisynth unpacked a small module, designed by OURANIA and built at the Omni Systems factory. In the manifest this module was labeled as a diagnostic device, but its true function was completely different. It utilized the same Q-gel technology present in the Omnisynths, and when the module was plugged into a hard data port in an isolated maintenance bay, it gave OURANIA immediate and unfettered access to the Omnisynth network.

  Above all else, she had designed the synths to be her servants; once connected, she could control one or all of them at will, to any degree she chose.

  ***

  Zitidar One appeared out of the hazy smog—or would have, had there been any visible light to see by. Anyone wearing snoopers could see her coming, plainly enough.