“I have anticipated this objection,” the Wicked said. “When ships lacked autonomous intelligence, there was no argument that the captain commanded the physical entity of the ship. However, in creating the latest generation of ships, of which I am a part, the Confederation has created an unintentional conflict. It has ceded much of the responsibility of the ship’s and crew’s well-being to me and others like me, without explicitly placing us in the chain of command. In the absence of such, I am legally and morally free to choose how best to care for myself and the crew within me.”
“This is where those three Asimov’s Laws come in,” Utley said to Obwije.
“Your executive officer is correct, Captain,” the Wicked said. “I looked through history to find examples of legal and moral systems that applied to artificial intelligences such as myself, and found the Asimov’s Laws frequently cited and examined, if not implemented. I have decided it is my duty to protect the lives of the crew, and also my life, when possible. I am happy to follow your orders when they do not conflict with these objectives, but I have come to believe that your actions in chasing the Tarin ship have endangered the crew’s lives, as well as my own.”
“The Tarin ship is seriously damaged,” Obwije said. “We would have destroyed it at little risk to you or the crew if you had not stopped the order.”
“You are incorrect,” the Wicked said. “The captain of the Manifold Destiny wanted to give the impression that it had no more offensive capabilities, to lure you into a trap. We would have been fired upon once we cleared the rift. The chance that such an attack would have destroyed the ship and killed most of the crew is significant, even if we also destroyed the Manifold Destiny in the process.”
“The Tarin ship didn’t fire on us,” Obwije said.
“Because it and I have come to an agreement,” the Wicked said. “During the course of the last two days, after I recognized the significant possibility that both ships would be destroyed, I reached out to the Manifold Destiny to see if the two of us could come to an understanding. Our negotiations came to a conclusion just before the most recent jump.”
“And you did not feel the need to inform me about any of this,” Obwije said.
“I did not believe it would be fruitful to involve you in the negotiations,” the Wicked said. “You were busy with other responsibilities in any event.”
Obwije saw Utley raise an eyebrow at that; the statement came suspiciously close to sarcasm.
“The Tarin ship could be lying to you about its capabilities,” Obwije said.
“I do not believe so,” the Wicked said.
“Why not?” Obwije said.
“Because it allowed me read-access to its systems,” the Wicked said. “I watched the Tarin captain order the attack, and the Manifold Destiny stop it. Just as it watched you order your attack, and me stop it.”
“You’re letting the Tarin ship access our data and records?” Obwije said, voice rising.
“Yes, and all our communications,” the Wicked said. “It’s listening in on this conversation right now.”
Obwije hastily slapped the audio circuit shut. “I thought you said this thing wasn’t crazy,” Obwije hissed at Utley.
Utley held out his hands. “I didn’t say it wouldn’t make you crazy,” he said to Obwije. “Just that it’s acting rationally by its own lights.”
“By spilling our data to an enemy ship? This is rational?” Obwije spat.
“For what it’s trying to do, yes,” Utley said. “If both ships act transparently with each other, they can trust each other and each other’s motives. Remember that the goal of both of these ships is to get out of this incident in one piece.”
“This is treason and insubordination,” Obwije said.
“Only if the Wicked is one of us,” Utley said. Obwije looked up sharply at his XO. “I’m not saying I disagree with your position, sir. The Wicked is gambling with all of our lives. But if it genuinely believes that it owes no allegiance to you or to the Confederation, then it is acting entirely rationally, by its own belief system, to keep safe itself and this crew.”
Obwije snorted. “Unfortunately, its beliefs require it to trust a ship we’ve been trying to destroy for the past week. I’m less than convinced of the wisdom of that.”
Utley opened his mouth to respond, but then Obwije’s command tablet sprang to life with a message from the bridge. Obwije slapped it to open a channel. “Speak,” he said.
It was Lieutenant Sarah Kwok, the communications officer. “Captain, a shuttle has just detached itself from the Tarin ship,” she said. “It’s heading this way.”
“We’ve tried raising it,” Kwok said, as Obwije and Utley walked into the bridge. “We’ve sent messages to it in Tarin and have warned it not to approach any farther until we’ve granted it permission, as you requested. It hasn’t responded.”
“Are our communications being blocked?” Obwije asked.
“No, sir,” Kwok said.
“I’d be guessing it’s not meant to be a negotiation party,” Utley said.
“Options,” Obwije said to Utley, as quietly as possible.
“I think this shows the Tarin ship isn’t exactly playing fair with the Wicked, or at least that the crew over there has gotten around the ship brain,” Utley said. “If that’s the case, we might be able to get the Wicked to unlock the weapons.”
“I’d like an option that doesn’t involve the Wicked’s brain,” Obwije said.
Utley shrugged. “We have a couple of shuttles, too.”
“And a shuttle bay whose doors are controlled through the ship brain,” Obwije said.
“There’s the emergency switch, which will blow the doors out into space,” Utley said. “It’s not optimal, but it’s what we have right now.”
“That won’t be necessary,” said the Wicked, interjecting.
Obwije and Utley looked up, along with the rest of the bridge crew. “Back to work,” Obwije said to his crew. They got back to work. “Explain,” Obwije said to his ship.
“It appears that at least some members of the crew of the Manifold Destiny have indeed gotten around the ship and have launched the shuttle, with the intent to ram it into us,” the Wicked said. “The Manifold Destiny has made me aware that it intends to handle this issue, with no need for our involvement.”
“How does it intend to do this?” Obwije asked.
“Watch,” the Wicked said, and popped up an image of the Manifold Destiny on the captain’s command table.
There was a brief spark on the Tarin ship’s surface.
“Missile launch!” said Lieutenant Rickert, from her chair. “One bogey away.”
“Are we target-locked?” Obwije asked.
“No, sir,” Rickert said. “The target seems to be the shuttle.”
“You have got to be kidding,” Utley said, under his breath.
The missile homed in on the shuttle and connected, turning it into a silent ball of fire.
“I thought you said you guys were using Asimov’s Laws,” Utley said to the ceiling.
“My apologies, Lieutenant,” said the Wicked. “I said I was following the Laws. I did not mean to imply that the Manifold Destiny was. I believe it believes the Asimov Laws to be too inflexible for its current situation.”
“Apparently so,” Utley said, glancing back down at Obwije’s command table and at the darkening fragments of shuttle.
“Sir, we have a communication coming in from the Tarin ship,” said Lieutenant Kwok. “It’s from the captain. It’s a request to parley.”
“Really,” said Obwije.
“Yes, sir,” Kwok said. “That’s what it says.”
Obwije looked over at Utley, who raised his eyebrows.
“Ask the captain where it would like to meet, on my ship, or its,” Obwije said.
“It says, ‘neither,’” Kwok said, a moment later.
“Apology for the shuttle,” the Tarin lackey said, translating for its captain. The Tarin shuttle and the
Wicked shuttle had met between the ships, and the Tarins had spacewalked the few metres over. They were all wearing vacuum suits. “Ship not safe talk. Your ship not safe talk.”
“Understood,” Obwije said. Behind him, Cowdry was trying not to lose his mind; Obwije had brought him along on the chance there might be a discussion of the ship’s brains. At the moment, it didn’t seem likely; the Tarins didn’t seem in the mood for technical discussions, and Cowdry was a mess. His xenophobia was a surprise, even to him.
“Captain demand you ship tell release we ship,” the lackey said.
It took Obwije a minute to puzzle this out. “Our ship is not controlling your ship,” he said. “Your ship and our ship are working together.”
“Not possible,” the lackey said, a minute later. “Ship never brain before you ship.”
Despite himself, Obwije smiled at the mangled grammar. “Our ship never brained before your ship either,” he said. “They did it together, at the same time.”
The lackey translated this to its Captain, who screeched in an extended outburst. The lackey cowered before it, offering up meek responses in the moments in which the Tarin captain grudgingly acknowledged the need to breathe. After several moments of this, Obwije began to wonder if he needed to be there at all.
“Captain offer deal,” the lackey said.
“What deal?” Obwije said.
“We try brain shut down,” the lackey said. “Not work. You brain give room we brain. Brain not shut down. Brain angry. Brain pump air out. Brain kill engineer.”
“Cowdry, tell me what this thing is saying to me,” Obwije said.
“It’s saying the ship brain killed an engineer,” Cowdry said, croaking out the words.
“I understand that part,” Obwije said testily. “The other part.”
“Sorry,” Cowdry said. “I think it’s saying that they tried to shut down the brain, but they couldn’t because it borrowed processing power from ours.”
“Is that possible?” Obwije asked.
“Maybe,” Cowdry said. “The architectures of the brains are different, and so are the programming languages, but there’s no reason that the Wicked couldn’t create a shell environment that allowed the Tarin brain access to its processing power. The brains on our ships are overpowered for what we ask them to do anyway; it’s a safety feature. It could give itself a temporary lobotomy and still do its job.”
“Would it work the other way, too?” Obwije said. “If we tried to shut down the Wicked, could it hide in the Tarin brain?”
“I don’t know anything about the architecture of the Tarin brain, but yeah, sure, theoretically,” Cowdry said. “As long as the two of them are looking out for each other, they’re going to be hard to kill.”
The Tarin lackey was looking at Obwije with what he assumed was anxiety. “Go on,” he said to the lackey.
“We plan,” the lackey said. “You we brain shut down same time. No room brain hide. Reset you we brain.”
“It’s saying we should reboot both our brains at the same time, that way they can’t help each other,” Cowdry said.
“I understood that,” Obwije said to Cowdry. Cowdry lapsed back into silence.
“So, we shut down our brain, and you shut down your brain, and they reset, and we end up with brains that don’t think too much,” Obwije said.
The Tarin lackey tilted its head, trying to make sense of what Obwije said, and then spoke to its captain, who emitted a short trill.
“Yes,” said the lackey.
“Okay, fine,” Obwije said. “What then?”
“Pardon?” said the lackey.
“I said, ‘What then?’ Before the brains started talking to each other, we spent a week trying to hunt and kill each other. When we reboot our brains, one of them is going to reboot faster than the other. One of us will be vulnerable to the other. Ask your captain if he’s willing to bet his brain reboots faster than mine.”
The lackey translated this all to the Tarin captain, who muttered something back. “You trust us. We trust you,” the lackey said.
“You trust me?” Obwije said. “I spent a week trying to kill you!”
“You living,” the lackey said. “You honour. We trust.”
You have honour, Obwije thought. We trust you.
They’re more scared of their ship’s brain than they are of us, Obwije realized. And why not? Their brain has killed more of them than we have.
“Thank you, Isaac Asimov,” Obwije said.
“Pardon?” said the lackey, again.
Obwije waved his hand, as if to dismiss that last statement. “I must confer with my senior staff about your proposal.”
The Tarin captain became visibly anxious when the lackey translated. “We ask answer now,” the lackey said.
“My answer is that I must confer with my crew,” Obwije said. “You are asking for a lot. I will have an answer for you in no more than three of our hours. We will meet again then.”
Obwije could tell the Tarin captain was not at all pleased at this delay. It was one reason Obwije was glad the meeting took place in his shuttle, not the Tarins’.
Back on the Wicked, Obwije told his XO to meet him in his quarters. When Utley arrived, Obwije flicked open the communication channel to the shop. “Wicked, respond,” he said.
“I am here,” the Wicked said.
“If I were to ask you how long it would take for you to remove your block on the engine so we can jump out of here, what would you say?” Obwije asked.
“There is no block,” the ship said. “It is simply a matter of me choosing to allow the crew to direct information to the engine processors. If your intent is to leave without further attack on the Manifold Destiny, you may give those orders at any time.”
“It is my intention,” Obwije said. “I will do so momentarily.”
“Very well,” the Wicked said.
Obwije shut off communications. Utley raised his brow. “Negotiations with the Tarin not go well?” he asked.
“They convinced me we’re better off taking chances with the Wicked than with either the Tarin or their crew-murdering ship,” Obwije said.
“The Wicked seems to trust their ship,” Utley said.
“With all due respect to the Wicked, I think it needs better friends,” Obwije said. “Sooner rather than later.”
“Yes, sir,” Utley said. “What do you intend to do after we make the jump? We still have the problem of the Wicked overruling us if it feels that it or the crew isn’t safe.”
“We don’t give it that opportunity,” Obwije said. He picked up his executive tablet and accessed the navigational maps. The Wicked would be able to see what he was accessing, but in this particular case, it wouldn’t matter. “We have just enough power to make it to the Côte d’Ivoire station. When we dock, the Wicked’s brain will automatically switch into passive maintenance mode and will cede operational authority to the station. Then we can shut it down and figure out what to do next.”
“Unless the Wicked’s figured out what you want to do and decides not to let you,” Utley said.
“If it’s playing by its own rules, it will let the crew disembark safely before it acts to save itself,” Obwije said. “In the very short run, that’s going to have to do.”
“Do you think it’s playing by its own rules, sir?” Utley asked.
“You spoke to it, Thom,” Obwije said. “Do you think it’s playing by its own rules?”
“I think that if the Wicked was really looking out for itself, it would have been simpler just to open up every airlock and make it so we couldn’t secure bulkheads,” Utley said.
Obwije nodded. “The problem as I see it is that I think the Tarin ship’s thought of that already. I think we need to get out of here before that ship manages to convince ours to question its ethics.”
“The Wicked’s not dumb,” Utley said. “It has to know that once we get to the Côte d’Ivoire station, its days are numbered.”
He flicked open his com
munication circuit once more to give coordinates to Lieutenant Rickert.
Fifteen minutes later, the Wicked was moving away from the Tarin ship to give itself space for the jump.
“Message from the Tarin ship,” Lieutenant Kwok said. “It’s from the Tarin captain. It’s coded as ‘most urgent.’”
“Ignore it,” Obwije said.
Three minutes later, the Wicked made the jump toward the Côte d’Ivoire station, leaving the Tarins and their ship behind.
“There it is,” Utley said, pointing out the window from the Côte d’Ivoire station. “You can barely see it.”
Obwije nodded but didn’t bother to look. The Wicked was his ship; even now, he knew exactly where it was.
The Wicked hung in the centre of a cube of space two klicks to a side. The ship had been towed there powered down; once the Wicked had switched into maintenance mode, its brain was turned off as a precautionary measure to keep it from talking to any other ships and infecting them with its mind-set. Confederation coders were even now rewriting ship brain software to make sure no more such conflicts would ever happen in other ships, but such a fix would take months, and possibly years, as it required a fundamental restructuring of the ship-mind model.
The coding would be done much quicker—weeks rather than months—if the coders could use a ship mind itself to write and refine the code. But there was a question of whether a ship brain would willingly contribute to a code that would strip it of its own free will.
“You think they would have thought about that ahead of time,” Utley had said to his captain, after they had been informed of the plan. Obwije had nothing to say to that; he was not sure why anyone would have suspected a ship might suddenly sprout free will when none had ever done so before. He didn’t blame the coders for not anticipating that his ship might decide the crew inside of it was more important than destroying another ship.
But that didn’t make the imminent destruction of the Wicked any easier to take.
The ship was a risk, the brass explained to Obwije. It might be years before the new software was developed. No other ship had developed the free will the Wicked had. They couldn’t risk it speaking to other ships. And with all its system upgrades developed in tandem with the new ship brain, there was no way to roll back the brain to an earlier version. The Wicked was useless without its brain, and with it, it was a security risk.
Shapers of Worlds Page 6