by Max Barry
This he had to see. He closed the board and headed to E Deck. Possibly the damage had caused the ship to stop recognizing it as a core bank, and therefore to stop knowing it shouldn’t attempt repairs. But even so, it should have been unsuccessful, because core bank repair was beyond the ship’s capability. It was concerning either way, because an AI rewriting its own core was a little like a human neurosurgeon opening up his own skull. Any errors could compound, affecting the ship’s ability to recognize that they were errors.
He walked right past the door to Eng-13, lost in thought. When he backtracked, he couldn’t find it. He stopped and checked his location on ping. He was where he’d thought. He took eight steps and stopped and touched the wall. On his film, Eng-13 was mapped right in front of him.
He peeled off his film. He looked both ways. He knew this corridor. He had been here during the engagement. There had been a door. Now it was gone.
* * *
—
“You’re telling me it moved the room?” Jackson said. They had gathered in Rec-2 at Gilly’s request. From Jackson’s demeanor, she had been asleep.
“I think the room is still there,” he said. “We just can’t get to it.”
“How do you know?” Beanfield said.
“Because it’s on ping.”
“But that data comes from the ship. It could be lying.”
“Pardon me?” he said.
“Or outdated.”
“The ship doesn’t lie.” There was so much insanity built into the assumptions of that comment, he didn’t know where to start. “It has only the most basic concept of our existence. It can’t imagine what we’re thinking, let alone know how to fool us.”
“You’re sure you went to the right deck,” Anders said.
“Yes.”
“I’m not saying you’re crazy. But you did spend months trying to fix imaginary valve leaks.”
Jackson was silent a moment. “Check it.”
“It’s not there,” Gilly said as Anders left. “I even took off my film.”
“Why?” Beanfield said.
“Because I thought it might be a puzzle.”
“Oh,” she said. “No, this one’s not us.”
“Are the other Engineering areas accessible?” Jackson asked. “Eng-12, Eng-14?”
“Yes.”
“So Eng-13 is still there, missing a door.”
“Right.”
“Or the ship deconstructed the room and is lying about it.”
“The ship doesn’t lie,” Gilly said. “It has no reason to.”
“It also doesn’t hide doors, was my understanding until five minutes ago.”
“Okay,” he conceded.
“So the question is why.”
“We can’t know that.”
“Explain that deeply unsatisfying answer,” Jackson said.
“Because it doesn’t think like we do. When we try to put ourselves in its shoes, we can’t help anthropomorphizing and imagining human motivations. Which is wrong.”
“Wrong?”
“Inaccurate. Misleading.”
“Well,” Jackson said, “I want to know where my door went.”
“Isn’t it obvious?” Beanfield said.
They looked at her. Gilly said, “Not to me.”
“I get what you’re saying about anthropomorphizing,” Beanfield said. “But you attacked it. Isn’t it natural for it to protect itself?”
“What?”
“You drilled into its brain. When you were taking the core offline.”
He stared at her.
“I mean,” she said, “isn’t this what the ship does? Learn from experience, so it’s better at defending itself the next time?”
He felt genuinely at a loss for words.
“Interesting,” Jackson said.
“I was forcing recognition of a faulty core bank. I was helping.”
“Sure,” Beanfield said, “but you can imagine the ship might not see it that way, right?”
He could not imagine it. Or maybe he didn’t want to. The idea felt grossly offensive in some way he couldn’t specify. He had been fixing a machine part.
“I’m not saying it’s angry or anything,” Beanfield said. “Just that, logically, it might want to keep you out of there so it doesn’t get drilled again.”
He looked at Jackson, who said nothing.
“Hey,” Anders said on comms. “Guess what? Gilly’s right. There’s no door.”
“Thank you,” Gilly said.
There was a rap rap rap; Anders knocking on the wall. “Weird,” Anders said.
* * *
—
The ship cut engines and coasted for two more days. They had never done this before and everyone projected their own assumptions onto it. “We’re retreating,” Beanfield said. “We’re getting ready to skip out of VZ.” She was hanging around while Gilly tried to burn through the wall with a plasma cutter to see whether Eng-13 was still there. So far, he had produced heat and light but no answers.
He sat back on his heels and studied the wall. “I don’t know about that.”
“No? We encountered a new weapon, took major damage, and developed intel about salamander information transmission. You don’t think it’ll want to relay that back to Service?”
Gilly didn’t say anything.
“Sure, okay,” Beanfield said. “We can’t guess what an AI is thinking. But even so.”
He poked at the wall with a gloved finger.
“Anything?”
“This is dense. Similar to the exterior hull. I can’t pierce it with these tools.”
“Is there something else you could use?”
“In the small-arms lockers, maybe.”
“Hmm,” Beanfield said. “Let’s not go shooting the ship, is my thinking.”
“Agreed,” he said.
“I mean, it’s not like we absolutely have to get in there.”
He nodded. It was true. Frustrating, but true. He began to pack away his tools. “Hey, have you seen my drill?”
“I don’t keep track of your tools, Gilly. What, is it your only one?”
“No. But I like to know where everything is.”
“I have noticed that about you.” She unpeeled herself from the wall. “Still think it’s unaware of our existence?”
“Hmm?”
“The ship,” she said. “You said it doesn’t know we’re here. But then we hurt it. So maybe we got its attention. Maybe we made it mad.”
He stopped. “You’re talking about software. Yes, very intelligent software, but it doesn’t think like us. It doesn’t, you know, feel things.”
“Not like we do, of course. But in its own way.”
“Look,” he said, “I can’t tell you what’s going on in the mind of a computer. But I’m really confident that it’s not what you’re thinking.”
She considered. “You don’t think the ship can feel hurt?”
“Uh,” he said.
“Or scared?”
He chose his words carefully. “I think it would be a mistake to consider the ship as anything other than a collection of logical processes.”
“Oh, Gilly,” Beanfield said. “You say that about everything.”
“No, I don’t,” he said. “I don’t view everything as a collection of logical processes.”
She smiled and shook her head. “Forget it. I’m starving. Let’s head back.”
“There is logic to everything,” he said. “You look deep enough, there’s always a rational process. You just need to figure out what it is.”
“There it is,” Beanfield said.
“I don’t know how you can believe otherwise. I mean, don’t you want to know there’s an explanation for everything?”
“I
can’t think of anything worse,” Beanfield said. She looked back down the corridor toward where Eng-13 had been. “I hope you find your answers, Gilly.”
There was something in her face that disturbed him, so he stopped. “You can trust the ship. No matter what’s going on, it will always want to protect us, because we’re its DNA. That can’t change.”
She smiled brightly, the way she did in clips. “Thank you for saying so, Gilly.”
“I promise you,” he said.
* * *
—
The next day Beanfield called him over to say she could feel the ship burning. “What?” he said.
“Little tremors.” They were outside Rec-4, which was configured as a gym. Her hair was plastered to her forehead, her white shirt sweat-soaked down the middle, a towel slung over one shoulder. “Are we accelerating?”
“I don’t think so.” He checked his film. “No.”
“You can’t feel anything?”
He knew she wanted him to lift his film, so he did, and listened. It was sometimes possible to feel whether the ship was burning, especially on aft decks.
“Maybe it’s not accelerating yet,” she said. “It feels like it’s about to.”
He wasn’t sure how anyone would sense that. “I guess I could pull an exterior view and look for any unusual activity.”
“I already did that.”
“Oh,” he said.
“It doesn’t matter.” She threw the towel over her face and began to rub with vigor. He remembered how, during the last engagement, Beanfield’s station had been breached. They had never really talked about that, but he wondered how she was dealing with it. It occurred to him that when he was worked up about something, he could talk to Beanfield, but Beanfield had no one.
“I’ll keep an eye on it,” he said.
And, funnily enough, when he woke the next day, he could feel something. He ran his film and confirmed it. They were accelerating. But they weren’t leaving VZ. They were going deeper in.
* * *
—
Three more days passed. They grew tense and excited. Anders didn’t sleep, as far as Gilly could tell. He developed bloodshot eyes and a braying, high-pitched laugh. “Jesus, Anders,” said Beanfield. “Your breath.” He wanted to play ninja stars but hid in such obvious positions that Gilly felt bad throwing shurikens at him. On the fifth day, Gilly discovered him trying to herd crabs into a jetpod. He came around a corner and there was Anders, waving his arms, corralling crabs. Anders saw him and stopped and lowered his arms and the crabs began to disperse. Gilly kept walking because he didn’t want to know what that was.
The ship opened fire. There was no klaxon. But they could feel the coil and release of power. They scrambled for station, thinking it was another emergency, another systems failure; now even the klaxon didn’t work. Gilly was near Weapons at the time and Anders came storming up the corridor with his eyes full of crazy energy and shoved him aside. Gilly reached station and strapped in and Jackson’s voice came through saying there was no enemy. The ship was launching drones and destroying them. It was practicing.
“Goddamn it,” said Anders.
“Just a drill,” Gilly said. “It’s testing countermeasures to the bomb, like I said.”
“Piece of crap.”
“Cool it,” Jackson said. There was silence. The ship ran through the same process over and over, launch and destroy, using different weapons.
“I want to do something,” Anders said.
Of course, they were hoping the ship would develop a hard counter to the hive bomb so that everything could go back to how it had been, when they had destroyed salamanders with ease and minimal human input. But it was hard to deny that when things had gone wrong, it had been the most exciting part of the last two years. It was the kind of thing Gilly had had in mind when he got interested in making crew. That must go double for the soldiers, he thought, and double again for a guy like Anders. He could understand someone nursing a small wish for the ship to need them again.
“I think we’re done here,” Jackson said, after the thirtieth drone. “Nothing’s happening.”
* * *
—
He tried to avoid the issue of seizing manual control of the ship, but Jackson pinged him regularly for updates and there were only so many times he could stall. In the mess, where you weren’t supposed to talk shop, or at least not too much of it, she sipped at a steel cup and asked whether he intended to have a report for her before the war was over, or afterward. He glanced at Beanfield, since the moratorium on shop talk was her policy, but she gave him nothing.
“It’s a complicated area,” he said.
“I didn’t ask for an assessment on how complicated it is,” Jackson said. “I asked you to tell me what we can do.”
“Look, the answer is nothing. I know that’s not what you want to hear, but it’s the truth.”
There was a momentary silence. “Gilly,” said Beanfield, “there are times when you make it incredibly clear you didn’t come through the military.”
“What about a kill switch?” Jackson said.
He looked at her, surprised. “What?”
“Isn’t there a mechanism to disable the AI in an emergency?”
He hesitated. The answer was yes. Furthermore, Jackson definitely knew that. It abruptly occurred to him that this was the endgame she’d had in mind for some time. She had never liked the idea of computers controlling everything and had seen her chance to change it. “There’s something like that.”
“Now we’re talking,” said Anders.
“But,” he said, adjusting his position on what felt like an abruptly hard seat, “it should never, ever be used.”
“Then why does it exist?” Jackson said.
“Everything would stop working. And I mean everything.” Jackson didn’t respond. “We’re in VZ,” he added. “There’s no backup if something goes wrong.”
“I appreciate your expertise with the ship,” Jackson said. “And I understand you have a career interest in whether it succeeds. But here’s what I see. We’re a long way from home, the last time we engaged we got our butts handed to us, and the AI has started hiding rooms.”
“One room.”
Jackson smacked the table with her palm. The plates and cups jumped. There was a short silence.
“If we activate the kill switch,” Jackson said, “we wouldn’t have to rely on the AI to use systems like Weapons. They’d be available for manual operation. Correct?”
“Yes, but—”
“I know these systems weren’t designed to be used that way. I know we wouldn’t be as efficient as the AI.”
“That’s putting it mildly. Extremely, extremely mildly.”
“So we understand each other,” Jackson said. “This is a card we shouldn’t play except as a last resort. Nevertheless, it’s one I want to hold. So make it ready.”
He took a breath. “Please promise me you won’t use this unless we absolutely have to.”
“I’ll do you one better. I won’t use it unless you say so.”
He wasn’t sure whether to believe that. It felt like something she might say now and retract later. But it did make him feel a little better.
“Yes?” Jackson said.
“All right.”
She picked up her steel cup and took a sip. “Well then,” she said. “Sounds like progress.”
* * *
—
He’d almost given up when it happened. He was in the can. The klaxon sounded. The walls glowed orange. He bolted upright and took a step before realizing his pants were around his ankles. Before he reached the ladder, Anders was reporting in over comms. “Weapons checking in.” He must have been camped outside.
Jackson: “Roger that. On my way to command, give me thirty seconds.”
“Is
it another drill?”
“You see those orange lights? That’s how you know it’s not a drill.”
Gilly popped the hatch to D Deck. “Almost at station.”
“Life, how far?”
“I’m here.”
“Check in, please. Let’s do this by the book.”
“Life, checking in. All systems normal. We have good O2, good pressure, good thermals.”
“Thank you, Life.”
Gilly strapped into his harness and brought up his board. “Intel, checking in. All systems online and functional.”
“Copy that, Intel.”
Anders: “What are we up against?”
“Give me a moment. I’m reaching station. Command . . . checking in. I’m seeing green-to-green. Sensors up. Armor up. Weapons up.”
“Hostiles?”
“Stand by.”
“How many?”
“Weapons, I will brief you when I have information you need, you understand? We have a six-pack of hives and they’re expelling. Nine hundred total so far. Contact in forty seconds.”
Gilly skimmed his fingers across his board. Not touching anything, just defining his area. He had the cores on permanent overlay so he’d know immediately if any of them flickered. Somewhere in that solid green rectangle was core bank 996, which he still didn’t trust.
Beanfield: “These are regular hives?”
“Looks that way to me. Verify, Intel.”
“Regular hives. No bombs.”
“Batteries one through twenty-eight relocating to fore,” Anders said. “That’s a lot of fucking lasers. There’s no way to put a few of those on manual?”
“Negative,” Gilly said.
“If we lose cores again—”
“Everything’s green, Anders.”
“Let me know the second anything gets even a little funky,” Jackson said.
“Roger that.”
“You have the kill switch prepared?”
“It’s ready.”
“Because if we need it, I don’t want to suddenly discover there’s an ‘Are you sure?’ or a sixty-second timer or any of that bullshit.”