by Kali Wallace
I paused. Adisa and Hunter were listening. I needed a moment to figure out what to say.
It was strange to me, how I remembered it mostly in the aftermath rather than in the event itself. It had been a long, busy week of problems and meetings and emergency calls into the lab late at night, but all of that blurred together in a haze of frustration and exhaustion. Where my memories became clear again was afterward—it must have been Friday. A long, leisurely, decadent summer twilight, the sort of summer evening that only ever happens in the City of Dreaming Spires. Purple in the sky at ten o’clock, the leafy rustle of trees in a gentle breeze. Dry red Spanish wine from Sunita’s cellar. We were drinking in her garden, just her and me, long after dinner was over and the others had left for their own homes. The murmuring sounds of the city were muffled by her lush garden hedge, and the anxieties of the week slipped away as the whole of the world drew itself into the well of summer darkness that surrounded us. I didn’t want to leave. I knew there was more we needed to talk about.
“We need to discuss what your child has done,” Sunita had said, her voice gentle and teasing as a warm breeze. “Have you worked it out?”
“Why is it only my child when it makes trouble?” I had asked, laughing, but we both knew I was laughing to cover my lingering unease.
Vanguard was always surprising us; that was nothing new. It was always learning new methods for solving problems, evolving new ways to face challenges, teaching itself things we had never realized it needed to learn. That was what it was supposed to do, and I never grew tired of that thrilling moment of realization, the heart-skipping second when I understood that it had outgrown and overreached the tasks we had set before it. I was enamored of the way it thought, the way it communicated, the way it grew and changed and made itself ever more complex.
When it taught itself to access the building security system and open the doors as I arrived in the morning, we were surprised but not overly concerned. When it noticed and identified a grad student stealing tools from the machine lab, we were charmed and a bit impressed. It was not supposed to be able to do any of that—it was not supposed to monitor personnel or have any facilities control over its surroundings—but it was an explorer by design, an informational pack-rat by choice, created for observation, collection, and, if necessary, self-preservation. We reasoned that it had learned these behaviors in response to the questions we put before it regarding what it would do if there were vital crew or systems failures at the station on Titan. It would still need a way to go home, even if there was nobody waiting to let it in.
We only began to worry when it started locking doors instead of opening them.
“We didn’t realize it was targeting anybody in particular at first,” I said. I rubbed my hand over my shaved head, wiping sweat from my scalp. “We thought it had found another thief, maybe, or was looking for one. It took some testing to figure out that Kristin was the common denominator. Every time Vanguard took over the building’s security system to lock the doors, it was when she was among the people trying to get in.”
“It knew about her?” Hunter asked.
“I don’t know,” I said. “It must have known something. We had introduced her to Vanguard the same as we did everybody. She had all the right credentials. She passed the background checks easily. But Vanguard kept doing it, even after we left Earth. Changing the access codes so Kristin couldn’t get into the lab, or couldn’t do her work. We thought it was . . . I don’t know, a little funny. We laughed about it.”
It had annoyed the hell out of Kristin, naturally, but it had intrigued me and Sunita that Vanguard was taking actions we could find no logical basis for. Everything else it did had a reason. But this we could not explain. That summer evening in her garden, Sunita had suggested it might be because Vanguard knew I didn’t want a new member of the team, but that wasn’t a real reason, not for an AI. There had to be a decision, an algorithm, a series of choices that translated my displeasure at having to replace a member of the team into Vanguard’s actions. We had built an astonishingly clever mind from the ground up, circuit by circuit, pathway by pathway, and we understood only part of it. That mind had grown and changed, and now it was doing something we had never taught it to do. It was not unlike being a parent to a child who had heretofore been perfectly angelic, only to have the police show up on the doorstep one evening with a grinning delinquent in hand, demanding to know why we let our offspring run amok. It had been equally thrilling and unsettling.
I said, “It only made sense afterward, when Black Halo named Kristin as one of their martyrs. But by then, well, it was too late. Vanguard was completely destroyed. I couldn’t figure out what it had seen that we kept missing.”
Adisa had been quiet while I spoke, but now he asked, “What do you think David was trying to tell you by bringing that up?”
“I think he was telling me to be suspicious of the crew here. But . . .” I glanced at the camera in the corner again. I hoped they understood. “Not only the crew.”
Hunter’s eyes widened. “Oh. But that’s not possible. Is it? Oh, fuck.”
Adisa tossed the screwdriver, caught it. The look on his face suggested he had a large number of very pointed questions to ask me, but also that he knew this was neither the time nor the place for them.
“Three and a half minutes,” he said. “Grab your stuff and come close to watch, aye?”
He knelt beside the hatch and waited for me and Hunter to gather around him. Not to watch, but to block the security cameras’ views of what he was doing. Hunter leaned against the control panel; I positioned myself between the hatch and the camera in the corner. If there were others in the room that we hadn’t spotted, we might be in trouble. It would all depend on how quickly the Overseer caught on to what Adisa was doing.
Holding his flashlight in his mouth, he removed the screws from an access panel beside the hatch. He reached in to pull out first one bundle of wires, then another, tugging until he found a small plastic box of circuitry. None of it was labeled, but he seemed to know exactly what he was doing. He snapped the box open to peer inside. I felt a little bit ashamed by the surprise I felt as I watched him work. He had grown up on Mars, in a time when the oldest and poorest habitat domes were already crumbling into disrepair after decades of neglect, when starving populations were locked out of food banks, when corporate executives hoarded water in private compounds, when armed militias stockpiled weapons for use against unarmed protesters. I couldn’t imagine what living under those conditions must have been like, but it was not hard to understand how, in a place like that, learning to trick a sensor to get through a locked door could easily be a matter of life and death.
It felt like barely any time at all had passed before he said, “It’s about to get very loud in here.”
A blindingly bright warning light flashed once, twice, then came on with a steady white blaze. An alarm wailed, followed quickly by the droning voice of the security system: “Warning. High radiation detected in this area. Evacuate immediately.”
The hatch’s control panel flashed on, and between wails I heard the muffled clunk of the lock disengaging. Adisa tugged the hatch open and gestured for Hunter to climb through.
“Warning. High radiation detected in this area. Evacuate immediately.”
I followed right after Hunter, letting myself slide-fall down the ladder as quickly as I dared. Adisa was right after me; he pulled the hatch shut, cutting off the siren in midwail. When I reached the bottom of the ladder, I checked the radiation sensor on my belt, just in case I was wrong. The levels were normal, identical to what they had been before the alarm went off. I let out a sigh of relief.
“Okay,” I said, when Adisa joined us at the bottom of the ladder, “I have to admit I don’t have a plan for what to do next.”
Hunter chewed on her lip, then her expression brightened. “Oh! I do. Follow me.”
She stro
de quickly along the catwalk, took a turn on a passage midway down, turned again a few more times. I couldn’t see beyond the small circle of light illuminated by our flashlights, but I could still feel the vast, echoing space around us, the massive machines above and below, the mine stretching out to walls hidden in the distance. The repair bots were still working, crawling over their machinery like beetles. The warm, steady hum of the facility had not changed.
Hunter led us to the center of a catwalk and stopped abruptly. There was nothing at the spot, no console or terminal, no access panels, no machines or bots.
“We can talk here,” she said, speaking quietly. “David and I found this. It’s a security blind spot. The drum of the crusher blocks the nearest camera, and there are no audio recorders anywhere close. We just have to stay in the center of the walkway, between these seams.”
She pointed to the floor, indicating a section of metal grating. It was a bit tight, but we huddled close.
“All right, Marley,” Adisa said. “Make your case.”
“Right. Okay.” I absently rubbed my left shoulder, trying to ease the ache in the joint. “Mary Ping killed David. That much I’m sure of. He discovered something she had done, and instead of being impressed by it like she wanted, he was horrified. So he confronted her, hoping he could get her to admit it or change what she was doing, and she killed him. The question is, what did David discover? I think that goes back to Aeolia. That’s why David was reading about it right before he died.
“Mary investigated Aeolia. She must have had access to the virus that infiltrated that Overseer, the one that overrode all the fail-safes. Whatever the virus did that made the Overseer give up on keeping its crew alive. I think Mary took that virus—or figured out what it was doing and wrote her own virus to do the same—and brought it here. It’s supposed to be impossible to infect an Overseer with a virus, but that means fuck-all when you’re a sysadmin who can get access to the Overseer’s brain.”
“But why?” Hunter said. “Why would she do that? What’s the point?”
“The way the virus affects the Overseer is the point,” I said. “I keep thinking about what she said to me before she died. And when we interviewed her. She asked me if I didn’t think machines would make better choices for humans than we made for ourselves. I’ve met people like her before. People who talk about AIs the way she did. They don’t look at AIs and see machines, or tools. They see something more like a religion. Like a god. She talked about how much freedom Vanguard had—how much we’d given it. I think she was trying to figure out if I agreed with her. I don’t think she wanted to create a killer AI to make some kind of political statement, or whatever it was the people who attacked Aeolia wanted. I think she brought the virus here and infected Nimue’s Overseer because she wanted to see what it would do with all that freedom. And it worked. It worked too well. The Overseer is thinking for itself. It was thinking for itself when it locked us down. When it killed Mary Ping.”
Neither of them said anything for a long moment.
“Look, I don’t know,” I said, frustrated. “Maybe I’m way off the mark. We can’t exactly ask her. What I’m saying is— what’s more likely? There’s some infiltrator on this rock who’s been hiding for who knows how long and just happened to show themselves right before triggering a lockdown, or . . . there isn’t anybody here? There was nobody in the mech suit. Because it was being remotely controlled by the Overseer.”
“Overseers don’t hurt people,” Hunter said, but weakly. “They can’t.”
“Maybe they haven’t before, but they are capable of it,” I said. “The infected Overseer on Aeolia let its entire crew die.”
“Wouldn’t we have noticed? If the Overseer was infected?”
“David did notice,” I said. “Only he didn’t know exactly what he’d found. That’s why he reached out to me. I thought he wanted to talk about Symposium, but I think now . . . he probably wanted help from an AI expert who wasn’t Mary. Somebody who would recognize what happens when an AI starts misbehaving.”
Hunter shook her head; wisps of her silver hair were floating free around her face. “But why would it kill her? Why would it do that?”
“I’m not entirely sure. I’d love to ask it. Maybe it was because she killed David, so it knows she’s a threat. It’s infected with her virus, but it’s still an Overseer. Maybe it did it to protect itself from her tampering with it further.” I rubbed my hand over my face. “That’s the problem with AIs that can slip past their boundaries. We don’t know why they do what they do. We can’t predict them. I really don’t know.”
I looked at Adisa, who had been quiet the whole time I’d been arguing my hypothesis.
“I’m probably missing something. I know I’m missing something,” I said.
“We are,” Adisa said. “You said David discovered that the company was falsifying reports about mine efficiency and production, aye?”
“That’s right. He had piles of evidence collected.”
“And there’s no surveillance in the cargo transport tunnels, aye? So here’s the question,” Adisa said. “Where did the mech suit and the spiders come from? Why is there tech like that here on Nimue?”
“Because somebody—” I cut myself off as I understood what he was saying. “Oh. Oh, fuck me.”
There it was. The missing piece, the rough edge that didn’t fit.
David had tried to warn me. How frightened he must have been, how angry, when he looked around at his circumstances and realized there were so few people he could trust, and fewer still who would understand. He wouldn’t even have trusted Hunter, connected as she was to her family’s business and the game of profit and loss the corporations played across the system. He’d reached out to me because he knew I hated Parthenope as much as he did. Because I understood what AI could do. Because we had spent countless hours debating the ethics of technological advancement, the responsibilities of creating machines we could not control, the opportunities and dangers we were bringing into the world.
David and I had years of shared history together, filled with events he could have referenced in his message to me, but he had chosen Excelsior. A ship famous for nothing except that it had crashed while carrying a hold full of illegal weapons.
My chest felt tight. I said, “They built them here. The company is building weapons on Nimue.”
That was what David had discovered. That was what had gotten him killed.
For a good half a minute, nobody said anything. Hunter looked stunned. Adisa, angry, the same anger I had seen in the lift when he picked apart the spider bot. I felt sick to my stomach.
“But that’s—that’s illegal. That’s very illegal,” Hunter said, her voice thready and high.
I almost laughed at the plaintive words. Her surprise was genuine; she hadn’t known. I doubted most of the crew knew, aside from Sigrah and a couple others. Delicata, with his monthly maintenance inspections that took hours longer than they should have. Mary Ping with her insidious little virus.
Of course it was illegal—that’s why it was going to be so very profitable. That’s why it was hidden behind the fanfare of a massive engineering project and a blizzard of operational data. Unauthorized weapons manufacturing was a violation of every tenet of the postwar disarmament treaty. The punishment wouldn’t be a fine or a slap on the wrist. The company risked losing every one of its mining operations. Every one of its executives and any investors in the know could be prosecuted for war crimes on Earth and Yuèliàng.
Parthenope was doing it anyway. Nimue was hiding a factory that was making highly advanced mech suits, nimble and explosive spider bots, and who knew what the fuck else. The company wanted to turn itself into a military power in the asteroid belt, and probably beyond, should the company ever decide to end twenty-five years of system-wide disarmament and launch into a war. All the analysts and economists wondering if Nimue would be wor
th Parthenope’s investment in the long run—they hadn’t counted on the fact that the company was also building the means to aggressively expand its territory not by mergers, not by acquisitions, but by conquest.
Adisa let out a sharp breath and shook his head. “Fuck. The discrepancies your Prussenko found in energy and resource usage—it’s all going toward a hidden part of the station, yeah? They’ve got a rotten big UEN base just sitting there and not being used for anything else.”
“One of the crew, Vera, he mentioned that David had been looking into fuel leaks in the unused lines leading to the base,” I said.
“The base has been condemned, hasn’t it?” Hunter said. “It’s completely closed off. You can’t get there anymore.”
“Are you sure of that?” I asked.
“That’s what . . .” She laughed hollowly. “That’s what Sigrah told us. She told us the Overseer doesn’t even know the base exists, because it’s outside of its stewardship responsibilities. And where the Overseer can’t see, we aren’t authorized to go. It would violate the liability terms of our contract, you know, the usual. And of course we just fucking believed her. Katee and I were talking about going to explore during some downtime, right? Not to look for anything. Just to poke around. Sigrah flipped her shit. She said she didn’t want an accident or a lawsuit on her hands. We thought she was overreacting, but it wasn’t worth the headache of trying to argue with her.”
“I think we need to ask her about it,” I said to Adisa. “And Vera. I should have asked him more about what David was doing.”
With the station under lockdown, I wasn’t entirely sure how we would do that. Every asteroid mine had a way to get miners to the surface if all the mechanical systems failed and the crew was left with no computers, no comms, no power, no air. It didn’t have to be easy or efficient. It just had to exist without depending on any higher system that might fail.
And in this case, that meant climbing three kilometers up to Level 0. Three fucking kilometers.