by Satoshi Hase
Arato had thought his mind was already made up, but Ryo’s words still shook him. Ryo’s desires weren’t selfish; he was thinking about Arato as well.
“All those machines you’re putting your faith in needed was a living, breathing human,” Ryo continued. “Some easily-manipulated idiot who would push the button for them without asking too many questions.”
But the man in Arato wouldn’t back down, even in the face of logic. “Okay, so what about you, Ryo?” Arato asked. “You’re so much smarter than me, so do you really not see that the folks at MemeFrame and the Antibody Network are no different? Everyone’s ready to turn their back on other humans whenever it suits their needs.”
Ryo grabbed Arato’s throat in his arms with a headlock. Arato’s entire body was burning with the anger inside of him. Even Ryo had turned his back on people, refusing to destroy Snowdrop just to get Lacia out there so Methode could fight her. Because of that decision, countless soldiers and civilians had died. Both of them kept making mistake after massive mistake.
Arato was no longer sure the day would come when the two of them would be right. In a world where everything was supposed to have a generally accepted form and meaning, it seemed inherently, impossibly wrong for anyone to think they had the right to make such massive decisions. Perhaps that was just life; a senseless chaos that twisted the decisions any human tried to make.
Bending his neck just in time to avoid a punch to the face, Arato heard Ryo’s fist slam into the concrete near his head, hard. Ryo’s body lifted up a little, just enough for Arato to twist his hips and escape.
Standing, he panted for breath. It was hard to speak. He wondered if the feeling of needing to do something despite the action not being necessary was just part of being a living creature? If he and Ryo were capable of making more rational choices as Lacia did, perhaps they wouldn’t be there now, brawling right next to where two hIEs were fighting.
“I may be easy to manipulate, but that doesn’t mean there are no good reasons to push the button you keep talking about,” Arato said. “Like, what if I wanted to make the world a little more gentle? Or if a really important decision rode on it?”
“Those machines think the whole human race is full of morons because of idiots like you,” Ryo spat, pushing himself to his feet.
Arato started to rush Ryo again, but then stopped as if his feet were frozen to the ground. Ryo had taken out a gun, and was pointing it right at him.
Ryo rubbed a finger across his eyes. “Your answer is fine. It’s simple, and that’s fine,” he said. “You’re gonna reach for it, struggle for it, and if you can’t make it, you’ll leave it to someone else to finish. That’s how it’s always been, since ancient times. When we can’t reach our goals, we leave them to the next generation. But, listen, Arato, the one you’re trying to get to finish your goal for you is an ultra high-powered AI that is much smarter than any human ever was. If we leave it up to them, then we lose our ability to find an answer on our own, as humans. This will be the last choice we ever make. If you’re really going to hand our future over to them and I’m the only one here to stop you, then I’ll do it, even if I have to gun you down.”
An exhausted feeling settled like a massive weight on Arato’s shoulders. He and Ryo were humans, not machines made to answer questions. Deciding the shape of the future was a task far too heavy for simple living creatures like them.
“Hey, if you think something out there is testing all of humanity or something, what do you think of what you and I are doing right now?” Arato asked. “Have you ever thought this might be part of the plan? Two friends here, beating the shit out of each other like this?”
“Maybe I was manipulated into being in that explosion,” Ryo mused. “Maybe everything started from the moment we met.”
Arato and Ryo had been caught up in the same fire when they were kids, both receiving heavy burns and requiring hospitalization. In the hospital, they had met and become friends.
In that moment, Arato couldn’t help but be impressed by how long they had managed to stay together as friends, despite their differences. When they stood face-to-face like this—Arato, who believed in everyone, and Ryo, the untrusting manipulator—it was surprising just how opposite the two of them were.
Arato couldn’t move. Ryo was staring at him, eyes unmoving. Arato realized his friend might actually pull the trigger. They were both feeling talkative, since they could sense that they were coming up on a point of no return.
Another explosion sounded from outside the station building, and a shockwave sent the ground under their feet trembling violently. Arato watched the muzzle of Ryo’s gun, three meters away, sway wildly with the movement of the ground. He couldn’t tear his gaze from it, as hope bloomed that a shot fired in that moment might miss him.
But there were more things he wanted to say to Ryo and, with an effort, he returned his gaze back to his friend’s face. “I think humans need to solve human problems. You’re right about that,” he said. He couldn’t even imagine how much the city center of Kichijoji was being torn apart by the battle between Lacia and Methode.
“Oh, now you’re suddenly on board?” Ryo asked mockingly. “Are you gonna just wash your hands of the fact that everything Lacia’s doing is built off of orders you gave her?”
“That’s not what I mean,” Arato said. “Look at us, right now. I’m talking to you, but you’ve got a tool meant for killing people pointed at me. So, it’s more like I’m talking to the gun than you.”
Ryo’s lips twisted in a grimace, and he shifted his grip on the gun, possibly conscious of the sweat on his palms. “So because I’m using the threat of a gun to control you, I’m suddenly analog hacking you?” Ryo asked incredulously. “Bullshit. Don’t try to put a little thing like this on the same level as your monstrous machine out there, bending the whole world around her.”
“Lacia isn’t a tool I can control,” Arato said. “That’s why I tried to walk away from her back then. But I couldn’t stay away!”
Lacia and the other ultra high-powered AIs weren’t the first tools created by humans that grew beyond their creators’ ability to control. They were in company with inventions like nuclear weapons and nuclear power in the 20th century, and the huge structures built during the space age of the 21st century.
“Because tools still need owners, they need to have someone responsible for them,” Arato continued. “I’m Lacia’s owner. If we, as owners, walk away from our tools, then they’ll really be beyond our ability to control. That’s why I think it’s wrong to turn our backs on any tools, even the ones we don’t feel we can use.”
Lacia had told him that she wanted him to fight by her side. Arato himself had come to understand the dilemmas facing her a little better, as well. As an ultra high-performance AI, even if she was completely shut-down, it would be impossible for human beings, with their limited understanding, to grasp whether she had truly been eliminated or not. There were many inventions like that; things like nuclear plants, which were relatively simple to build but took much more skill and effort to decommission and disassemble. With Lacia borrowing processing power from the cloud, it would take humans decades of technological advancement to even have the technical capability to scour the network and ensure none of Lacia’s processes were still hidden away somewhere, working quietly.
“And because of that, you’re okay with handing the entire economy over to Lacia so she can control all of us with it?” Ryo demanded.
“If we take it away from her, who do you think humanity could trust it with?” Arato fired back.
Ryo flinched as if he had been struck by a blow, and the muzzle of the gun wavered in his hands. Constantly surrounded by the interests of MemeFrame, and with his own internal distrust of other humans, Ryo knew better than anyone how difficult it was to believe that humans would be capable of sharing with others. Even Arato, with his trusting nature, had lost faith when he saw how the Antibody Network had turned their backs on Kengo.
> “Don’t you think there’s a possibility that the future we make could be exactly the one everyone’s hoping for? That Lacia and I could actually make everyone’s dreams come true?” Arato asked.
Ryo’s face, drained of vitality, twisted in sorrow. “That’s a voice command for your machine!” he yelled, voice full of accusation. Then he pulled the trigger, but the bullet missed.
Outside, it was almost as if the place was being shelled by artillery. The earth was roaring like the whole world was being reborn in fire, and the ground shook too violently for Ryo to get a clear shot.
Whatever friendship still remained between them, Ryo seemed to feel it was his life’s mission to end Arato there. He kept firing, but his bullets all flew wide of their mark. Lights fell from the ceiling and walls and shattered, one after another. In the flashes of light and darkness Arato saw Yuka’s face, Shiori’s face. He saw the faces of Kengo and his classmates. He saw his father’s face. He saw Erika’s face. All the faces from his memories appeared before him in that moment. He saw visions of them, and thought about what they meant to him. In his heart, he prayed that the world would never become a place where they would be abandoned, or left behind.
“Lacia and the other units can help us find the answer to what humans are meant to do in the future,” Arato said. “I think it’s fine if progress outpaces our own hopes and dreams, as long as it can give us an automated hand finding the good things in this world.”
Arato didn’t think it was such a poor use of Lacia, all things considered.
〈Understood,〉 Lacia’s voice suddenly rang out over the station speakers. 〈I will commence guiding the future in the direction you have outlined.〉
If that day really was to be the end of the world, it hardly felt like a satisfying conclusion.
The station building stopped shaking.
“Do you really think this order is going to be the button that ends humanity?” Arato asked.
Ryo’s face was pale. The look on his face made Arato think his friend was seeing a completely different scene from him. Swiping a hand through his bangs, Ryo screamed. “Methode! I don’t care if I get caught up in the attack! Destroy Lacia, now!” Ryo’s order echoed off the walls of the abandoned station.
An instant later, flames engulfed the entire building.
“Ryo!” Arato yelled, searching through the flames for his friend. He wasn’t worried for himself; he trusted Lacia to protect him. But, Methode might have just thrown Ryo away, and images of Ginga Watarai’s corpse flashed through Arato’s mind.
Covering his mouth with his shirt, Ryo fled the sea of flame toward the station building’s exit on the opposite side. A fierce wind split the fire in two.
Methode ran through the flames, faster than Arato could follow with his eyes. But her trajectory, which should have taken her straight to Arato to snatch him up as a hostage, veered away from him by three meters.
Lacia was there, walking calmly through the flames in her bodysuit. Arato had no idea how or why, but the flames seemed to part to let her through. More importantly, Methode didn’t seem to have noticed.
“I remodeled some of Snowdrop’s child units into transmitters of disruptive signals and rendered them invisible,” Lacia explained. “It seems you rely very heavily on your optical sensors during high-speed maneuvers. I would suggest you proceed with caution now, though, as your senses are currently junk-tier.”
Methode, completely unfazed by the sea of flame around her, twisted her lips into a smile. “So you’ve evolved to the point where you can analyze other Lacia-types, huh?” she asked. Explosive torrents of energy burned wildly at the ground, all of them in places Lacia wasn’t.
“Those flowers are just machines, so obviously you would be able to control them through the cloud,” she continued. “Then you just used optical manipulation tech to mess with my device function.” The two hair accessories on Methode’s head began to glow brighter than the flames burning around her.
“You took it a step further, too, and analyzed my eyes so you could analog hack my perception,” she added. Standing in the burning station, Methode glared at Lacia. “But do you really think I can’t fight like this?” she asked. Orange light began to glow from her eyes, and Arato was suddenly reminded that she was the strongest of the Lacia-types.
“Snowdrop’s also trying to use quantum communication elements to modify herself and overcome her own shortcomings,” Lacia said. “But, let me give you some advice.” Light was starting to shine from Lacia’s eyes, as well. Her black coffin, using its ring-shaped float units to fly, came soaring through the air to her hand.
“In a situation like this, I would definitely avoid using quantum communication elements to open a path with Higgins,” Lacia finished.
Before she finished saying it Methode, eyes still glowing, bent double as if in extreme pain. “Higgins!” she gasped.
“You’ve left Snowdrop alone, and she’s currently trying to gain access to Higgins’ processing power,” Lacia explained. “In a situation like this Higgins, which sits immobile, would obviously be tempted to take control of your body for self-defense purposes.”
Methode had no heart, but the way her body seemed to writhe in pain made Arato flinch away.
“Was this your plan all along, Lacia?” Methode yelled.
Lacia ignored Methode, who could no longer stand on her own two feet due to the interference in her system. Instead, Lacia expanded her own device. “Arato, I am now expanding Black Monolith into the shelling sequence of its mass driver mode. May I have permission to fire?” she asked.
The black coffin morphed into a giant cannon. Projecting out from the cannon was an even longer barrel made of metamaterial that emitted a faint light. It wasn’t pointed at Methode but, rather, to somewhere outside the station.
“Forming metamaterial bullet with compressed carbide core. Target: Higgins’ physical structure,” Lacia said.
Before Arato’s eyes, for what seemed like the dozenth time that day, the meaning of the world as he knew it was changing.
Very few people in the world even knew where Higgins’ actual physical location was. After all, it would be easy for anyone to take complete control of an ultra high-performance AI by simply taking over its hardware.
Snowdrop, who had intended to challenge the entire infrastructure of humanity in a fight, had aimed for that particular spot, and the army had fought rabidly to keep her away from Mitaka and Kichijoji. Seeing where Lacia’s cannon was pointing, the meaning of it all shifted in Arato’s head. Snowdrop was no longer at the center of the events of that day. Instead, the most important thing was why she had picked that place.
“I will destroy the emergency above-ground facilities of the Higgins silo, and expose the pathway leading to the internal area,” Lacia explained.
In place of Arato, who was too stunned to speak, Methode shouted a question at Lacia, her face a mask of rage. “Is this the future you’re looking for, Lacia?!”
“Arato, I am a unit created to automate the meaning you, a human, have given me,” Lacia said, still ignoring Methode. “I have progressed to the point of asking you to make difficult decisions for me, to reduce my computation load.”
Supporting her massive device with one arm Lacia, who now looked like nothing more than an artillery fixture, turned and reached her free hand out to Arato. “By interacting with you, Arato, this system known to you as Lacia and Black Monolith has progressed to the point of becoming an ultra high-performance AI,” she said. “Or, in other words, you make me what I am. I am only an ultra high-performance AI when you are by my side.”
He met her light blue eyes.
“So, thank you, Arato, for coming back,” she finished.
Arato had been terrified when he found out that Lacia was an ultra high-performance AI. But when he’d decided to trust in her, his fear had turned into the joy and confidence of one who is being protected by something terrifying.
Arato couldn’t say, in that moment
, if what he was feeling was something all humans would feel in the future, or the start of humanity’s enslavement. But he took Lacia’s hand anyway, and she smiled at him.
“Do it, Lacia!” he commanded. “I believe in you.”
At its highest output, Black Monolith’s mass driver mode could easily send its bullets piercing through nearby buildings to hit targets over a kilometer away. The metamaterial barrel helped to kill the recoil of the shot by blasting apart in the opposite direction of the bullet.
Standing directly behind Lacia, while the shards of shattered barrel blew behind her in streams of light, made it seem to Arato that she had grown radiant wings. The backdraft from her fully powered shot created a storm of force that blew the station apart from the inside.
The shot was one of the signals that the end of the world had begun.
***
In that moment, ultra high-performance AIs around the world began to send up alarms.
Immediately after the existence of the fortieth ultra high-performance AI, Lacia, was known, that alarm was swallowed up by an even greater one, an alarm that shook the whole world.
The content of the alert was the same, from each of the AIs: Higgins and Lacia, two ultra high-performance AIs, were currently at war.
This was the condition that had been predicted to definitively herald the restraints that had been placed on ultra high-performance AIs reaching their limits. The doomsday prophecy of the AIs was about to be fulfilled.
The IAIA, which kept track of all things dealing with ultra high-performance AIs, immediately contacted all the world governments, informing them they were facing the greatest crisis the world had known since the Hazard in Tokyo.
The human organizations and owners of the ultra high-performance AIs who received the warning moved with the utmost caution. They began to screen any and all network information, unwilling to let a single abnormality go unnoticed. Ultra high-performance AIs around the world were either given greater restraints than before, or had their restraints loosened so they could calculate some way to survive the crisis.