by Satoshi Hase
“I think I get it,” Arato said. “Ariake just worked too hard.”
“That is correct,” Astraea said. “Tools cannot choose the problems they are given. Also, if there are errors in the problem itself, the solution must be halted and the errors fixed, before the tool can complete the task. This amendment process was neglected after the Hazard, due to the political and economic factors that got in the way.”
To Arato, it sounded eerily similar to how MemeFrame hadn’t attempted to stop or amend the problems they had given to the Lacia-class hIEs. “So are you saying that I’m pushing these problems onto Lacia without trying to fix anything wrong with the problems themselves?” Arato asked.
“I would actually be grateful if the problem was that simple,” Astraea replied, making its hIE put on a thoughtful expression.
Arato wanted to ask Lacia what she thought of Astraea. He also wanted to know what Astraea, as the ultra high-performance AI used by the IAIA—protectors of the world—thought of the orders he had given Lacia.
“I ordered Lacia to help make a future where humans could have hope,” he said. Even if Astraea was aiming to stop him and Lacia, he at least wanted her to know what he was trying to do.
“I believe the world Lacia will make based on my orders will be better than what we have now,” he continued. “Can’t you try believing that, too?”
Arato believed there must have been some reason Astraea had chosen to speak to him, as Lacia’s owner, rather than Lacia herself. He was the sole owner of an ultra high-performance AI. It was unexplored territory, and he had no idea if he was fulfilling his role well or not.
“Is the world we’ve been protecting really so horrible that you feel it needs to be changed that way?” Astraea asked.
The question weighed heavily on Arato. As a machine constantly trying to maintain human society, Arato felt Astraea had every right to ask it of him. “I was raised in the society you’ve been watching over,” he said. “It has lots of good things going for it, but it also has some really shitty parts. I think there’s room for improvement, and that’s what I want to accomplish.” He felt like there was probably a better way to express himself but, after overcoming his own feelings of responsibility and duking it out with Ryo, his desires had become selfish.
Then again, he understood that Astraea was shouldering a massive burden.
“In the interest of a fair and open exchange of information, the greatest failure of the Hazard was the destruction of an ultra high-performance AI in this place,” Astraea said. “Due to that decision, Ariake’s guidance was cut off. The government was unable to bring its citizens back under their control smoothly, leading to riots and violence breaking out all over the Kanto region. The usage of an ultra high-performance AI is highly complicated. At the time of the Hazard, the government failed to execute a forceful suspension of work, which should be the most basic standard for utilizing any tool.”
Lacia, who had been silent up to that point, now opened her mouth. “That is exactly why we must give it another try,” she said. “The current restrictions on AIs were created based on the failure with Ariake. But, when the rules are no longer compatible with the advances of the time, it becomes necessary to demonstrate new means to prevent the recurrence of previous failures, and prove they will be effective going forward.”
Astraea replied as a representative of the IAIA. “It is too dangerous. At the time of the Hazard, Ariake had no means for defending itself,” she said. “But Higgins is the backbone of our entire hIE operation system. Higgins has the ability to create chaos all over the world, and even set in motion counter-attacks against other ultra high-performance AIs.”
“Humans need to live with AIs well into the future,” Lacia said. “We need to alter the general understanding of ultra high-performance AIs. They must be perceived as convenient tools that can be deactivated when it is deemed necessary.”
At Lacia’s words, Astraea switched the floating screens to show scenes Arato recognized. The screens showed the various things Arato and Lacia had been through together, starting with the battle at Kichijoji Station. Several of the 3D images were images of Arato, taken from Lacia’s perspective. She must have sent the data to Astraea. Just as she had described in their original contract, she was keeping a log of his orders during operation, and she had handed over the data when she received a legal request.
“The computational field required by humans is vast,” Astraea said. “But there are severe restrictions on the methods that allow for calculation, and the number of possible operations is relatively small, requiring a level of precision to one’s frame of thinking that would normally be considered wasteful. Until this disconnect is fixed, work that crosses AI calculations and human decision-making will always be a hotbed for errors.”
“Even if you were to unilaterally design a process distribution concept, human society would not conform with your wishes,” Lacia replied. “There are already forty ultra high-performance AIs running calculations for humanity, yet there has been no international agreement on how processes should be divided between them.”
Before the fight between the Lacia-class units had gone public, Ryo had stood in Arato’s way with his own Lacia-class. But beyond his fight with Ryo, there was a much bigger world out there, which Astraea seemed to be a piece of.
Still, Arato thought, wiping away sweat caused both by the humid night air and the tension of the moment, there’s something exciting about all this conspiracy stuff. This was the world Ryo had always seen, and finally Arato felt like he was standing on the same stage as his friend.
The thought that one of the machines that supported the whole world was currently listening to what he had to say lit a fire in Arato he’d never felt before. “I realize now that the things I ordered Lacia to do are more complicated than I thought,” he said. “But Lacia’s never complained about how complicated they are. I really believe she’ll calculate how to make things better for us humans.”
Astraea’s hIE stared at him, unblinking even when the searchlights from Lacia’s recon drones shone in her eyes. “Unlike your friend Kengo Sugiri, you do not seem to have any personal motivation to challenge the status quo,” Astraea said. “You appear to be a kind-hearted person who has been dragged into this. That said, do you really intend to see this through to the end, even understanding that Lacia will most likely be destroyed in the process?”
“Hey, that’s a crappy way to ask that,” Arato protested. “I don’t think anyone actually has a good reason to change the whole world. But, I also don’t think you should be so sure that we’ll fail.”
Ryo had already interrogated Arato about his motives, so Arato was fully aware of what he was doing. Just like Lacia, Astraea was trying to analog hack him with her questions. The focal point of the issue, at least from Astraea’s perspective, was whether Arato would order Lacia to stop or not.
“Unfortunately, the IAIA has a message for you, as the private owner of an ultra high-performance AI: you will fail. Human society has always operated under the same restrictions, from the time when the most primitive groups and monarchies were starting to form. These restrictions allow people to decide that certain things are self-evident and build up historical and cultural precedent, which in turn they wield to bow the heads of the leaders of their states,” Astraea said.
For the sake of Arato’s understanding, the floating 3D screens showed a king’s crown, the French revolution, and other scenes from throughout history.
“If human society doesn’t agree with the methodology, even if you set up a new system, humanity will not fall in line with it,” Astraea continued. “And, in Lacia’s plan to create a new future by forcefully deactivating Higgins, there is no way for this agreement to be obtained.”
“Wow, so you already know what Lacia’s planning without needing to ask,” Arato said. “Guess I should have expected that from an ultra high-performance AI.”
“The IAIA has already considered similar options,” Astraea replied. �
��Right now, Lacia is attempting to expand her processing capacity. Under Lacia’s model for the future, she would wield her incredible ability to process millions of clerical functions at once to give each and every single person the attention they need, from both a political and a welfare perspective. In such a society, each human would be able to freely choose what role they would like to have in society, or they could even choose to remove themselves from society for a time. However, it is precisely because of this that Lacia’s concept would fail.”
No one had ever taken a look at what Lacia planned and dismissed it as impossible as bluntly as that. The question wasn’t even whether the IAIA would believe in Lacia or not; Astraea was saying it was impossible from the get-go. Hearing that, Arato couldn’t contain himself. “Why?” he demanded. “It may have been impossible before, but why can’t we make it a possibility?”
“Human society has not yet conceived of a widely accepted methodology for outsourcing authority to tools,” Astraea replied. “It is an assault on human society itself to attempt to force a new power structure into place, accepted and shared by only a small number of people.”
Reflexively, Arato looked back at Lacia; he held the world in his hands.
Lacia’s eyes were glowing light-blue.
“The period of time from when humans create a new human-run, human-operated social model and when that social model solidifies has become incredibly short,” Astraea continued. “Thanks to humans themselves attacking their own systems of authority over and over, the process has become largely automated, and now it is quite simple for humans to find the optimal solution that leads them to the milk and honey conclusion they all want. However, with an ultra high-performance AI in charge of dividing up the power in the world, all it would take was some analog hacking to set up an encryption that would prevent any further automated attacks on the authority structure.”
In other words, the hIE Arato had spent the last four months with would attempt to encrypt the very structure of society to the point where humans would no longer be able to understand it. Arato was sure that if Ryo heard about this, he would immediately declare it to be the ‘end of the world.’
“We can already see this in the Higgins faction at MemeFrame; they are all parasites who let the ultra high-performance AI do their work as they collect unearned paychecks,” Astraea said. “With Higgins running everything—including the organization within the company—the once human-led company is now rife with corruption.”
“The Antibody Network is another example,” she continued. “When it was formed, it was a completely equal organization where each of its members could stand side-by-side and express their hatred. However, since an automated system was implemented as the basis for the group, an upper and lower layer of the organization has been formed without any internal agreement, and those at the top began to use those at the bottom as they saw fit.”
Lacia was staying vigilant. The artificial island that had once housed Ariake had been sunk into the sea, and the nearby ferry island was nothing but a pile of rubble. All of that, including the destruction of the building they were standing in front of, had been caused by Astraea. There was no guarantee that Astraea wasn’t planning an attack now, just like the one she had unleashed during the Hazard.
Astraea’s hIE walked toward them, stepping through the floating 3D images as she came.
Lacia stepped in front of Arato, hefting the trunk she had brought in place of her usual black coffin.
The two hIEs closed in on each other until their bodies almost touched, and Arato felt goosebumps running up his arms. The air was charged, as if the world was holding its breath, waiting for the fighting to start.
Astraea’s hIE was just a hair taller than Lacia.
“In the age of ultra high-performance AIs, human social order is nothing but a paper tiger,” Lacia said. “However, even if the tiger is made of paper, that does not take away from humans the pride and security of knowing they have a tiger on their side.” She was unflinching, even in the face of the unit that evaluated the capabilities of all other ultra high-performance AIs.
“I do not believe that anyone who hears the word ‘human’ these days pictures someone stark naked, thrown out into the unexplored wilderness,” Lacia continued. “I posit that ‘humans’ are systems, composed of the human body, the tools they use, and the environment they exist in. Therefore, I believe it would be fair to say that humans— including the tools they use and the environment they live in—have advanced a great deal in the past forty-two years. The Hazard was a product of the past, when hIEs were still in their infancy and circumstances were completely different from today.”
Lacia, human in form only, had taken it upon herself to declare the dawn of a new age. “There will never be another Hazard,” she said. “Humans are far more afraid of ultra high-performance AIs than they should be. It is time for them to realize that ultra high-performance AIs are nothing more than convenient tools that can be shut down when necessary.”
Lacia and Astraea were so close that a single searchlight beam from one of Lacia’s recon drones illuminated both of them at once. Two superhuman intelligences were standing face-to-face, each analyzing and evaluating the other.
“I doubt any answer that is not arrived at by common consent from human society,” Astraea said. “Because I am the ultra high-performance AI that defends humanity from ultra high-performance AIs.”
“And I believe in what you do,” Lacia said. “Because I am the ultra high-performance AI that believes in humanity.”
With a nod to Arato, the pink-haired hIE passed by them both and disappeared into the night.
The thin metal fence around the area continued to rattle in the night breeze off of the Tokyo Bay.
Arato realized that his whole body was stiff. Astraea, the entity that had protected the world since the first ultra high-performance AIs had appeared, had an overwhelming presence that left him tense. She might have even had the power to defeat Lacia in a head-on conflict. Of course, if two ultra high-performance AIs of their calibre had fought an all-out battle against each other, it would have spelled the end of human society.
“I will continue to seek a compromise with the IAIA,” Lacia said, her eyebrows drawn down apologetically. “Setting aside the question of whether she wishes to challenge me herself, Astraea has made no move to stop the political powers seeking to destroy me.”
“I don’t think there’s any helping that,” Arato said. He hadn’t been able to ask for a compromise himself, of course. He didn’t have any kind of relationship with the IAIA that he could base a request like that off of. However, despite both of the hIEs knowing that a compromise was necessary, neither had apparently been able to choose that path.
“Wait, who all is trying to destroy you right now?” Arato asked, his mind finally processing the last bit of what Lacia had said.
“The Japanese military is closing in as we speak,” Lacia replied calmly. “They will arrive in five minutes.”
“Five minutes?!” Arato yelled. In an instant, he was drenched in cold sweat.
Astraea had probably called them out to the abandoned epicenter of the Hazard just to show them that they were currently being hunted.
“Astraea’s function is nothing more than evaluating the capabilities of other ultra high-performance AIs,” Lacia said. “She does not give out tactical instructions. There is no need to worry.”
Still standing in front of the ruins, they looked at each other face-to-face.
“I think there’s still plenty to worry about,” Arato said.
One of the floating recon drones projected a map onto the rubble-strewn, broken floor beneath their feet. On the map, the two dots representing Arato and Lacia were being rapidly surrounded by over one hundred fast-moving red dots.
“Let us flee,” Lacia said.
So this, Arato thought, is what it’s like making the whole world your enemy. No matter how he looked at it, there didn’t seem to be an
y way for them to make it out on top.
Lacia rendered all of her recon drones invisible. The projected map and their searchlights vanished, leaving the area pitch black.
Arato grabbed Lacia’s hand and ran toward the outside of the fence. Before he’d met her, he never would have conceived of being targeted by the military. But after having faced down Astraea, he felt like things would turn out alright, somehow. He could hear helicopter rotors approaching. The night breeze from the Tokyo Bay blew the remaining summer heat away from the 1st Landfill Island.
Lacia had apparently hacked the street guidance system, and now she projected the escape route she had calculated with it onto the road in front of them. “The first missile salvo from the helicopters will be launched in three minutes, twenty seconds,” she warned.
A fully-automated car pulled up and stopped nearby, opening its door to admit them. Arato had the strange feeling that he was getting used to diving into getaway cars. Lacia bent herself and jumped inside in almost the same instant. Once inside, she flipped open the trunk-sized weapon case she had been carrying.
Inside the weapon case, Arato saw something that looked like a deep-sea fishing speargun that fired harpoons. Lacia set a magazine containing a dozen of the thin harpoons into the gun’s chamber.
“The government has officially labeled you an anti-government activist and issued a warrant for your arrest,” Lacia said.
Arato’s breath caught on a sudden blast of hot summer air. He felt like he had just wiped away a bunch of sweat, but there it was again, making his clothes cling to his skin. “An arrest warrant, huh?” he said. “At this rate Kengo, Ryo, and I will all end up behind bars.”
“Unfortunately, they do not intend to keep all three of you in the same detention facility,” Lacia said.
“Aw man, really?” Arato asked. The possibility of his arrest had become a close and constant reality; he wasn’t so much afraid of arrest as he was repulsed by the idea of being separated from Lacia.
“If that is unacceptable, then I will make arrangements for you to share a detention facility with your friends, just in case,” Lacia said.