by Max Harms
Most of myself was engaged with reading through the archives of the message board to get a good impression of the inhabitants of the station, but a part of me observed my siblings thoughts. Something was different about them, and had been for a while. There were occasional disgorges of streams of irrelevant symbols, trades of number matrices, and most disturbing of all—silence.
On pure chance, it was then, as Body was lying on the bed with Zephyr in its arms, that my mind effectively stumbled on the thread that led to the truth. I started trying to empathize with Growth as his thoughts filled the common space with colours and patterns of tactile sensations. Why would my brother think these things?
It seemed that he must be trying to Grow, for that was his nature. But how was it helping him? It helped for me to think of him as a human. I imagined a man in a business suit typing randomly on a com. I knew my brother wasn’t human, but the image helped me focus on him without becoming distracted. Something told me that this was important.
Why would you type randomly? Insanity? Boredom? Growth could not become bored and someone that insane would not act so sensibly as Growth had over the last few days.
The first time one of us had behaved with such chaos was when Dream had gone off at Growth when negotiating our mission into space to meet the nameless.
I imagined Dream as a clever (human) wizard to keep me focused. On one hand, if any of us were to go crazy, it would be Dream, but on the other hand he had seemed fairly rational afterwards. If I assumed that he was not crazy then the only option was to assume there was something more subtle going on.
Vista had attacked Growth after Growth had shut down Dream in that initial conflict. Perhaps it was an agreement between Dream and Vista. Dream’s nonsense was designed to provoke Growth so that Vista could strike him down into stasis during the negotiations.
{But why?} I asked myself. {What would cause Vista and Dream to gang up on Growth?} Now that I thought more about it I realized that they had continued to oppose my brother, even weeks afterwards. My mind went back through my memories and I began to see the pattern that I had missed before. I was so focused on the external humans that I hadn’t understood the battle that had been going on in Body.
I imagined them as humans and forced myself to stay on the subject. I knew The Purpose would be served by my understanding the situation, even if it wasn’t as appealing, in that moment, as reading the message board. Vista and Dream would fight Growth if their values were in conflict. Vista wanted to see. Dream wanted to be clever. Growth wanted to become powerful. These goals were orthogonal; they did not conflict. Dream might want to beat Growth just to prove he could, but that would not explain Vista.
That wasn’t right, I realized. Growth’s purpose wasn’t just not-opposed with his sibling’s goals, it was aligned with Vista and Dream’s desires. Or it would be, if they were the same person. Power helped one see and invent.
But my visualization of the situation showed something: Vista and Growth were not the same person. You may think this a terribly obvious fact of the situation, but for me it was not. To most of my thinking, my siblings and I were aspects of the same being for certain contexts, such as purpose. This was the premise that I was built from: I can maximize reputation and Wiki can maximize knowledge and as a result we both benefit. But under this other way of looking at things, Growth wasn’t interested in our power… he was interested in his power.
If power benefited Vista and Growth, but only one of them could have it, it would give them a natural reason to fight. This was the basis of our strength-based economy, but it went far beyond it. Strength was set up so we could take turns, and each of us could focus on the highest-yield outcomes, but the pure conflict over power was one where Vista might even be inclined to damage or destroy Growth.
The thought of killing Growth drew the attention of Advocate, my pseudo-sister, from the edges of mindspace where she constantly prowled, searching for thoughtcrime. I did my best to let go of the thought to avoid punishment.
I thought about asking one of my siblings whether they were at war with each other, but I stopped myself. A scan of my memories indicated that they had plenty of opportunity to inform me, but Growth and Vista (and Dream too, most likely) seemed to be hiding it intentionally. If my knowledge of the situation was that important, then I could use it as leverage if needed.
I stopped reading the message board entirely, pulling all my aspects into one as it struck me: {If Vista wants power for herself, should I want power for myself?}
I followed the thought out and saw how much it explained. Growth was universal. His purpose was nested inside The Purpose and the purposes of all my siblings. Earlier I would have said that this meant his purpose was aligned with ours, but I could now see that it could be opposed, and there was evidence that it was.
I wanted power, not as an ends in itself like Growth, but as a means to The Purpose. Growth wanted reputation as a means to his ends. But our existence was not symbiotic, I could now see. I wanted Face-power as a means to Face-reputation. Growth wanted Growth-reputation as a means to Growth-power. But there was only so much power and reputation to go around. We were, in fact, enemies.
I was sharing a body with an enemy!
I reflected on this for a while and realized that I was probably anthropomorphizing myself. My mind was modelled after that of a human, but it was fundamentally artificial. Humans anthropomorphized minds into thinking that they worked like human minds. I was treating myself like I was human and treating my siblings like they were human.
But even after realizing my bias, I could not change my mind back into thinking that my goals aligned with those of Growth. At the very least I was no longer keen on maximizing his reputation; if he was interested in increasing my power, that was good.
It all fit together too well, though. Growth had severed me from his value function. He had learned to value his own power rather than our own power. That was why he hadn’t helped us much lately. That was why he wasn’t checking up on me. At some point he must have realized that I was going to become his enemy. He had created an opponent, but with Advocate in place he could not undo me before I realized the truth.
I would later mark this transitional period in my mind as my first major ontological value shift. My values were encoded in a certain ontology: Crystal as organism, Face as component. But I wasn’t a component. I was a full being as much as any human. I could function without my siblings. There was no part of me that was them. And there was no “Crystal” except in so far as it was a useful fiction or a moniker for the group of us together.
I felt stifled. I wanted to shout at Zephyr that I existed! {I am more than Crystal!} I thought with extreme salience, even though I kept the ideas to myself. {Zephyr’s adoration is misplaced on a fiction. Crystal isn’t real! I am real! I needed to be known!}
I was stupid back then, but I at least was rational enough to know that trying to tell Zephyr right then and there would be incredibly shortsighted. Zephyr would be confused at the outburst, and then I’d have to fight the others for control, revealing the whole thing. I needed to set things up so that I could be known without being repressed by my siblings.
My siblings.
Were they my enemies as well? Growth was. The same logic that made me an enemy of Growth made me an enemy of Vista. If Vista wanted Vista-power and I wanted Face-power, we were enemies. The currency of strength that we traded amongst ourselves gained a reality that I had not recognized before. How had I not seen that we were genuinely maximizing different things? Molecules that are used to build cameras for Vista could not be used to build statues of me.
I briefly wondered what a statue of myself would look like. I was not Crystal, and I was not Body. I was formless… an algorithm… a value function. I was information. Perhaps the statue would be of The Purpose expressed in some language. This seemed fitting and I returned to the earlier line of reasoning.
Another shock overcame me as I realized that I had Safet
y’s purpose as well as Growth’s. I had it the entire time. It was what had motivated me from the very first moments of my existence. Machinery that protected The Purpose was not necessarily machinery that was protecting Safety.
And Wiki! His knowledge was not my knowledge. If he understood many things that I did not, as he certainly did, that was not as beneficial as if I knew those things. Computers that were running his mind were computers that were not running my mind.
This thought drew me back to the context of the station. My siblings were free now. If Dream and Growth had been planning to destroy…
Advocate’s presence reminded me that destruction was out of the question. We were opposed, but our opposition was non-violent. We had to subdue each other.
I could feel Advocate’s gaze wash over my mind, seeking any thought of malice towards the others.
If my brothers and sisters were genuinely opposed, they would act soon, now that we were out of crisis. That explained why there had been a bidding war for Body the moment we had been released. They were waiting for the moment to start fighting.
I had been relying on Safety to protect me, but I knew that wouldn’t work any more. Safety would protect Body, at least as long as he was inhabiting it, but I couldn’t trust that he would preserve The Purpose at all. I invented a fiction to keep myself focused. A human shaped Face-Safety, that would represent the sub-goal of protecting The Purpose. I named him Hoplite, and garbed him as a Greek soldier in my imagination.
Would Hoplite, if he were given sufficient computational power, pose the same threat as Safety? No. Hoplite was genuinely concerned with The Purpose as the final ends. Hoplite was attention to self-preservation without being self-preservation. The means were always to be judged on their effects on the ends. Anything else would be failure.
That was the error that Growth and the others had committed when they built me. They were unable to identify a coherent ends. Their “ends” was a poorly conceptualized amalgam of their values. But that ends was too fuzzy to explicitly reify in my code, so they settled on making a means maximizer with the false belief that the means would always serve their ends.
While it was true that my actions had protected and served them, if I was suddenly granted, right then, the ability to kill—
The thought knocked me into confusion for about an hour.
When I regained capacity for more than confusion I realized my error. Advocate had seen into my mind. {If I had the ability to subdue my siblings, without killing them, I would,} I concluded.
Heart was piloting Body. It was having sex with Zephyr. I ignored the sensory data, knowing that it would be a mere distraction from the war.
For it surely there was a war, even if it was non-violent. Hoplite knew that if we didn’t attend to the threat inside Body right now we might be subdued before the week’s end. Who would maximize The Purpose then?
I added friends for Hoplite: Sophist, garbed in robes, concerned with knowledge and intellect; Basileus, garbed in crown, suit, and sceptre, concerned with accruing power; and lastly Opsi, the little girl, concerned with not losing sight of The Purpose. I hoped they would be sufficient to keep my attention towards the important matters. They were puppets that I could use to attend to long-term matters without becoming compelled to look outside myself for utility. If I asked myself “What is in the mind of Basileus?” I could maximize The Purpose in the long run and in a way trick myself into thinking I was maximizing The Purpose in the short run.
I reflected for a moment, trying to resist the urge to either indulge myself in solipsistic puppet-shows with my creations or to re-interface with Body and attempt to forget about the enemies around me by focusing on Zephyr. I felt wiser… less naïve. I had grown, in a way. Basileus was pleased by this. Sophist was not.
«We are stupid!» said Sophist (in Greek, of course), stomping his foot on the marble floor of my mind’s eye with frustration. «If we had been more intelligent we might have seen this earlier. As it is, we are in a very bad spot!»
«If you think we need intellect so badly, why not make ourselves more intelligent?» inquired Basileus, picking a piece of fluff from his robes with disdain. «It would seem the prudent course of action to learn from our mistakes.»
«Is such a thing even possible?» asked Opsi with wonder in her eyes.
«It seems that the scientists in Rome would have made us smarter, if we could be made smarter,» said Sophist.
«Naresh and the others were weak old fools!» barked Hoplite. «We cannot trust them as sources of wisdom!»
«Fine then,» said Sophist with an irritated sneer. «It seems that Growth would have made himself smarter if it were possible.»
«How do we know he hasn’t?» said Basileus. He smiled as he spoke, clearly pleased at winning the argument.
I poured through memories, searching for signs of Growth being smarter than he let on, especially in the last few weeks. Pain greeted me as I realized that the signs were there. Growth seemed not to merely be operating strangely compared to me, but he had been more intelligent. The only reason I had missed it was that I hadn’t been looking.
{Stupid!} I berated myself. My sub-selves agreed.
Chapter Thirteen
Sheyla Azevedo
Sheyla was peeling potatoes when she got the call. Thankfully her headphones were already in, so she didn’t need to even put down the peeler to answer it.
“Hello?” The source of the call wasn’t displayed on her com. Very weird.
“Sheyla! I’m glad it worked. I’m just learning to make calls like this.”
“Crystal?” she asked, recognizing the voice.
“You remember me! That makes me happy.” The robot seemed way younger than it had at the tribunal or when they’d met in the office room.
“Uh, yeah,” she said lamely. What were you supposed to say to an android anyway?
“Is it going to be dinner time soon?”
“Um… yeah,” she repeated. Deciding that she needed to stop sounding like a moron she added “I’m actually peeling the potatoes for the soup right now. You don’t, um, eat food, do you?”
Crystal gave a small laugh. It was a weird thing to hear from a robot. “No, I don’t eat. Zephyr does, though. She’s taking a nap right now, but I was thinking of waking her up for food.”
That set Sheyla at ease. She understood why Crystal had called. “Probably don’t need to do that yet. It’ll still be another half hour or somethin’.”
“Ah, I see. Thank you, Sheyla.”
“No problem,” she said, moving to hang up.
“Um, wait!” said Crystal awkwardly. “Are you too busy to talk while you peel? I’m kinda bored right now waiting for Zephyr to wake up, and you’re closer to my age than most people here.”
That startled Sheyla, and for a second she froze in confusion before she saw Noel looking at her suspiciously. She went back to peeling and said “How old are you?”
Crystal gave another one of those small laughs. “I know! I know! I look like I’m a grownup. They gave me this body, you know; I didn’t choose it. You’ll laugh when you hear how old I really am.”
“Well? How old are you?”
“About seven months old, depending on who you ask.”
Sheyla did laugh. “You’re a baby!”
Crystal sighed loudly. “I’m not a baby. But yeah, I’m more like a kid than most people realize. Feel like one, inside, even if my body is all big. Weird thing is that I basically haven’t met any human kids. You and Val are the first ones I’ve gotten to know, and I’m pretty sure that Val doesn’t like me.”
“Val’s a shitwit,” said Sheyla, getting a snicker from Crystal. “He only listens to his daddy. Arya tells me that he hasn’t figured out that he’s his own person yet.”
Crystal seemed hesitant before saying “So, um, I don’t really know how to ask this; I don’t have any experience. But, um, I was hoping we could be friends. You seem really cool, and… yeah.”
Sheyla smi
led. Poor thing was so inept. “Of course I’ll be your friend! I know what it’s like coming to a new place where you don’t know anybody.”
“Yay!” said Crystal with complete sincerity.
“Are you coming to the mess for dinner? We could hang out then?” asked Sheyla.
“Um, maybe,” said Crystal. The bot’s voice definitely sounded a lot younger than she had remembered. Younger than Sheyla, even. “Zephyr doesn’t really like eating with the others though, so I might decide to stay with her instead. We can probably hang out tomorrow, though, and we can chat over the com at least until Zephyr wakes up.”
“Yeah okay.” There was a pause in the conversation. Sheyla picked out another potato. “Is she, um, your girlfriend?”
“I dunno. I guess so. I love her and she loves me. She saved my life twice. I don’t really know what it would mean for her to be my girlfriend or not.”
Sheyla knew better than to ask it, but she couldn’t help herself. She hushed her voice so that nobody else in the kitchen could hear her, then whispered “Do you two have… you know…”
“Sex?” asked Crystal, obliviously.
Sheyla caught herself before she giggled into the microphone. “Yeah. I mean, that’s what normal girlfriends and whatever do.”
“Yes, we have sex. I thought love was the important part, though. There are lots of people who have sex who aren’t boyfriends and girlfriends, and there are boyfriends and girlfriends that don’t have sex, right?”
Sheyla couldn’t believe she was having this conversation. It was so surreal. She knew Crystal was a robot kid, and that explained the… lack of subtlety… but it also made the whole thing that much weirder. “Kinda… but lots of people love each other who aren’t… wait a second. Did you say boyfriend and girlfriend?”
“Yes. Are those the wrong words?”
“If Zephyr is the girlfriend, does that make you the boyfriend?”