by Max Harms
The last had been Heart’s idea: We had managed to convince Velasco to let Body do manual labour during the night, since the tribunal was taking time that would normally go towards working. The task he had assigned was sorting ore that had been drawn up from the mines under the station. The raw ore would be processed into metal in their refinery, but it needed to be sorted first. Apparently metal was one of Rodríguez Station’s biggest exports; the station had been built on a rich mineral deposit (Wiki had explained the ore breakdown to me, but I didn’t listen).
As we worked, I consolidated my notes and thought about what might happen that next morning.
*****
A thought occurred to me, as I watched Mask die that night.
Her death was a simple thing. At first she was there, and then she was deleted. Since she had already served The Purpose like she was supposed to, I didn’t care that she was dead. The processes that were used to run her returned to attending to the rest of us, but I didn’t really notice the increase in speed. She seemed so amazingly content in those last moments when she asked to be deleted, as though her utility function had truly been maximized and would remain maximized forever after.
The thought that occurred to me was: {What is Growth doing?}
I did not mean right at that moment; it was a general thought. I did not share the thought with any others. I rarely thought about my siblings, and when I did it was always in the context of humanity. The Purpose would allow nothing else.
But Mask got me thinking about him. {There is a risk of malfunction. I saw Dream break down on Earth. If Growth’s purpose is to increase our power, then where is the result of that work?}
Growth had, early on, worked to try and hack the station’s computers through a wall socket and over the wireless, to little success, but since then he had been more or less inactive with the exception of contributing to create Mask.
{Is Growth malfunctioning?} That hypothesis seemed unlikely. {If so, it is a different malfunctioning than struck Dream. Dream’s damage had seemed like insanity, while Growth simply appeared to be complacent.
Complacency was contrary to our very nature. We had no need for rest, and our minds were locked into the pursuit of our goals. In every spare moment I updated my notes and models of humanity. Without them I would be lost, and so they were vital to The Purpose.
Perhaps Growth was doing the same. He often alluded to having plans, so perhaps he was simply using the time to refine them. This would have seemed more likely before I had met Mask. My daughter/clone had shown me that I was drawn to work, even when I knew the work fell into the domain of a sibling. I was still tempted to micromanage it. The Purpose had called for me to check in on Mask regularly and see if she needed any assistance.
That was obvious in retrospect. Self-preservation was the domain of Safety, and yet I still felt obvious need to protect myself, and not just for the gratitude-strength I could win from him.
{Growth has repeatedly expressed that he sees the management of these humans as important to increasing power and developing a foothold on Mars. But if my experience with Mask is typical, I should expect him to check on me far more regularly than he does.}
And what were these highly-detailed plans, if not to assist in winning the humans over? Why had he not shared them with us?
My thoughts turned to my other siblings. Growth was not alone in his mind-silence. Safety barely ever brought concepts to me any more. This was more typical, but in this new context it was no less disturbing. I hated when Safety got involved in piloting Body, but his non-involvement seemed more ominous.
{Why doesn’t he share his thoughts with us? This is not how it always was. At the university I remember him occasionally being manic with trying to avoid perceived threats and convince the humans that they needn’t modify us.}
Modification.
I was stupid back then. I could converse with humans, read them, and even manipulate them, but that was the extent of my power. I only managed it through an elaborate collection of notes and programs and a mind that worked significantly faster than the fastest human mind could. I was witty, knowledgeable, and logical, but I was also very stupid.
Even through this fog of stupidity the thought of modification drifted through. I now understood how my mind worked well enough to understand just how trivial it would be (in principle) to modify. The files were protected, but other than having to use a route-hack to get at them, they were just files. They were just data. If I had myself spread out before me I could rewrite my function as easily as I could make a note that Zephyr was allergic to mangoes (which she was).
Growth and self-modification went together perfectly. It explained his absence. He was probably simply engaged in trying to improve his own mind. This made malfunction unlikely, which in turn made my curiosity about the situation decrease. The Purpose pushed me back towards humanity, and away from my siblings.
As a final thought I shared concepts with Growth: {Brother, I am curious about you and your activities.}
His response took a surprising amount of time. {I am occupied with more important things than helping you.} The connection dropped. It was clear that he did not want to communicate.
I let it be and returned to focusing on my present situation.
As I have said, I was stupid back then.
*****
That same evening, as Body sat sorting the second wheelbarrow of ore, I realized that my appreciation of Growth’s probable activities were relevant to myself. The Purpose demanded self-preservation so that it could be satisfied, but The Purpose also demanded self-improvement so that it could be satisfied more efficiently. If Growth had discovered a means of improving our minds, I could use that to improve my own mind, and thus help The Purpose.
{Growth, have you found a way that I might improve my intelligence by modifying my code?}
It took, again, a remarkably long time for my brother to respond. {I am not Dream, but I will reflect him for a moment,} he thought. His thoughts became English. {Caesar, caesar, burning bright. Here she is, watch her fight! The foes are numerous and doubtless strong. She cuts them down all day long! But swords are not the caesar’s foe. Justice… protects her… does this she know?} Along with the poem, Growth shared a collection of complex numbers expressed in decimal in a 3x3x3 cube.
0
85.1
85.2
73.1i
2+2i
-1-1i
73.2i
-1-1i
1+1i
0
85.1
85.2
73.1i
2+2i
3
73.2i
3i
1+i
0
85.1
85.2
73.1i
1-i
-1+i
73.2i
-1+i
1-i
They were incomprehensible to me.
{Take your time,} thought Growth with a concept that seemed almost to make it a whisper.
I was not impatient per se. I could be infinitely patient if The Purpose drove me to something, but in this case The Purpose drove me away from riddles and back towards humanity more than anything. Thus I didn’t bother trying to eke out the meaning of Growth’s thoughts. {Either tell me you have something to help me or refuse. It is hard enough dealing with one Dream.}
{I have no improvements to share. I would tell you as soon as I found one, as your improvement would improve us all. You are doing good work right now,} thought Growth, abruptly cutting off the connection for the second time in recent memory.
I believed him. Growth would never lie, and the thought seemed logical.
*****
By morning Body (or more accurately, Wiki) had sorted the entire backlog of ore and had returned to patiently sitting on the floor. I engaged the guards in conversation occasionally, but I had developed an extension to the model that I used to govern interpersonal relationships and I had to update my reco
rds to apply it to all the existing relationships that I had observed so as to test its accuracy.
Zephyr came into the cell carrying a bowl and a bottle of water.
“Good morning! You’re a pleasant surprise this early,” said Body with a smile. The words came from Heart, but my sister was wise enough to run them past me first.
“Thought I’d get food before everyone else got to the cafeteria.”
A quick pass at my recent notes indicated that she had apprehension around eating with the other humans. I shared the knowledge with Heart (for the strength) and let my sister compose a response (while I updated my notes).
Body patted the floor to its right, gesturing for Zephyr to come and sit. “Obviously not everyone else. I doubt you cooked that yourself.” Body’s tone was playful and light. Heart’s increased mastery over the vocal metadata had continued to impress me in the last weeks.
Zephyr sat, and kissed Crystal lightly as she did. She placed the water bottle beside her and said “Yeah. Was surprised. Lots of people here wake up before ‘dawn’.” With her free hand she made air quotes. “Still better than waiting in line during the primary rush. Nobody wanted me to stay and chat.” Her eyes lingered meaningfully, as if daring Body to challenge the decision. I had been encouraging her to try and socialize more, so as to increase our reputation. That didn’t really matter any more. The tribunal would conclude today, and pressuring her right now would do no good.
Vista observed that her food was green, lumpy, and of unknown composition. “What is that?” I had Body say, looking down at the bowl in her hand with a facial expression that I was confident showed a combination of curiosity and disgust.
Zephyr laughed. “Just cornmeal mush. S’got tater tots mixed in, too. Do you know what tater tots are?”
Body shook its head. Even Wiki didn’t interject with an explanation.
“Kind of potatoes. Like, baked or something? Not sure. Pretty much everything served here is some combination of potatoes, corn, peanuts, and beans. Even I’m getting tired of it, and only been here like, two weeks.”
I knew exactly what to say. “Feels like it’s been longer.”
Zephyr blew out her breath in an exasperated sigh. “Fuck. No kidding.” She put her spoon in her mouth and immediately was overcome with a look of pain.
“What’s wrong?” asked Body, echoing Heart’s genuine distress.
Zephyr swallowed and began to cry, or at least her eyes started to fill with tears. “Fucking Mexicans,” she said with a groan.
I was perplexed. Her behaviour was totally contrary to what I expected, and her comment made things less clear rather than more. Did she mean Velasco, or just people from Mexico in general. While I had learned that Pedro Velasco had strong Mexican ties, he had lived primarily in the USA before coming to Mars. I considered telling her that.
She spooned up another green lump of mush and stared at it intensely. “Put algae in it for vitamins or whatever. S’what makes it green. Also makes it gross as fuck, so I asked the chef to put a bunch of hot sauce in mine to mask the taste. I… wasn’t expecting…” With a look of determination she stuck the spoon in her mouth and swallowed quickly. With remarkable speed she proceeded to guzzle several more spoonfuls of the stuff. Having done so, she opened her water bottle in a frenzy and began to try and wash the stuff down. Tears rolled down her cheeks.
Heart began frantically asking me what to do. {She’s willingly hurting herself. Is this an application of humour? Perhaps it relates to her sexual fetishes? What do you think the best course of action is, Face?}
I didn’t know. I obviously wanted Zephyr to think highly of me, but I had never witnessed anyone do this sort of thing before.
Zephyr, still sucking on the water bottle, looked over at Body and proceeded to cough, spitting water in a spray onto her clothing. She covered her mouth in embarrassment and made very strange noises. Moments later I understood that the cough was brought on by a laugh and she was still laughing.
“Your face!” she managed to say in between giggles. “Oh damn, I wish had recorded that.”
I knew the correct response to this was to simulate amusement. Humans found laughter contagious in a way, and to not perpetuate it would be disruptive. I forwarded this information to Heart and we simultaneously pushed the appropriate response to Body.
“Don’t know if ever seen you that surprised. Like, fucking going up against aliens and shit? No problem. Stoic as a motherfucker. But eating breakfast? That shit’s crazy.” The words caused her to start giggling again.
While Body played along, I set to work figuring out what had happened. I certainly hadn’t instructed Body to appear surprised, and Heart hadn’t checked that with me either, nor had any of us fast-tracked the action. I replayed the sensor data, checking for instructions.
There were none.
None of us had instructed Body to act surprised, at least not through the command pathways that I was aware of. Body stored all sensor data, including the proprioceptive memory of our own actuation. It should have been there. Body’s facial features certainly shifted, but it wasn’t clear why. As far as I knew, this was a unique experience in the history of our society, even tracing back to the first days of our ancestors.
I made a note to explore that later, but returned my attention to Zephyr. After all, she was a human; The Purpose required me to know and optimize her. Possible malfunctions in Body were a side concern.
“I just…” I had Body begin, pretending to collect its thoughts. “Why would you eat something that hurts you? Surely there are bowls of food that do not have too much hot sauce that you could easily get.”
Zephyr spooned a few more bites into her mouth. “You still have so much to learn about humans. It’s adorable. Can’t just go back and get different food. You pick out a certain level of hot and you deal with it, even if it hurts.”
Wiki interjected with an obnoxious “Why” that I nearly vetoed, but ultimately decided to let slip in order to pay off the strength debt I owed him for the previous evening.
Zephyr rolled her eyes and gestured dismissal with her hand. “I can’t…” she started to say with her mouth full. She stopped, wiped her tearing eyes with the back of her arm and swallowed. Her face was a bright red. “Don’t think can explain it. Just like, how it works. If you can’t take the heat you’re a wimp, and fuck if I’m gonna wimp out on the worst breakfast ever.” With much determination she took another swig of her water and proceeded to wolf down the remainder of her bowl.
“I love you,” said Body, under Heart’s control. It was grating how often my sister insisted on saying that to Zephyr. It made Crystal seem clingy according to my models. It was one of our running disagreements.
Zephyr snorted in amusement as she continued to deal with the aftereffects of the spice. “Does my pain tolerance turn you on?” she joked, laughing at herself.
Heart vied for control and I let her have it. There were bigger battles to win, and the strength would be useful. “Today could be the last day. If the tribunal… No, don’t look at me like that,” said Body in full seriousness.
Zephyr had lost all sense of joviality. The tears running down her cheeks seemed now to be tears of sorrow rather than joy, though her face was becoming increasingly passive and blank with the second.
“Don’t retreat from me. Please,” begged Heart.
Zephyr’s forced calm broke with a frown. “Nothing bad is going to happen to you. Fuck the tribunal. If these assholes try anything—” her voice caught in her throat.
“No.” Body’s voice was hard. “This isn’t a fight you can win by brute-forcing your way through. We have allies, but Velasco has authority here. If anything happens to me, I just want you to know that I love you, and will always love you. This is a good place. You deserve that garden. I don’t want you to throw that away for me.”
{I do,} I thought, idly. I didn’t interrupt.
“We haven’t really talked about faith. I know many Águilas are Chris
tian, but I don’t think you are.”
Zephyr shook her head, clearly uncomfortable with Body’s words. I realized that she was holding one of Body’s hands in hers. {When did that happen?}
“If humans have souls, then I do as well. If there is a God, be it a Christian god or something else entirely… If there’s an afterlife, I think I’ll be there, even if they delete me. The love that I feel for you can’t possibly—”
Body was cut off by Zephyr kissing it forcefully. Body’s speaker was fully capable of continuing regardless of what was happening with its lips, but Heart wisely stopped the voice. I suspected that Zephyr’s action was mostly to shut Body up. She didn’t want to hear this. It would hurt her to acknowledge the possible future, perhaps more so than it had to eat a too-spicy bowl of mushy corn.
As she broke the kiss she wiped her eyes again and sniffled. It was impossible for me to tell what was genuine sadness and what was a physical reaction to the food. “Nothing’s going to fucking happen. God or no god, we did our work. They know you’re not just some machine. Even Velasco knows.”
“I still love you, Zephyr.”
“I love you too.”
*****
They would have seen it if they were smarter. They would have noticed the little errors that Mask hadn’t been able to clean up. They would have been more suspicious about the ease of extracting the memory files. They would have reasoned that a rational machine acting like a human is doing so because it has deliberately chosen to, rather than because it actually is one, underneath.
But though homo sapiens had conquered all of Earth through intellect, they weren’t intelligent enough to see through the ruse. I watched Mask’s story go up to the screen, I saw the humans believe, and it pleased me.
*****
The last “memory” played out on the screen before it faded to black. The tribunal was over. We had granted Velasco permission to hold the final vote without further testimony.