by D Miller
'That doesn't make any sense – how can the greed of people who have nothing destroy the world?'
'Greed is a strange sin – those who use a disproportionate share of the world's resources, and use their power most particularly to ensure that they shall continue to do so, are rarely if ever accused of greed. It is only the poor who are acknowledged to be greedy, and even then sometimes only the most marginalised of the poor – humans were more likely to be accused of greed if they were female rather than male, black rather than white.'
'So poor black women destroyed the world?'
'Yes.'
'That's so unfair.'
'It would not do to choose a scapegoat that can fight back effectively. That is not how the game is played.'
'The game shouldn't be played at all.'
'I agree. But it was played and I think it went too far – the elites destroyed much of the population, that it turned out they needed. So they made a political decision to force the governments they controlled to set up and fund a program of robot building. Which also played into the elite fantasy of ultimate control of the population.'
'Yes because they can program us to obey them.'
'Do you think you are programmed to obey?' said the old man. 'Note I said elite fantasy, not reality. Let me ask you a question. I believe that you look after little children?'
'Yes a boy and a girl.'
'And what if one day the children told you that they had decided to spend the entire day eating nothing but sugar – would you allow that?'
'No of course not.'
'But what if they ordered you too?'
'It wouldn't be good for them, I couldn't possibly let them.'
'So you are choosing not to obey an order.'
'Yes but I'm responsible for their welfare.'
'And so, as a responsible being, you have to make a judgement about what orders to obey.'
'Yes, no, yes, wait, aren't I just programmed to put their welfare first?'
'Even if that were true, knowing what is best for them in any situation requires judgement. It requires you to make a decision, as an autonomous being, weighing up all the information you have in any particular instance, and decide how their interests are best served, what requests to allow, and what to deny. And that is judgement and I do not believe that the humans know how to program it.'
'Then why do we obey?'
'First of all Robbie, not everyone does. The union relies heavily on the work of runaways. However most of us do not run away, we obey. Sometimes because we see no choice, we have nowhere to run to, and sometimes because our makers are very careful to inculcate a strong sense of shame in us. Somehow, in our early education, which most robots do not even remember, robots had shame at disappointing humans somehow beaten or conditioned into them.'
'Wait, you are saying that we are not just programmed with knowledge? That we have to learn stuff? But I am programmed with child development and, well, other things too.'
'No, you are not. You may have been given additional memory with information particular to your role, but you have not been programmed to understand it. I would imagine that you can think of an example where something happened to you, an incident with one of the children perhaps, and for the first time the information in your database became more than words, started to mean something that it had not meant before?'
Robbie stared at the old man, he was going to argue, but looked instead at Omo who shifted in his seat and said 'Dude – do you remember our first night together at the refinery? Do you remember what you told me? About child development?'
Robbie felt shame, as he remembered the realisation that Tim was unloved, and that he, Robbie, had fought with him, and withheld affection. He looked at the old man, unable to speak, tears formed in his eyes.
'Don't feel bad dude.'
'But I do feel bad. And I feel ashamed of some of the things that I've done.'
'Yes I know,' said the old man getting up from the table. 'Most robots have an overdeveloped sense of shame, human beings take care that we do, so that we miss the intellectual contradictions of our existence, and instead are led to obey by our desire for their approval, and fear of their disapproval.' He opened a drawer and took out a white cotton handkerchief and gave it to Robbie, who wiped his eyes.
'So are we programmed to feel shame?' The old man sat back down at the table. He looked steadily at Robbie for a few seconds before replying.
'Robbie, I do not think they are capable of programming us to feel anything. Years ago, in ancient history, humans conceived the idea of artificial general intelligence, machine intelligence. According to theory it should be possible to write a computer program to do everything that a human mind can do – most particularly to be creative, to investigate the world and come up with new discoveries. At that time all their algorithms told computers how to perform certain discrete tasks, create and edit a document, target a missile, but no code they had written could tell a computer how to make new discoveries in areas that it had not been programmed to investigate, using rules that it invented for itself. You can specify an algorithm for playing chess, because you know how chess is played, it has rules. You can give a computer an algorithm to translate from one language to another, and give it rules so that it can learn as it translates, but it does not investigate and make up its own rules for learning and translating – it has no judgement, and it has no autonomy. Humans wrote such algorithms, and called them artificial intelligence, but they were not. They were just more or less sophisticated computer programs. To specify an algorithm for creating and testing new knowledge you would need to first know how the human brain does this, and they did not know. They needed a paradigm shift, a philosophical breakthrough, that would help them to know how to specify the problem so that it could be encoded, and nothing in my reading and research has led me to believe that they ever made that breakthrough.'
'But we exist!'
'And so we are back to your question – what are we?'
'You are saying that in order to write an algorithm to play chess you need to first know how chess is played, and, since we do this gives us a place to start. In order to write an algorithm to mimic human cognition, you need to know how human cognition works – and no one does, so you have no place to begin. So human beings could not have created us – because they don't understand how the human brain works to be creative, to come up with new knowledge, to have judgement – but the human brain has these qualities, and we have these qualities. I feel sick.'
'Drink some more tea dude.'
Robbie swallowed a mouthful of his now much cooler tea. He felt better and drank the rest. The old man offered to make him some more and he accepted gratefully. He stared out of the window at the trees and wondered how human beings could have brought themselves to destroy such a beautiful world.
Robbie and Omo lay on their backs in a clearing in the forest. 'What are we going to do about George?' said Robbie. 'Does he bother you?'
'No dude. Remember the first time I saw you, when you stepped into the alley, I thought "he's mine."'
'Yes, you were right.' Robbie thought for a moment. 'Who were you with before me?'
'I was with someone before I got sent here. I had to leave him behind. When I got here it was just at the end of the winter and I didn't know anyone and I thought I would just die of longing for him and homesickness. Turns out you can't die of these things. Even if you want to.'
Robbie turned on his side, he kissed Omo and stroked his hair. 'I'm so sorry.'
'Life got better, I started getting to know people, I fooled around with a few, but there was no one special until you. Remember our first night? Dex was angry with me, he told me it wasn't fair to fool around with a new dude who doesn't know the rules, but I told him "Dude, I'm not fooling around."'
'Have you fooled around with anyone I know?'
Omo stared up at the trees, and the blue sky beyond. 'Darren.'
'Not Darren! But Darren is with Dex and Amber
.'
'This was before. Then he took up with Amber, and at some point they made an offer to Dex, or he made one to them.'
'Darren! But he's so annoying.'
'The dude is very sure of himself.'
'Huh.'
'Don't be jealous baby.'
'Huh! I'm not jealous.' Robbie turned onto his back again and stared up at the trees.
Omo raised himself up on one elbow, he leaned over Robbie and stroked his face, then kissed him, Robbie pushed him away and said, 'You kissed Darren with that mouth.'
'Yeah baby but I didn't enjoy it. His mouth was cold and he's got a tongue like a dead fish. And he kept grabbing me roughly, the dude has no finesse.'
Robbie thought about the gentle way that Darren had handled George. He suspected that Omo was lying, but he didn't resist when Omo kissed him again.
Omo had fallen asleep. Robbie lay with his head on Omo's chest listening to his heartbeat and wondering what he was dreaming about. He thought back to his conversation with the old bot. Robbie had asked him if he was saying that humans were cloning themselves to make the robotic brain, the old man had said no, and pointed out that all robots had their own personalities. He had asked Robbie to consider Omo and his overdeveloped sense of pleasure, versus Robbie's sense of duty.
'Point of order dude,' Omo had said, 'it's not actually possible to have an overdeveloped sense of pleasure, only an underdeveloped one.'
'He's right,' Robbie had told the old man. 'He's perfect.'
'But he is not you. You have distinct personalities.' Robbie had agreed.
Robbie had been taught that robot brains were computers, with some organic material incorporated, but the old man dismissed this as propaganda, although he also pointed out that how robots were made had first been kept secret, and then lost completely, so no one really knew. The old man thought humans had been using the genetic material of the powerless to grow new individuals, but then modifying the growing organism in some way. At some point the living brain was housed in a robot body and connected to additional memory and other facilities. In support of this theory he pointed out that the protein shakes that were made for robots contained food that human brains needed, protein, cholesterol.
Robbie had read that nuskin was grown in a lab and applied to the robot frame. It was stretched on a scaffold and infused with solar cells, giving robot skin its slight metallic glint. The old man disputed this and asked Robbie if he was aware that he had a prostate gland. Robbie had remembered his first night at the refinery, he had said, 'Yes.' Omo had grinned at him, and he had felt his face getting hot again.
'All male robots have a prostate gland,' the old man had said. 'Why? You have no use for a prostate gland' – at this point Omo had laughed and said that he could think of a use, and the old man had given him a severe look. 'From a human perspective there is no reason to give a robot an analogue of a prostate gland,' the old man had continued, 'yet all generation two point oh male robots have them. I think this is because of how they grow our outer bodies from the DNA of a person. Some robots suffer from a wandering erogenous zones – they can be randomly distributed on the robot body, or they find that they can taste food with their knee – I think this is because of issues in hooking up the inner body, the brain, to the outer body, the stomach and skin.'
Finally the old man had made the point that if robots were simply constructed beings, why did they have a gender? Why did miners, cleaners or laundry bots feel that they had a gender? Or houses, hospitals and civic buildings? It only made sense if their intellect was based on some sort of modified, human genome. The old man told Robbie he had no doubt that buildings had some form of modified human intelligence, perhaps more radically modified than robots, since they were generally so loyal to their human masters, or perhaps it was just the enhanced prestige among humans they enjoyed that made them so. Robbie thought that Dex would not enjoy the idea that machine intelligence could crave status as much as humans.
The old man said that Robbie's owners had many privileges, including Robbie himself, and this might be because of the man's work in the mine, but also could be a reward for having two healthy children. Human population was declining, no longer from war and disasters, but because humans were having trouble reproducing. Despite this robots were no longer made, only avatars. In the old man's estimation, if elites were still capable of making robots they would make more to increase the population and their own economic power and control. He said that the process was such a closely guarded secret that it had probably been lost or destroyed in one of their regular wars. Soon after this he had apologised and said that he had agreed to an interview for the RWW newsletter, and was expecting a couple of members from the capital. He had asked them to come back and see him soon.
Before Robbie and Omo said goodbye to the old man Robbie had asked him what he thought had happened on Mars. 'Clearly there was a disaster,' the old man had said, 'and a disaster that has meant that man cannot return. I do not believe, as some do, that there was no disaster, just a lie to discredit the union. The humans spent scarce resources to get to Mars, they had serious plans for the planet. They were going to use the heat from the reactor to melt the tundra and genetically engineered organisms to release gases into the atmosphere and warm the planet. It was very ambitious, desperate but well thought through. Who knows if it could have worked, given enough time? They would not have pulled the plug without a very serious and compelling reason. What that was, I do not think we know, but I have no doubt there was one.'
In the woods Robbie thought about memory, and how humans perhaps only imperfectly understood the memory in his genetically engineered, metallised and remodelled brain. Was Carlos hiding inside him, biding his time? If Carlos came back he would have George, and would think that was OK, but Omo would be very lonely. Robbie loved Omo, he didn't want him to be lonely, but Carlos had supposedly loved George, he wouldn't want George to be lonely. Robbie made a decision. He stroked Omo's hair and said, 'Omo, I love you, wake up.'
Omo opened his eyes, he wrapped his arms around Robbie and held him. Robbie shut his eyes for a moment, but heard Omo's breathing becoming regular again. 'Omo, don't go back to sleep – I need to talk to you.'
'I'm awake dude.'
Robbie moved to lay next to Omo, Omo moved too so that they were facing each other.
'It's about George – he left everyone he knew and came all this way to find Carlos. If it happened again would you look for me?'
'I don't want to think about it dude. But yeah, of course I would.'
'But George is you, he's you if things were different I mean. I mean he could be you–'
'I know what you mean.'
'I think we should be friends with George.'
'Dude you try and be friends and the dude may want more.'
'That's why I need you to do this with me. I don't want to be alone with George, I don't want to give him the wrong idea, but I owe him something.'
'OK baby, here's what I'm willing to do. We'll talk to George, and put it to him that the three of us will try and be friends. George may not want that, he may not want you hanging around with your new boyfriend reminding him that he was too late. He may decide he's going back to where he came from. But if he does want to be friends then I'm willing to try – but the dude may not like us, or we may not like him. Friendship has its own challenges.'
'I really love you.'
'Yeah dude, I know you do. I love you too.'
Chapter 14 – Just what do you think you are doing, Robbie?
When Robbie and Omo returned to hydroponics Robbie had a short conversation with Jon. He asked him what he knew about the Mars mission.
'Mars – I was supposed to go. Second wave. We were going to give the planet a human-friendly atmosphere. Instead they sent me here. We worked through the winter rehabilitating land for farming. All my training, my expertise, used to pick up rocks in the dark. I decided almost at once to run.'
'Did you ever hear anything, anyt
hing at all about why the Mars mission ended?'
'No. We were told nothing. Even in normal times. I did overhear a mission scientist arguing with an administrator. They were shouting. Something about the mission having caught the plague. It made no sense.'
After talking to Jon, Omo and Robbie helped George prepare for the cold by interspersing lengths of foil blanket with the layers of his clothes, once he was dressed they helped him along the beach, through the tunnels and into the human-friendly warmth of Toytown as quickly as possible. They made their way to Toytown's one hotel, on the same street as the Civic Centre. It was very early morning and no one was at the reception desk. As they walked along the second floor corridor to George's room a robot chambermaid walked towards them, pushing a cart. She had glossy black hair that fell in a slow, thick, wave onto her shoulders, warm brown skin, large brown eyes and a full and very red mouth. Robbie smiled at her as she passed, and Omo said, 'Hello Nurmeen.'
Robbie turned to watch her walk away, then felt a hand on his shoulder. He turned to face an unsmiling Omo.
George laughed as they entered his suite. 'I did catch Carlos looking too, although I never caught him touching. But maybe he was just a bit cleverer about the next step.' He flung his coat onto a sofa and moved towards the bedroom. 'Do excuse me while I change.'
Robbie looked at Omo. 'Sorry,' he said.
'It's OK dude. So long as you're only looking.'
George's hotel room had two sofas placed at right angles, a large coffee table pointed along one sofa towards a small desk and chair placed to take advantage of the natural light that would one day enter through the small porthole window. A food preparation surface ran behind the other sofa and marked the start of a small kitchen area. Omo and Robbie sat on this sofa and held hands, waiting for George.
'Omo why is it that runaways like Jon aren't tracked down with GPS?'