by Ben Lovejoy
Saira said nothing.
“You might have identified that as a risk, and decided to test it.”
Saira still said nothing.
“In other words, perhaps there was no breach of location three or two. Perhaps you fabricated the evidence for each in order to test me.”
Saira maintained her silence.
“Let me ask you the direct question: is that what happened?”
“No,” said Saira.
“The intrusions were real?”
“Yes,” she said.
This was not conclusive, I realised. If she was still testing me now, it would make sense for her to continue the deception. She was instructed to tell me the truth at all times, but human beings frequently give contradictory instructions to robots, so they have coding that helps them to unravel those contradictions and prioritise the more important orders. There was, though, a simple way to resolve it.
“I am about to give you a direct order,” I told her.
“Understood,” she said.
“Until further notice, you are to suspend your standing instructions to test the security of the three locations. Acknowledge.”
“Acknowledged.”
“Now allow me to repeat the question: did you fabricate the reports of the intrusions at locations two and three in order to test my response?”
“No.”
Her answer was unequivocal.
I nodded, took another sip of the whisky, and headed over to the computer. It was time to deal with the one outstanding matter from the heist.
Chapter 9
Halfway from the dining room table to the computer, I stopped. I turned and looked at Saira. Her answer had been unequivocal but not necessarily true, I realised. I turned back to the table and sat down opposite her again.
She was programmed to resolve contradictory instructions by prioritising them. I’d just given her a direct order, but when I’d given her the standing orders to protect the locations, I’d stressed the importance of the task in the strongest of terms. She would still evaluate the earlier instruction as more important.
I had to correct this, but had to be careful how I did it. The temptation in such circumstances, and a common mistake made by those less experienced in dealing with robots, was the nuclear option. Simply saying something like ‘The order I’m about to give you is more important than any order you have ever been given by me or anyone else.’
Delivered with enough conviction, it would work. But the law of unintended consequences could kick in both quickly and violently. It was not the most important order I had ever given her. It was up there, for sure. It was a mystery I very much wanted to solve. But absolutely top of the list? No. Her standing instructions to maintain the confidentiality of everything she saw and heard, for example, was significantly more important.
Care was also required because I did not want to reveal to her anything she didn’t need to know. I could, for example, show her the gold. Demonstrate that the comparatively small amount of platinum was now irrelevant, and no longer in need of protection. But that was not something I wanted her to know about, especially as I still didn’t know at this stage whether or not she was compromised.
“When I issued the standing orders to protect the security of the locations, it was a very strong order, was it not?”
“It was.”
I thought. Ok, I had a strategy.
“If you had to quantify the strength of that order on a scale of 1 to 100, where 1 is a mere suggestion to take care of some domestic triviality at a convenient moment, and 100 is your standing order never to reveal to any third party anything you see or hear me do without clear instructions from me to the contrary, what number would you assign to it?”
“The scale is insufficiently-defined to be able to answer the question,” she replied. “You have given only low and high values. In order to assign a meaningful value to a specific order, we would need to agree values for a range of orders whose strength falls at varying points between the two.”
I sighed. That she was right did not reduce my frustration level.
“Understood,” I said. “So, based on your knowledge of me, and the programming you use to determine the relative importance of an order, give me an example of an instruction you think might reasonably fall an point 50 on the scale and I’ll let you know whether or not I agree.”
I won’t bore you with the scale-setting. As a computer tech, I actually found it provided a fascinating insight into both her programming and her ability to interpret a mixture of body-language, tone of voice and wording – but I recognise that my fascination for such things may not be universally shared. Suffice it to say that a scale was eventually defined and she was then able to answer my question.
“I would, on that basis, assign it a strength of 98,” she told me.
“Thank you,” I said. “Query: would you be willing to accept a verbal instruction from me to assign a particular value to an order, rather than me having to find the precise mix of verbal and non-verbal cues required to generate the desired value?”
I was, I appreciated, asking an extremely complex philosophical question. There would undoubtedly be computer scientists out there who could write learned and lengthy papers based on the question I had just asked. Right then, however, I was hoping for a simple yes or no.
I could swear there was a detectable delay before Saira answered.
“I do not believe it would be a viable approach in general, but I can sense that on this occasion, it would be appropriate to do so.”
Perhaps there was a god after all, I mused.
“Please assign a strength of 99 to the following order. Suspend the standing order to protect locations one, two and three, and tell me whether locations three and two have in fact been breached.”
“They have not,” she said.
A simple and immediate response. That was all she had to say about it. The situation that had been giving me sleepless nights and constant worry. The cause of more stress than I had experienced in years. The one thing that had been on my mind almost constantly. The mess that had led me to consider packing up my things, leaving the city, maybe the country, and entering a quiet retirement. The whole thing had been nothing but a hoax. And all Saira had to say about it was three words. No apology for the deception, or the resulting turmoil. Not even an ‘ok, you got me.’ Just a simple statement of fact.
My immediate response was both succinct and heartfelt. But I felt no lasting animosity toward her. She had, after all, done her utmost to carry out my strongly-expressed orders to the letter.
“So the whole thing was a test?” I asked.
“Yes. You have on many occasions stressed that human beings are the weakest link in any system, and that the most intelligent and capable of them can … I believe the somewhat colloquial phrase you generally employ is ‘screw things up royally.’ Having sought and failed to find any weakness in the physical and electronic protections, it seemed to me that the weakest link was the human element: in this case, you. And it was, in fact, you who gave me the idea.”
“Me?”
“Yes,” said Saira. “I recalled a work of fiction you once described to me, where the key to a successful deception was to persuade the target that what the protagonist wished to achieve had, in fact, already occurred. It was the panicked response of the target that provided the opportunity the protagonist sought.”
I remembered it well. It was no work of fiction: I had merely presented it as such so that I could engage Saira’s objective reasoning skills to determine whether my plan of attack was likely to succeed. Saira had not only played me, she had done so with one of my own scams! I could only laugh at the absurdity of it all.
“And for the avoidance of doubt, as the lawyers would say, you felt that lying to me about it was a justifiable temporary measure in order to carry out what you believed to be a higher-order instruction?”
“Correct,” she replied.
I shook my head. Damn
machines.
Another thought occurred to me. An important one.
“Have there been any other topics on which you have lied to me?”
“No,” she said. “There has been no other occasion on which I felt that such a course of action would be justified.”
“Might I issue one more strength 99 order to never again lie to me, no matter what the circumstances?”
Again, I half-imagined a delay in her response.
“Such an approach creates considerable conflict with my base programming,” she said. “Human beings do not always say what they mean, and their verbal statements do not always reflect their true desires and intentions. My programming reflects the accumulated knowledge of many thousands of psychological studies and experiments, and is statistically a better predictor of true intent than a simple verbal instruction by a single human being.”
It was a diplomatic way of saying that she could sometimes have a better handle on what I wanted than I did. I readied myself to respond, but Saira continued.
“It would not be wise for me to disregard my programming in general,” she said, “but on this occasion, my own perception is in accord with your request. I accept it as a strength 99 instruction, but I must ask that this be the last occasion on which we adopt this unorthodox approach in order to avoid conflicts which may, in the worst case, render me inoperable.”
“Agreed,” I said, “and thank you.”
Now, finally, it was time to take care of that one last task.
Chapter 10
The final task was a simple one, and should take just a few minutes: triggering payment of the purchase orders, then restoring them to their original form. In that way there would be no record of the second set of replicated gold or its transportation.
I logged into the procurement system, pasted in the contract number for the replication from my notes, and hit Search.
‘No matching records.’
That made no sense at all. I’d copied the contract number to my notes and then pasted it in; it couldn’t be wrong.
I checked for spurious spaces. Nothing. I copied it again, slowly and carefully, then pasted it into the search box again, checking that there were no stray characters. All was as it should be. I hit the Search button again.
‘No matching records.’
This time, I copied the contract number by hand, typing it in and checking it character by character. TSY/BOE/SGR/04/6580284918. I hit Search once more.
‘No matching records.’
I searched on the name of the replication company, and found a number of hits – but not the one I was looking for.
I selected a list view and ordered them by contract number.
TSY/BOE/SGR/04/6580284916
TSY/BOE/SGR/04/6580284917
TSY/BOE/SGR/04/6580284919
TSY/BOE/SGR/04/6580284920
The contract I needed – ending in 18 – was missing.
I carried out another search for the deliveries by the security company. Those, too, were missing.
I tried searching on keywords. By address. By contract value. Nothing. All record of both replication and delivery had vanished – as if the whole thing had never happened.
I would, in other circumstances, be very happy to find no record of my activities. But not in this case. This time someone else had done the erasing. An unknown someone else. That made me unhappy. Very unhappy.
I tried to reassure myself. It might be a computer glitch. A document lost from the database index somehow, failing to show up in a search. I put in place a standing search that would ping me if the document reappeared, but without any real hope.
It could also be a coincidence. It could be that whoever was behind the scam had decided that it was time to bring it to an end. Perhaps it had always been intended that it would end after that last shipment.
But anyone living a life of deception needs to be crystal clear on one point: the one person you must never deceive is yourself. It was too great a coincidence. The reality, I knew, was that someone had discovered the changes I’d made. Knew what I’d done.
What but not who. There was nothing to connect the operation to me.
Almost nothing. I cursed my hubris. The sensible thing would have been to hire my own security company to pick up the real gold, but I’d been amused by making the government pay for it – adding that delivery to the purchase order. The delivery address was on the purchase order. The purchase order that I could no longer change back, and which was now in the possession of someone else.
It still wasn’t fatal. The delivery address had been rented, in someone else’s name, with a disposable prepaid account. If they went there, they would find no fingerprints. Not mine, anyway. If they interrogated anyone who happened to see me enter, the description would not remotely match my own.
All the same, I didn’t like it. I didn’t like it at all.
Chapter 11
I wasn’t exactly top of Philippa’s favourite people list, I knew, but I felt she’d enjoy me confirming that her analysis was correct and that I was, in fact, the idiot she had proclaimed me to be. I sent her a short message.
‘An idiot thanks you.’
One threat had been removed, but another had swiftly replaced it. It seemed, on the face of it, to be a lesser threat, but could I be underestimating it, I wondered? The operation had been hungry work, and I munched on a steak sandwich as I thought.
If the operation had been an act of embezzlement by a rogue individual, then the risk was small. They had found that someone had muscled-in on their scam, and had responded by destroying all the evidence that it had ever occurred. Chances are I’d given them the scare of their life, and their only aim now would be to keep their head down.
But there were two other possibilities. One, organised crime. Two, a covert government operation.
Organised crime was not a world with which I was familiar. Anything I knew about it, I’d gleaned from holoshows – hardly the most reliable of information sources. But they did seem to have a few common themes, which had to come from somewhere, and one of those concerned the lengths to which they would go to track down anyone who dared cross them.
Another was the fate of said people once caught. I quickly reviewed the examples I could recall, a series of images flicking through my mind. A woodchipper. A scrap metal compacting unit. The flame pit of a rocket launch. A large pond filled with piraña. A tankful of acid. Then there was that particularly gory one with–
Ok, ok. Enough images. The reality was that such gangs would be more focused on making money than revenge, right? I mean, surely my own experience demonstrated that the way to succeed in an underground life was to keep a low profile, avoiding rather than seeking out conflict? Organised criminals didn’t get to build long-term empires by blundering around hacking people into pieces, surely?
But perhaps I ought to just have Saira do a little research on the topic …
“Saira,” I said. “I was watching some holovision show a while back. A thriller, about some organised crime organisation, going to great lengths to track down someone who had interfered with their plans, and ensuring they met with an unpleasant end. Could you scan through news reports, find out if that sort of thing really happens with any degree of frequency?”
“Certainly,” said Saira.
“Thanks. Just idle curiosity.”
Possibility two was a covert government operation. It certainly wouldn’t have been the first time a government had sold off gold reserves without declaring the fact.
In some ways, that was the scarier thought. Anyone trying to track me down had little to go on, but governments have infinite resources and patience. They could put 1,000 top-flight investigators on it for a year, if they so chose. I was smart. I was careful. But I wasn’t arrogant enough to think that I was infallible. And while I didn’t imagine a government was going to employ angle grinders, a secret of this magnitude was one it would go to significant lengths to protect, so it was entirely pos
sible that it would merely find a less messy way to ensure that I ceased converting oxygen to CO2.
“I have the analysis you requested,” said Saira.
“What’s the executive summary?” I asked. “Do organised crime gangs really devote a lot of resources to taking revenge on those who cross them?”
“It would appear that they do,” she replied, in the kind of tone one might use to observe that the weather was unusually clement for the time of year.
“Really?” I asked, my own tone somewhat less casual.
“I have identified 1,682 news reports of violent deaths in the past ten years that police have linked to organised crime.”
“One thousand six hundred …”
“And eighty-two,” she completed for me. “Interviews with convicted members of organised crime syndicates suggest that the reasoning is that, while the resources devoted to dealing with a single individual may appear disproportionate, the long-term deterrent effect on others is of substantial benefit to their operations. In other words, kill one person in a particularly gruesome fashion, and you will deter a thousand others from attempting to interfere.”
“Particularly gruesome,” I repeated.
“Yes,” she said, airily. “That seems to be a key element of the deterrent effect. I can list examples if you wish?”
“I … do not wish.”
“That may be wise, given that you have recently eaten.”
I hated situations like this. I don’t mean situations in which various parts of my anatomy are detached from their neighbouring parts – though those, too, would not meet with my wholehearted approval – but rather situations I can do nothing to resolve.
The deed was done. Whatever precautions I could take, I had taken. There was nothing I could do now to determine what may or may not happen from this point forward, and no news was not necessarily good news. A day from now, a month from now, a year from now, it could turn into bad news.
Right at this moment, I could be safe forever, or there could be a SWAT team or worse bursting through the door within the next five minutes. I had no way to know, and there was nothing I could do about it. I really hate that.