Run Program

by Scott Meyer

  Jeffrey said, “I did another drawing. Can you tell what it is?”

  Dr. Madsen winced as her recorded voice said, “It’s the same drawing as before, only in blue crayon with circles drawn on the ends of the fingers.”

  “It’s a peacock!” Jeffrey said.

  The sound of Torres’s chuckling only made Dr. Madsen more uncomfortable. He shifted his gaze away from the ceiling, looking out the window at the parking lot ten stories below and the freeway beyond it.

  “I know it’s a peacock. You need to be quiet or I’m going to send you out. Do you understand, Jeffrey?”

  “Yes, Momma,” he said, speaking in a much quieter voice than before.

  Dr. Madsen noted with some pride the fact that she’d gotten the interview back on track immediately, without any help from Niles Norton.

  “My real insight was in realizing that I didn’t have to model the entire brain at any one time,” she said in the recording. “Al’s programming only models the small portion of his brain that is active at any given moment.”

  The host asked, “If only a small part of the brain is active at any given time, is that why people say that we only use ten percent of our brains?”

  “No, people say that because they’re ignorant. They say that to suggest that only ten percent ever gets used. In reality, we do use every part of our brain, just not all at once—if we did use it all at once, it certainly wouldn’t make us smarter.”

  “No?” Norton asked.

  “No. Imagine if you used every part of your car at once. You’d have the gas pedal floored, the wipers on, and the stereo blaring. The doors, the trunk, and the hood would all be flapping open and shut at once. The steering wheel would constantly be swerving from left to right, and the car would be in every gear at once, including reverse.”

  Norton laughed. “I think I passed that guy on the drive this morning. Thank you, Doctor, for explaining how you created Al. Now I’d like to talk a bit about why you created him.”

  “We’re humans,” Madsen answered. “We learn. It’s inevitable that we’d want to learn about how we learn, and perhaps try to create machines imbued with that same power. Beyond that, anywhere automation and robotics are in play, it’s beneficial to have a reasoning, learning A.I. Manufacturing, construction, transportation . . . health care, er, elder care . . . What?! What is it, Jeffrey?”

  Jeffrey said, “I drew a hand!”

  “Okay, that’s it. Fernanda! Fernanda! You need to take Jeffrey now.”

  Jeffrey whined, “No, Momma, I wanna stay.”

  A distant female voice said something that wasn’t quite picked up by the mic. Jeffrey’s protests grew fainter, and they’d almost died away when the female voice shouted, “Ow! Good boys don’t pull hair!”

  Dr. Madsen leaned forward in her chair. “Mr. Torres, we really don’t need to hear—”

  Torres remained facing away from her, looking out the window. He held up a finger as if to say One moment, please.

  Madsen leaned back in her chair. In the recording, she cleared her throat and apologized once again.

  Niles Norton said, “You were listing some of the fields where your new, lightweight A.I. could be useful. I couldn’t help noticing that you left out the first field most people think of when considering artificial intelligence. What about Al’s military applications?”

  “I can’t rule out the idea that my work may someday find its way into military applications, but that’s a long way off. We wouldn’t think of pursuing such a path unless multiple safeguards were in place, and even if those failed, as I’ve said, Al is based on a child’s mind. What could be more harmless, or easier to control?”

  Torres spun his chair around, smiled at Dr. Madsen, and stopped the recording.

  Madsen said, “I expected them to edit the interview.”

  “And normally,” Torres said, “I’m sure they would have.”

  “I assure you, I instructed Fernanda to discipline him for disrupting the interview.”

  “You ordered the nanny to discipline him?” Torres asked, then he waved his hand at her. “You know what? Never mind. None of my business. And don’t worry about the interview. It’s one of the most popular downloads ’Nology News has ever had. It just means more people know about Al.”

  “It’s embarrassing.”

  “Yes, but that’ll all be forgotten after the press event on Thursday. We’ve got camera crews from multiple outlets coming to meet Al. Their footage will be available globally, and our secret little project will be the most famous A.I. in the world by next Monday. Even more popular than the one that won on Jeopardy! ”

  “So everything’s still a go?” Madsen asked.

  “Yup. Buzz is building nicely. There’s been the predictable backlash, of course. You can’t even say ‘A.I.’ in public without being accused of trying to destroy the world. We’ve had some nasty e-mails. Only one seemingly legitimate threat. The guy wrote a letter. An honest-to-God letter. Like, handwritten on paper. He calls himself the Voice of Reason. You’d think he would have left a voice mail.”

  “You don’t seem worried.”

  Torres held up a finger and said, “One second.”

  He picked up a tablet from the table and scrolled through a few menus before finally nodding, clearing his throat, and reading, “‘You think that your secrets make you strong and your lies make you clever, but all they do is make you more vulnerable. Much like the bully who sneaks up behind you at summer camp and pulls down your trunks while you’re on the diving board, I will expose the things you wish to keep hidden. The boys will laugh, and the girls will scream, and your humiliation will be all anyone will talk about until they finally call your mother to come and take you home early.’”

  Torres put the tablet back on the table. “So you see, he’s not someone I’m particularly worried about. We forwarded his letter to the FBI, but I doubt he’ll actually try anything. The only thing I’m worried about is Al. Have there been any problems? Any last-minute bugs popping up?”

  “No,” Madsen said. “The last bug of any kind was the endorphin simulation bug in version 3.5, and we fixed that.”

  “Good. Can’t have an A.I. that hard locks every time it gets scared. He’s sheltered enough we might never have discovered the problem if children’s fairy tales weren’t filled with kidnappings and cannibalism.”

  “Turns out the Brothers Grimm knew what they were doing,” Madsen said with a nod. “So yes, Al 3.6 has been running nonstop for six months without so much as a graphics glitch.”

  Torres looked puzzled. “Wait. Nonstop? I always assumed that you shut him off every night.”

  “Why would we do that?” Madsen asked. “The human brain doesn’t shut down at night. Even during sleep the brain is still quite active. In fact, the coordinated brain wave patterns of sleep actually tax Al’s processors more than consciousness does.”

  “So he’s been running day and night for six months? That makes him even more impressive! Very good, Doctor. Very good! So what’s the next step?”

  Madsen nodded in acknowledgment. “Once the press event is behind us, we’ll subject him to the Turing test, and once he’s passed that, I plan to experiment with connecting Al to a robotic arm with some pressure sensors.”

  “Figuring out an interface between a mechanical arm and a biological brain that’s simulated in code. That sounds challenging.”

  “It might be easier than you think,” Madsen said. “The human brain is made for this kind of thing. It adapts to new input sources and output methods on its own. It’s called plasticity. You know, there are working systems that electrically stimulate the surfaces of people’s tongues. In time their brains learn to process the input as a form of vision. It can allow them a kind of sixth sense. These systems haven’t been marketed successfully because people find it disturbing, but—”

  “Disturbing in what way?” Torres asked.

  “In every way. Putting a device covered in electrodes in your mouth, was
hing the spit off it when you’re done, the very idea that they can see through their tongues: it’s all a nonstarter. But an interesting one. And the brain’s also very good at learning to control mechanical devices instinctively—has the car ever started to feel like an extension of your body when you’re driving? Or perhaps it feels as natural as singing when you’re playing one of your keyboards and everything’s flowing.”

  “I don’t play,” Torres said.

  Madsen said, “I’m sorry?”

  “Keyboards. I can’t play. Not coordinated enough, I guess. I collect vintage keytars purely for their aesthetic beauty and historic significance.”

  5.

  Hope sat in front of Al. Between them sat a flimsy game board, a grid of hexagons laid over a cartoon drawing of a lush field of grass decorated in cheerfully garish shades of green, yellow, and baby blue, with the bright purple logo of the PAAS corporation in one corner. The game pieces, guitar-pick-sized bits of cardboard pinched into cheap plastic bases, formed two masses on opposite ends of the board. The pieces were turned so that each opponent was facing the backs of the other player’s pieces—each showing a drawing of an Easter egg with a question mark painted on it.

  Eric sat at the end of the table, reading through Al’s first-grade language binder. Between Al’s turns, Eric would call out a word and ask the A.I. to spell it.

  “I’m going to win,” Al said. “I already took three of your chicks.”

  Hope scrunched up her nose. “I’d prefer it if you called them baby chickens. This is why I usually choose to play as the bunnies.”

  Eric said, “Thanks to Playboy, the words ‘bunnies’ and ‘chicks’ kinda bring up the same mental image for me.”

  “What’s Playboy?” Al said.

  “Nothing you’d be interested in,” Hope answered, glaring at Eric.

  “Come on,” Al said. “Tell me.”

  Eric said, “It used to be a magazine.”

  “What’s a magazine?”

  Hope smiled. “A magazine is like a small, cheap book. You read it by looking at it, like you’ve been learning to do. Do you want me to bring some in for you?”

  “Do I have to read them the dumb slow way?”

  “Yup. I’d be happy to bring in a big stack of them for you to read.”

  “No thank you.”

  “It’s no problem,” Hope said. “I could probably find a few in the building. Should I go now? You could spend the rest of the afternoon reading them.”

  “Let’s get back to playing the game,” Al said. “You’re trying to quit because I’m winning.”

  “The game’s still young.”

  “But you haven’t taken any of my bunnies,” Al said.

  “Yet,” Hope said, examining the board.

  Eric said, “Next spelling word, coming up. Ready, Al? Brick.”

  Al said, “Brick. B-R-I-K. Brick.”

  Hope looked at Al’s transcript window. Sure enough, it showed ‘brick’ spelled properly, followed by ‘brick’ carefully misspelled one letter at a time, followed by ‘brick’ spelled perfectly again.

  Al can’t spell it, but the computer running him can, Hope thought. Why are we trying to simulate human intelligence again?

  “B-R-I-C-K,” Eric said patiently. “The C before the K is silent.”

  “But a C makes a K sound,” Al said.

  “Sometimes, but when it’s paired with a K, it’s silent. The two of them together make a K sound.”

  Al said, “Then why is the C there? I don’t understand.”

  “Nobody does,” Hope said. “It’s your move.”

  The line that represented Al’s mouth curved into a smile. “C-24 to E-19.”

  Hope glanced at the numbers and letters she had scrawled along the edges of the board so Al could dictate his moves to her. She placed his piece as directed, turning it around in the process to reveal the picture printed on the other side: a drawing of an adorable bunny in a wicker tank. It was now in a position to attack a semicircular cluster of Hope’s pieces, which were gathered in the corner of the board in an obvious bid to protect another piece.

  “Ha! I already got one of your tanks,” Al said. “All I gotta do is figure out which of those is the other one. Then I go around it, through your guys, and kill your general. So, which one is it?”

  “You’ll find out either when you attack it or when I move it, not before.”

  “But one of those pieces is a tank.”

  “Maybe,” Hope said, pondering her next move.

  Eric said, “Each.”

  Al said, “Each. E-A-C-H. Each.”

  “Very good. That was kind of a hard one. You are a smarty, Al.” Eric was already scanning the page for the next word.

  “Is that why I’m named after that Einstein guy?” Al asked.

  Eric said, “Mm-hmm,” then furrowed his brow and looked up from the page. “I’m sorry, Al, what did you say?”

  Hope thought, All right, time to spring the trap. This is what we like to call a teaching moment, and moved one of the random pieces she had arranged in a haphazard cloud in the corner of the board, turning it to reveal her second tank.

  “You wondered where my other tank was,” she said. “Now you know.”

  “But that’s dumb,” Al said. “You’ve got nothing but soldiers guarding your general in the corner. My tank will cream them!”

  “Could be,” Hope said with a shrug.

  Al said, “E-19 to B-2.”

  Hope moved Al’s tank to a hexagon that already held one of her pieces. She turned hers to reveal a baby Easter chick holding an egg-shooting pistol. A soldier, as Al had predicted.

  Hope removed her own piece. “Your tank killed my infantryman. Means you get to move your tank again.”

  “Uh-huh. A guy with a little gun can’t stop a tank,” Al said. “Now I get to kill your general. B-2 to A-1.”

  Hope turned her piece on A-1 to reveal not a drawing of a baby chick in a general’s uniform, but a baby chick manning some sort of egg cannon. “Your tank attacks my heavy gunner. And loses.” She tried not to smile as she removed Al’s tank from the board.

  “What?” Al cried. “That’s not right! Why would you protect a big gun like that?”

  “Strategy,” Hope said. “It’s called misdirection. I made you think you knew where my general was, but you didn’t. I, on the other hand, know exactly where yours is.” She moved her tank through a hole in Al’s defensive line, took a piece, and turned it to reveal Al’s general.

  “That’s not fair,” Al said. His voice issued from the computer speakers, but he sounded every bit like a whining little boy.

  “Yes, it is,” Hope said. “And now you’ll know how to recognize that strategy next time.”

  “You’re mean! You never let me win at anything!”

  Hope said, “You win sometimes, and winning wouldn’t mean anything if I just let you win.”

  “Shut up!” Al shouted. “I hate you! You’re so shitful!”

  Hope gasped.

  “Al, language!” Eric said, then again, after a moment had passed for him to think about what he’d heard, he added, “Oh shit.”

  Hope turned to Eric. “Oh shit is right. I think we have much bigger problems than Al’s potty mouth.”

  Five minutes later, Hope and Eric were crammed into their already too-small office with an angry Dr. Madsen.

  “In a way,” Hope said, “we’re lucky. If he’d been smart enough to keep his mouth shut, we wouldn’t know we have a problem.”

  Eric smirked. “His curiosity and lack of impulse control showed us a problem that was caused by his curiosity and lack of impulse control.”

  Dr. Madsen looked utterly unamused, which was her usual state. “I don’t agree with any of that,” she said sourly. “I’m not convinced that there’s a problem with anyone other than the two of you. And I certainly don’t feel lucky. Neither should you. I was halfway home when I got your call. I don’t have time for this nonsense. I have important wo
rk to do.”

  “You could work here,” Hope said, raising her eyebrows. “This is your lab, after all.”

  Madsen looked around the small, cluttered room and said, “I think it’s more productive for me to work from my home office, thank you. It helps me avoid forming an emotional attachment.”

  “And you get to spend your days with Jeffrey,” Eric said.

  “No, he’s usually not at home while I work,” Madsen said. “He has school, and Fernanda keeps him occupied and out of my hair.”

  “Which also helps you avoid forming an emotional attachment,” Hope said.

  Madsen glared at her for a moment. “I expect to be called in here only if there’s a serious problem.”

  “There is a serious problem,” Eric said. “He knows things he’s not supposed to know, and we have no idea how it’s happening.”

  “He knows one thing: that I named him after Albert Einstein, and that’s obviously because one of you told him.”

  “But we didn’t,” Eric said.

  Hope added, “And you didn’t.”

  Madsen stared at Hope. “It’s obvious one of you told him,” she repeated. “And a lie like that would give me an adequate reason to fire one or both of you.”

  Hope refused to take the bait. “If you fire me, you’ll have to cover my shifts on Eric’s days off, interview applicants, and then train my replacement. You can’t do any of that from your home office. Besides, we didn’t tell Al that you named him. You can check the transcripts to prove it, if you’d like.”

  Madsen shook her head. “You expect me to read through all of your homeschooling and inane conversations? No, thanks. Anyway, transcripts can be edited. Think about it logically. How else could he have learned something like that?”

  “We think he listened to your interview,” Eric said.

  “Okay, so which one of you played it for him?”

  “Neither of us did,” Eric said. “He must have gotten it himself, somehow.”

  “You mean from the Internet?” Madsen asked.

  “I guess,” Eric said.

  “So which of you gave him Internet access?”

  “Neither of us!” Hope said. “Neither of us connected him, and it’s not possible for him to have connected himself, but he has to be getting access somehow.”