The Phoenix Project

by Gene Kim

  “Erik, it’s been a long time since gait and Orlando,” the audit partner says warmly. “I was sure our paths would cross again, but I never would have guessed it would be at a client engagement! What have you been up to lately?”

  Erik laughs and says, “Mostly, happily sailing on my boat. A friend asked me to join the Parts Unlimited board, partly due to their external auditors making trouble with a bunch of young, bottom-up auditors who strayed off the reservation. I should have known you’d be involved.”

  The audit partner looks genuinely embarrassed, and they huddle together, whispering.

  For the past five hours, John, Wes, and I sat on the sidelines while the business managers walked the auditors through a precise discussion about how the it control issues simply couldn’t lead to an undetected financial reporting error. They pulled out something called the “gait Principles” document and cited some of the enclosed flowcharts.

  Like watching a tennis match, the ball went back and forth between our team and the auditors, using words like “linkage,” “significance,” and “controls reliance.” On occasion, Dick would trot in a bunch of experts from the relevant business areas to show that even if someone malicious managed to cause a failure in the it control, the fraud would still be caught by another control somewhere downstream.

  Managers from Materials Management, Order Entry, Treasury, and Human Resources showed that even if the application, database, operating system, and firewall were riddled with security holes and thoroughly compromised, the fraudulent transaction would still be caught by some daily or weekly inventory reconciliation report.

  Over and over again, they went through scenarios that assumed all the it infrastructure was made of Swiss cheese, where any disgruntled or wrongdoing employee or external, malicious hacker could log in and commit fraud with impunity.

  But they would still detect any material error in the financial statements.

  Once, Dick pointed out that an entire department of twenty people is responsible for spotting erroneous, let alone fraudulent, orders. They, and not an it control, served as the business safety net.

  Each time, the auditors, often reluctantly, agreed that controls reliance was placed on finance doing reconciliations. And not on the it systems or the it controls within.

  This was news to me. But I certainly wasn’t going to disagree with them. In fact, if shutting up and staying silent would allow Parts Unlimited to escape all the audit findings, I’d be happy to drool and pretend to be unable to read.

  “You have a minute to talk?” I hear John say beside me in a scratchy voice.

  He’s still slumped over, his head in his hands.

  “Sure,” I say, looking around at the nearly empty room. It’s just John and me at the large conference table, while Erik continues his whispered powwow with the audit partner in the far corner.

  John looks awful. If his shirt were just a little more wrinkled, and maybe had a stain or two in front, he could almost pass as a homeless person.

  “John, are you coming down with something? You don’t look so hot,” I say.

  His expression turns ugly, “Do you know how much political capital I’ve spent over the last two years, trying to get everyone to do the right thing? This organization has been kicking the information security can down the road for a decade. I put absolutely everything on the line. I told them the world would end if they didn’t go beyond lip-service, and at least try to fix some of these systemic it security issues… I mean, we need to at least pretend to care.”

  From the other side of the room, I see Erik turn to look at us. The audit partner doesn’t seem to have heard John. Nevertheless, Erik puts his arm around him and collegially moves the conversation into the hallway, closing the door loudly behind him.

  Oblivious, John continues, “You know, there are times when I think I’m the only person in this entire company that actually cares about the security of our systems and data. Do you know how it feels to have the entire Dev organization hiding their activities from me, and having to beg people to tell me where they’re meeting? What is this, elementary school? I’m only trying to help them do their jobs!”

  When I don’t say anything, he just sneers at me. “Don’t look at me like that. I know you look down at me, Bill.”

  I look at him with genuine surprise.

  “I know you never read my e-mails. I have to call you to even get you to open them up—I know, because I get the read receipts while we’re on the phone, you asshole.”

  Ah.

  But I’ve read many of his e-mails without him having to call me first. However, before I can respond, he barrels forward, “You all look down on me. You know, I used to manage servers, just like you do. But I found my calling doing information security. I wanted to help catch bad guys. I wanted to help organizations protect themselves from people who were out to get them. It came out of a sense of duty and a desire to make the world a better place.

  “But ever since I’ve been here, all I do is fight the corporate bureaucracy and the business, even though I’m trying to protect them from themselves.” Laughing harshly he says, “The auditors were supposed to put the screws on us. They were supposed to punish us sinners for our ungodly ways. And you know what? All afternoon, we just watched the audit partner pamper us with kid gloves. What is the point of even having an information security program at all? Even the auditors don’t care! Everything just got brushed under the rug for the cost of a golf game.”

  John is almost shouting, “Our auditors should be put on trial for incompetence! All those findings they dismissed were basic hygiene issues! We live in a churning cesspool of risk. I’m amazed this place doesn’t just collapse under its own weight from lack of caring. I’ve waited for years for everything to come crashing down upon us.”

  He pauses, whispering, “And yet, here we still are…”

  Just then, Erik enters the room again, slamming the door behind him. He grabs the seat closest to the door and looks sternly at John.

  “You know what your problem is, Jimmy?” Erik says, pointing his finger at him. “You are like the political commissar who walks onto the plant floor, proudly flashing your badge at all the line workers, sadistically poking your nose in everybody’s business and intimidating them into doing your bidding, just to increase your own puny sense of self-worth. Half the time, you break more than you fix. Worse, you screw up the work schedules of everyone who’s actually doing important work.”

  This is going way overboard.

  John sputters, “Who do you think you are? I’m trying to keep this organization secure and keep the auditors away! I’m—”

  “Why, thank you for nothing, Mr. ciso,” Erik says, interrupting him. “As you just observed, the organization can keep the auditors away without you having to do anything at all. You are like the plumber who doesn’t even realize that you’re servicing an airplane, let alone the route you’re flying, or the business condition of the airline.”

  By now, John is white as a sheet, his jaw hanging open.

  I’m about to intervene on his behalf, when Erik stands up and shouts to John, “I don’t have anything further to say to you until you prove to me that you understand what just happened in this room. The business managed to dodge the sox-404 audit bullet, without any help from your team. Until you figure out how and why, you don’t have any business interfering with the daily operations of this organization. This should be your guiding principle: You win when you protect the organization without putting meaningless work into the it system. And you win even more when you can take meaningless work out of the it system.”

  He then turns to me and says, “Bill, you just may be right. You guys around here sure seem to have completely screwed up information security.”

  I never said any such thing. I turn to look at John, intending to convey that I have no idea what he’s talking about, but John doesn’t notice me. He’s staring at Erik with an expression of intense hatred on his face.

 
; Erik says to me, pointing his thumb at John, “This guy is like the qa manager who has his group writing millions of new tests for a product we don’t even ship anymore and then files millions of bug reports for features that no longer exist. Obviously, he is making what you and I would call a ‘scoping error.’”

  John is shaking with outrage. He says, “How dare you! As a potential board director, I can’t believe you’re telling us to put our customer data and financial statements at risk!”

  Erik looks calmly back at John. “You really don’t get it, do you? The biggest risk to Parts Unlimited is going out of business. And you seem hell-bent on making it go out of business even faster, with all your ill-conceived, irrelevant technical minutia. No wonder you’ve been marginalized! Everyone else is at least trying to help the business survive. If this were an episode of Survivor, you’d have been voted off a long time ago!”

  By now, Erik is standing over John. “Jimmy, Parts Unlimited has at least four of my family’s credit card numbers in your systems. I need you to protect that data. But you’ll never adequately protect it when the work product is already in production. You need to protect it in the processes that create the work product.”

  Putting his hands in his pockets, he says more softly, “You want a clue? Go to mrp-8 plant and find the plant safety officer. Go talk to her, find out what she’s trying to accomplish and how she does it.”

  Erik’s expression brightens slightly and he adds, “And please convey my regards to her. I’ll be ready to talk with you again when Dick says he actually wants you around.”

  With that, he walks out the door.

  John looks at me, “What the hell?”

  Pulling myself out of my chair, I say, “Don’t let it get to you. He says similar things to me. I’m exhausted and I’m going home. I suggest you do the same.”

  John stands up wordlessly. With the calm expression remaining on his face, he pushes the three-ring binder off the table. It hits the ground with a large thump, all the contents scattering everywhere. Hundreds of pages are now strewn across the floor.

  He looks at me with a humorless smile and says, “I will. Go home, that is. I don’t know if I’ll be in tomorrow—or ever. What’s the point, really?”

  He then walks out of the room.

  I stare at John’s binder, not quite believing he discarded it so carelessly. He’s been carrying it around for over two years. In front of where he was sitting is a single piece of paper, almost blank with a few lines scribbled on it. Wondering if it’s a suicide note or a resignation letter, I sneak a quick peek at what appears to be a poem.

  A haiku?

  Here I sit, hands tied

  Room angry, I could save them

  If only they knew

  CHAPTER 22

  • Monday, September 29

  The Monday following the audit meeting, John disappeared. There is a betting pool in the noc speculating whether he suffered a nervous breakdown, was fired, is just hiding, or worse.

  I see Wes and some of his engineers, all laughing loudly, presumably at John’s expense.

  I clear my throat to get Wes’ attention. When he walks over, I turn around so that my back is to the noc, shielding everyone from hearing what I’m telling Wes. “Do me a favor? Don’t fan the rumor mill about John. Remember what Steve was trying to impress upon us at the off-site? We need to build a mutually respectful and trusted working relationship with him.”

  Wes’ smile disappears and after a moment, he finally says, “Yeah, I know. I’m just kidding, okay?”

  “Good,” I say, nodding. “Okay, enough of that. Follow me. I need to talk to you and Patty about the monitoring project.” We go to her office, where she’s sitting at her desk, typing away in a project management application, full of Gantt charts.

  “Got a half hour?” I ask her.

  When she nods, we gather around her conference table. I say, “I talked with Erik on Friday before the audit meeting. Here’s what I learned.”

  I tell them how Erik validated that we can release the monitoring project and how important this project is to further elevate Brent. I then try to explain the thought process of how we can determine which projects we can safely release, based on whether they have any dependencies on Brent.

  “Wait a second. Bill of resources and routings?” Wes says, suddenly looking very dubious. “Bill, I don’t need to remind you that we’re not running a factory here. This is it work. We use our brains to get things done, not our hands. I know Erik has said a couple of smart things here and there, but come on… This sounds like some sort of consultant parlor trick.”

  “Look, I’m having trouble getting my head around this, too,” I say. “But can you really say that the conclusions we’re making based on his thinking are wrong? Do you think it’s unsafe to release the monitoring project?”

  Patty wrinkles her forehead. “We know that it work can be projects or changes. And in many of the projects, there are many tasks or subprojects that show up over and over again. Like setting up a server. It’s recurring work. I guess you could call that a subassembly.”

  She stands up, walks to the whiteboard, and draws some boxes. “Let’s use the example of configuring a server. It involves procurement, installing the os and applications on it according to some specification, and then getting it racked and stacked. Then we validate that it’s been built correctly. Each of these steps are typically done by different people. Maybe each step is like a work center, each with its own machines, methods, men, and measures.”

  With less certainty, she continues, “But I’m not sure if I know what the machine would be.”

  I smile as Patty scrawls on the board. She’s making some leaps that I haven’t been able to make. I don’t know where she’ll end up, but I think she’s on the right track.

  “Maybe the machine,” I speculate, “is the tools necessary to do the work? The virtualization management consoles, terminal sessions, and maybe the virtual disk space that we attach to it?”

  Patty shakes her head. “Maybe. The consoles and terminals sound like they could be the machine. And I think disk space, the applications, license keys, and so forth are all actually inputs or the raw materials needed to create the outputs.”

  She stares at the whiteboard. At last, she says, “I suspect that until we do a couple of these, we’ll just be stumbling in the dark. I’m starting to think that this whole work center notion actually describes it work pretty well. For this server setup example, we know that it’s a work center that gets hit by almost every business and it project. If we nail this down, we’ll actually be able to provide better estimates to Kirsten and all her project managers.”

  “Give me a break, guys,” Wes says. “First, our work is not repetitive. Second, it requires a lot of knowledge, unlike the people who just assemble parts or tighten screws. We hire very smart people with experience. Trust me. We can’t standardize our work like manufacturing does.”

  I consider Wes’ point. “Last week, I think I would have agreed with you, Wes. But I watched one of the final assembly work centers on the manufacturing floor for fifteen minutes last week. I was overwhelmed with everything that was going on. Frankly, I could barely keep up with it. Despite trying to make everything repetitive and repeatable, they still had to do an incredible amount of improvisation and problem solving just to hit their daily production goals. They’re doing a whole lot more than tightening screws. They’re performing heroics every day, using every bit of experience and smarts they have.”

  I say adamantly, “They really earned my respect. If it weren’t for them, we all wouldn’t even have jobs. I think we have a lot to learn from plant floor management.”

  I pause. “Let’s start the monitoring project as soon as we can. The sooner we start, the sooner we’ll get the benefits. We need to protect each of our resources as if they were all Brents, so let’s get this done.”

  “There’s one more thing,” Patty says. “I keep thinking about the lanes of wo
rk we’re trying to create. I’d like to test some of these concepts with the incoming service requests, like account add/change/deletes, password resets, and—you know—laptop replacements.”

  She looks uncomfortably at my giant laptop, which is in even worse shape than when I first got it three weeks ago. I’ve had to put even more duct tape on it to keep it from falling apart, due to some further damage I caused when I used my car keys to pry it open. And now, half the paint on the screen lid has flaked off.

  “Oh, for crying out loud,” Wes groans, looking at it, genuinely embarrassed. “I can’t believe we haven’t gotten you a replacement. We don’t suck that much. Patty, I’ll find someone for you to dedicate to the laptop and desktop backlog.”

  “Fantastic,” Patty replies. “I have a little experiment in mind that I’d like to try out.”

  Not wanting to get in the way, I say, “Make it so.”

  When I get to the office on the following Monday, Patty is waiting for me. “You have a second?” she asks, obviously eager to show me something.

  Next thing I know, I’m standing in Patty’s Change Coordination Room. I immediately spot on the back wall a new board. On it: index cards arranged in four rows.

  The rows are labeled “Move worker office,” “Add/change/delete account,” “Provision new desktop/laptop,” and “Reset password.”

  Each row has been divided up into three columns, labeled “Ready,” “Doing,” and “Done.”

  Interesting. This looks vaguely familiar. “What is this? Another change board?”

  Patty breaks out into a grin and says, “It’s a kanban board. After our last meeting, I went to mrp-8 myself. I was so curious about this work center notion that I had to see it in action. I managed to find one of the supervisors that I’ve worked with before, and he spent an hour with me showing how they managed the flow of work.”

  Patty explains that a kanban board, among many other things, is one of the primary ways our manufacturing plants schedule and pull work through the system. It makes demand and wip visible, and is used to signal upstream and downstream stations.