by Kim, Gene
Holy crap.
Patty had promised laptop delivery for Friday, and I’m receiving it two days early.
I log on to make sure it’s been configured properly. All the applications seem to be there, all my data have been transferred, e-mail is working, the network drives show up like before, and I can install new applications.
I feel tears of gratitude welling up when I see how fast my new laptop is. Grabbing Patty’s schedule, I go next door. “I love the new laptop. Two days ahead of schedule, even. Everyone ahead of me got their systems, too, right?”
Patty grins. “Yep. Every single one of them. A couple of the early ones we delivered had a few configuration errors or were missing something. We’ve corrected it in the work instructions, and we seem to be batting one hundred percent delivering correct systems for the past two days.”
“Great work, Patty!” I say, excitedly. “Go ahead and start publishing the schedule. I want to start showing this off!”
Chapter 23
• Tuesday, October 7
As I drive into work the following Tuesday morning, I get an urgent phone call from Kirsten. Apparently, Brent is now almost a week late delivering on another Phoenix task—allegedly something that Brent said would only take an hour to do. Once again, the entire Phoenix testing schedule is in jeopardy.
On top of that, several other of my group’s critical tasks are late, putting even more pressure on the deadline. This is genuinely dispiriting to hear. I thought all our recent breakthroughs would solve these due-date performance issues.
How can we unfreeze more work if we can’t even keep up now?
I leave Patty a voicemail. To my surprise, it takes her three hours to call me back. She tells me that something is going terribly wrong with our scheduling estimates and that we need to meet right away.
Once again, I’m in a conference room, with Patty at the whiteboard, and Wes scrutinizing the printouts she’s taped up.
“Here’s what I’ve learned so far,” Patty says, pointing at one of the sheets of paper. “The task that Kirsten called about is delivering a test environment to QA. As she said, Brent estimated that it would take only forty-five minutes.”
“Sounds about right,” Wes says. “You just need to create a new virtualized server and then install the OS and a couple of packages on it. He probably even doubled the time estimate to be safe.”
“That’s what I thought, too,” Patty said, but she’s shaking her head. “Except it’s not just one task. What Brent signed up for is more like a small project—there’s over twenty steps involving at least six different teams! You need the OS and all the software packages, license keys, dedicated IP address, special user accounts set up, mount points configured, and then you need the IP addresses to be added to an ACL list on some file server. In this particular case, the requirements say that we need a physical server, so we also need a router port, cabling, and a server rack where we have enough space.”
“Oh, for chrissakes…,” Wes says, sounding exasperated, reading what Patty is pointing at. He mumbles, “Physical servers are such a pain in the ass.”
“You’re missing the point. This would still be happening, even if it were virtualized,” Patty says. “First, Brent’s ‘task’ turns out to be considerably more than just a task. Second, we’re finding that it’s multiple tasks spanning multiple people, each of whom have their own urgent work to do. We’re losing days at each handoff. At this rate, without some dramatic intervention, it’ll be weeks before QA gets what they need.”
“At least we don’t need a firewall change,” Wes says, snidely. “Last time we needed one of those, it took John’s group almost a month. Four weeks for a thirty-second change!”
I nod, knowing exactly what Wes is referring to. The lead time for firewall changes has become legendary.
Wait. Didn’t Erik mention something like this? For a firewall change, even though the work only required thirty seconds of touch time, it still took four weeks of clock time.
That’s just a microcosm of what’s happening with Brent. But what’s happening to us right now is much, much worse, because there are handoffs.
With a groan, I put my head on the conference table.
“You okay?” Patty asks.
“Give me a second,” I say. I walk up to the whiteboard and struggle to draw a graph with one of the markers. After a couple of tries, I end up with a graph that looks like this:
I tell them what Erik told me at MRP-8, about how wait times depend upon resource utilization. “The wait time is the ‘percentage of time busy’ divided by the ‘percentage of time idle.’ In other words, if a resource is fifty percent busy, then it’s fifty percent idle. The wait time is fifty percent divided by fifty percent, so one unit of time. Let’s call it one hour. So, on average, our task would wait in the queue for one hour before it gets worked.
“On the other hand, if a resource is ninety percent busy, the wait time is ‘ninety percent divided by ten percent’, or nine hours. In other words, our task would wait in queue nine times longer than if the resource were fifty percent idle.”
I conclude, “So, for the Phoenix task, assuming we have seven handoffs, and that each of those resources is busy ninety percent of the time, the tasks would spend in queue a total of nine hours times the seven steps…”
“What? Sixty-three hours, just in queue time?” Wes says, incredulously. “That’s impossible!”
Patty says with a smirk, “Oh, of course. Because it’s only thirty seconds of typing, right?”
“Oh, shit,” Wes says, staring at the graph.
Suddenly, I recall my conversation with Wes right before Sarah and Chris decided to deploy Phoenix at Kirsten’s meeting. Wes complained about tickets related to Phoenix bouncing around for weeks, which delayed the deployment.
It was happening then, too. That wasn’t a handoff between IT Operations people. That was a handoff between the Development and IT Operations organization, which is far more complex.
Creating and prioritizing work inside a department is hard. Managing work among departments must be at least ten times more difficult.
Patty says, “What that graph says is that everyone needs idle time, or slack time. If no one has slack time, WIP gets stuck in the system. Or more specifically, stuck in queues, just waiting.”
As we digest this, Patty continues. “Each of those sheets of paper on the board is like this Phoenix ‘task,’” she says, making air quotes with her hands. “It looks like a single person task, but it’s not. It’s actually multiple steps with multiple handoffs among multiple people. No wonder Kirsten’s project estimates are off.
“We need to correct this on Kirsten’s schedule and her work breakdown structure, or WBS. Based on what I’ve seen, fully one-third of our commitments to Kirsten fall into this category.”
“Just great,” Wes says. “It’s like Gilligan’s Island. We keep sending people off on three-hour tours, and months later, we wonder why none of them come back.”
Patty says, “I wonder if we could create a kanban lane for each of these ‘tasks?’”
“Yes, that’s it,” I say. “Erik was right. You’ve just found a big pile of recurring work! If we can document and standardize this recurring work, and gain some mastery over it, just like you did with laptop replacement, I’m sure we can improve flow!”
I add, “You know, if we can standardize all our recurring deployment work, we’ll finally be able to enforce uniformity of our production configurations. That would be our infrastructure snowflake problem—you know—no two alike. How Brent turned into Brent is that we allowed him to build infrastructure only he can understand. We can’t let that happen again.”
“Good point,” Wes grunts. “You know, it’s odd. So many of these problems we’ve been facing are caused by decisions we made. We have met the enemy. And he is us.”
Patty says, “You know, deployments are like final assembly
in a manufacturing plant. Every flow of work goes through it, and you can’t ship the product without it. Suddenly, I know exactly what the kanban should look like.”
Over the next forty-five minutes, we create our plan. Patty is going to work with Wes’ team to assemble the top twenty most frequently recurring tasks.
She will also figure out how to better manage and control tasks when they are queued. Patty proposes a new role, a combination of a project manager and expediter. Instead of day-by-day oversight, they would provide minute-by-minute control. She says, “We need fast and effective handoffs of any completed work to the next work center. If necessary, this person will wait at the work center until the work is completed and carry to the next work center. We’ll never let critical work get lost in a pile of tickets again.”
“What? Someone assigned to carry around tasks from person to person, like a waiter?” Wes asks in disbelief.
“At MRP-8, they have a ‘water spider’ role that does exactly that,” she counters. “Almost all of this latest Phoenix delay was due to tasks waiting in queues or handoffs. This will make sure it doesn’t happen again.
“Eventually,” she adds, “I’ll want to move all the kanbans, so that we don’t need a person acting as the signaling mechanism for work handoffs. Don’t worry. I’ll have it figured out in a couple of days.”
Wes and I don’t dare doubt her.
Chapter 24
• Saturday, October 11
The following Saturday was relatively peaceful. In fact, it’s the most relaxing weekend my family has had since I took my new job. With Halloween a few weeks away, Paige insisted that we take the whole family out to the pumpkin patch.
It was a chilly Saturday morning, so we were exhausted just bundling up the kids and getting them in the car. When we arrived at the nearby farm, Paige and I laughed uncontrollably at Parker, who looked like a giant, angry sausage stuffed into his blue parka. She couldn’t resist taking pictures while Grant orbited us in excitement, taking pictures with his own camera.
Afterward, we went to a local microbrewery, enjoying lunch on the patio in the warm afternoon sun.
“I’m so glad we could do this,” Paige says. “It’s really nice. You seem less stressed lately. I can really tell things are getting better.”
She’s right. It felt like we had turned a corner at work somehow. Just like how I wasn’t wasting as much time fighting my ancient laptop, it seemed like my team was spending more and more of our time doing productive work and less and less firefighting.
Although I know getting a new laptop has absolutely nothing to do with our organizational performance, getting rid of that old clunker was like getting rid of a thousand-pound anchor that someone had tied around my neck while I was swimming across the ocean.
We’re still grappling with gradually melting the project freeze. I’m guessing that we’ll probably be able to unfreeze twenty-five percent of all the projects, along with a bunch of other new projects designed to further elevate Brent.
There were still so many uncertainties. But unlike before, our challenges feel within our ability to understand and conquer. Our goals finally seem achievable. I no longer feel like I am always on my heels, with more and more people piling on, trying to push me over.
With the business agreeing, with the exception of Sarah, on what the priorities are, my job actually seems fair. It feels like we have the initiative and are attacking the problem, instead of the other way around.
I like it.
I look up to see Paige smiling back at me and then yell out in panic as I see Parker knocking over her glass of beer.
The rest of the afternoon passes too quickly, but it’s one of the best days I’ve had all year.
* * *
Later that evening, Paige is curled up with me on the couch. We’re watching the Clint Eastwood movie Pale Rider. The kids have gone to sleep, and this is the first time we’ve actually watched a movie together in months.
I laugh uncontrollably watching the main character, “the Preacher” played by Eastwood, methodically pick off the gang of nine evil deputies. Paige looks at me with amused disapproval.
“What exactly is so funny about this?” she asks.
This makes me laugh harder. When another deputy gets shot in the background, I say, “Look at that! You know what’s going to happen, but the marshal just stands there in the middle of the street, watching the carnage! Look at the way the wind rustles through his coat! And his gun isn’t even drawn! I love it!”
“I’ll never understand you,” Paige says, shaking her head with a smile.
Just then, my cell phone rings. I instinctively reach for it.
Holy crap. It’s John. No one has seen or heard from him since that audit meeting, over two weeks ago. We’re pretty sure he hasn’t been fired, but no one knows more than that. I’ve been meaning to check the local hospitals to make sure he’s not convalescing alone somewhere.
As much as I want to talk with him, I don’t want to leave Paige and the movie. I look at the clock, and see that there’s probably only fifteen more minutes until the end. Not wanting to miss the final gunfight, I mute the phone. I’ll call him back when the movie is over.
A couple of seconds later, my phone rings again, and I again hit the mute button.
My phone rings again. For the third time, I mute the phone, but quickly send him a text message: Grt hearing from u. Can’t talk right now. Will call u in 20m.
Unbelievably, my phone buzzes again, so I turn the ringer off, putting my phone underneath some cushions on the couch.
Paige asks, “Who keeps calling?”
When I say, “John,” she rolls her eyes, and we watch the remaining ten minutes of the movie.
“I can’t believe I haven’t seen that movie until tonight!” I say, giving Paige a squeeze. “That was such a great idea, darling!”
“It’s been a great day. It’s so nice having a normal life again,” she says, returning my hug, and then with a smile, she gets up, taking the empty beer bottles with her.
I agree. I pick up my phone, my heart skipping a beat when I see “15 missed calls.”
Suddenly afraid that I may have missed something truly disastrous, I quickly look to see who called. Every call was from John. I call him immediately.
“Billy, so good to hear your voice again—my friend—my dear, dear old—dear ol’ friend,” he slurs. Good Lord. He’s completely drunk.
“Sorry I couldn’t return your calls right away. I was out with Paige,” I say, feeling guilty at my slight mistruth.
“No problem. Look, I just wanted to see you one—one lasht time before I left—be leaving,” he says.
“Leaving? What do you mean ‘leaving’? Where are you going?” I say with alarm, wondering how long he’s been drinking. Maybe I should have called back sooner. I suddenly visualize him on the other end of the phone with an open bottle of sleeping pills in his hand, already half-empty.
I hear him laugh, maybe a bit hysterically. “Don’t worry, Billy. I’m not going to kill myself. Haven’t had enough to drink—yet. Har-har! I just wanted to see you before I left town tonight. Let me buy you one last drink.”
“Uh, can’t this wait until tomorrow? It’s almost midnight,” I say, slightly relieved.
He tells me that he’ll be long gone by tomorrow and convinces me to join him at the Hammerhead Saloon downtown.
When I pull into the parking lot, I immediately spot John’s Volvo station wagon. Hitched behind his car is a U-Haul trailer, and there’s a pile of empty beer cans right outside the driver’s side door.
I find him in a booth at the back of the crowded bar, and it’s obvious he’s been here all day. He doesn’t seem to have bathed or changed his clothes since I last saw him. His hair is greasy and wildly askew as if he just woke up, his face is covered in stubble, and there are food stains on his shirt. His keys and wallet have been tossed c
arelessly next to the salt and pepper shakers.
John urgently waves over a waitress, taking a moment to rehearse his words, but still slurring as he says, “I’d like two double scotches, neat, for me and my friend here. And those yummy nachos… Please.”
She looks over at me questioningly, obviously having already served him plenty. I nod, but say quietly, “Let’s start with two cups of coffee, please. I’ll take care of him.” As I say it, I reach over and take his keys off the table.
For a moment, she looks dubious but gives me a small smile and walks away.
“Dude, you look like shit,” I say candidly.
“Thanks, pal. So do you,” he replies, before bursting out laughing.
“Nice. Where the hell have you been? Everyone has been looking for you,” I say.
“I’ve been at home,” he says, grabbing some popcorn from our table. “I’ve been mostly reading and watching TV. Whoa, there is some crazy shit on TV these days. Crazy! But then I started thinking it’s time for me to move on, so I spent most of today packing. I just wanted to ask you one little question before I left.”
“You mentioned that on the phone,” I say, as the waitress arrives with two cups of coffee and the nachos. John looks with confusion at the mugs on the table, so I say, “Don’t worry. Our drinks are on the way.”
When I get him to take a sip of coffee, he asks, “Just tell me straight. Is it really true that I haven’t done anything of value for you? In all the three years that we’ve worked together, I’ve never, ever been helpful?”
I take a deep breath, trying to decide what to tell him. A friend told me years ago, “To tell the truth is an act of love. To withhold the truth is an act of hate. Or worse, apathy.”
I had laughed at those words at the time, but over the years, I’ve realized that having people give you honest feedback is a rare gift. Looking across at John, even though he looks like a completely broken man, I wonder whether the right thing to do is to just let him off easy and tell him what he wants to hear.