by Gene Kim
Steve stares at the door for several moments before he lets out a long sigh. He turns to the rest of the room. “Unlike Sarah, my bets are on the growth thesis. If we’re not growing, we’re shrinking, and that’s not why I’m here at Parts Unlimited. There’s no question that we need to make Horizon 3 bets. Dick and I have already modeled this scenario. I propose carving out $5 million to fund Maggie’s innovation efforts,” he says.
Maxine’s heart leaps, realizing that Maggie’s proposal may still have a chance. Then Steve says, “But Sarah might be able to cut us off at the knees if she convinces Bob to cut staff and expenses.”
“Maybe Bob and the rest of the board won’t be that short-sighted,” Dick replies. At Steve’s skeptical expression, he says, “Well, if it happens, we’ll figure something out. In the meantime, what would we do with that five million? How do we put it to work?”
Over the next three exciting hours, an outline of a plan emerges. Kurt, Maggie, and Maxine will be part of the Innovation team, reporting directly to Bill Palmer. Kurt and Maggie will be “two in a box,” occupying the same position in the org chart, sharing responsibility for the business and technical outcomes.
“You will be responsible for nurturing new, promising ideas and exploring market risk, technical risk, and business model risk,” Steve continues to summarize. “Each initiative will have clearly defined business outcome measures, such as customer acquisition, repeat customer usage, or customer satisfaction. At the end of each quarter, we’ll review progress on each initiative, and we’ll make a decision: continue funding the project; kill it, reassigning the team to the next best idea; double-down on the project; or graduate it to Horizon 2. We’ll also decide whether the entire program needs to be grown or shrunk.
“Your charter is to find new ways to use technology to better enable supporting the nearly century-old vision: help our customers keep their cars running so they can conduct their daily lives”
Steve turns to Bill and says, “To capture the best ideas, go create an Innovation Council. Find fifty of the most respected people from across the entire organization, including store managers, sales managers, technicians, engineers, and of course, the technology organization.”
Bill nods, taking notes on his clipboard. Maxine sees everyone around the table nodding in agreement.
As they adjourn, Steve says to everyone, “I have the utmost confidence in Bill, who will be leading this effort. This is something that we’ve never done before, so it will be new to almost everyone. So, let us know what we can do to support you and your effort.”
“Roger that,” Bill says. Gesturing at his clipboard, he says, “I already have a list of things I’d like your help on.” As Bill rattles them off, Steve quickly mobilizes resources from across the entire company. Maxine marvels at how efficiently Bill gets what he needs from Steve, and the resources Steve is putting at Bill’s disposal.
After the meeting, Bill asks Kurt, Maggie, and Maxine to walk with him back to Building 5. The entire walk back, Maxine hears Kurt and Maggie share how elated they are at how the meeting went. Maxine is excited to be working with Bill, who has always seemed level-headed and a straight-shooter, and he’s already demonstrated how effectively he can drive large initiatives.
To Maxine, it seems like Rebellion victory is within sight, with all the sponsorship, funding, support, and energy they need to win.
But there’s still the question of Sarah. If she succeeds in convincing Bob Strauss to break up and sell the company, then the Evil Empire will have won. But not even Sarah can pull that off, right?
CHAPTER 17
• Friday, December 12
“What do you mean we’re losing all our people?” Kurt says, looking shocked.
It’s one week after the meeting with Steve and Dick, and Maxine has been pleased with how the agreed upon plan is rapidly progressing. Bill, Maggie, and Kurt are starting to put together the Innovation Council, and work within the Unicorn Project is going faster than ever, preparing for the massive Christmas holiday promotion launch.
The Orca team continues to study the data from the Thanksgiving campaign, and they are certain they’ll have an even higher response rate this time around by incorporating what they’ve learned. All their experiments and results are again being poured back into the Panther data platform. They are also continuing to harden the infrastructure to handle the resulting onslaught.
However, many of the things they took for granted after last Friday’s exciting meeting with Steve and Dick no longer seem certain. Which is why Chris summoned Kurt and Maxine to his office.
“They were never ‘your people,’ Kurt. You were temporarily loaned a bunch of engineers for the Unicorn Project,” says Chris. “All those engineers were already assigned to other projects in the new fiscal year, which starts in a few weeks. They’re all important business projects with people depending on them being fully staffed. All those business managers are raising hell because we’ve reassigned those resources, and they’ve banded together in revolt.”
“But why now?” Kurt asks, incredulous. “What got everyone so riled up?”
Chris laughs humorlessly. “Sarah is stirring the pot, egging them all on. Bill is pulling together another meeting with Steve and Dick to figure out how to deal with the shenanigans she’s pulling.”
“I can’t believe Sarah is stirring up a counter-rebellion, to umm, counter our Rebellion,” Kurt mutters, sounding offended that Sarah had stolen his playbook.
Later in the day, Bill writes:
Continue as planned. We’ll figure out how to backfill those positions. We are stuck in the middle of a huge political battle. Sarah and a board faction are on one side, and we’re on the other side with Steve and Dick.
Throughout the day, they discover that Sarah is indeed an incredibly effective corporate guerilla fighter, having successfully raised an insurgency army against them over the past week.
Maxine is grudgingly impressed with her resourcefulness, despite driving her batshit crazy. She wants nothing more than for Sarah to just give up and go away.
“In so many ways, Sarah is a remarkable person,” she tells her husband at dinner. “In a slightly different universe, Sarah could have been an amazing force for good. If this were a superhero movie, she’d be the gifted person who turns into the villain after some traumatic life event. And now she goes out of her way to crush all sparks of joy she can find.”
Monday morning, Kurt and Maggie meet with Bill to stave off Sarah’s efforts to undermine them. Maxine stays behind, resuming her work with Cranky Dave on a technical issue jeopardizing both the Unicorn promotions campaign and the core Phoenix application. Over the last month, they had started building tons of automated tests for Phoenix so that they could better and more safely make changes. The effort was incredibly successful. However, with so many tests, running them now takes hours, and developers are starting to avoid checking in their changes, not wanting to wait for the long test times.
Worse, some of the automated tests were failing intermittently. Last week, she cringed watching as a developer, whose tests failed, just ran them again, and they also failed. So, he ran it a third time, as if it were slot machine in a casino. This time it passed. This is no way to run a development shop, Maxine thought with embarrassment and distaste.
Recognizing that this will soon become a new bottleneck for developers, she had the teams work on parallelizing the Phoenix tests so they could be run across multiple servers. But they discovered that running parallel tests caused Phoenix to occasionally deadlock or crash entirely—and if it’s crashing during testing, it’s probably crashing in production too.
“Maxine, we’ve narrowed it down to an uncaught exception somewhere in the Phoenix order fulfillment module,” says Cranky Dave. Maxine is with Cranky Dave and another engineer with laptops open. When she pulls up the code on her laptop, she physically recoils. “Wow,” she says, speechless as she scrolls down the file … and scrolls … and scrolls …
<
br /> “Yeah,” says Cranky Dave, laughing. “This is two thousand lines of code to determine whether we can ship to the order location. A bunch of architects made this framework fifteen years ago, predating Phoenix. Even the TEP-LARB came around and realized this was all a terrible mistake, but the people who wrote that framework are long gone.”
Maxine keeps scrolling and scrolling, gob smacked that she can’t find any business logic, just boilerplate code: loops iterating through orders, order items, line items, just as dangerously as the middle school girls had done so many months ago. There’s null checking everywhere, as well as type tests, downcasting, coercing, and all sorts of horrible contortions to get at the desired data, through enumerated types or polymorphic supertypes without concrete subtypes. There’s so many object methods that she can’t keep them straight in her head: getOrderLines, getItemLines, getShippingLines …
She opens her mouth, but nothing comes out. “This is … incredible,” she says at last, with apoplectic horror and disbelief. She closes her eyes, trying to summon some relentless optimism, thinking of the Hoare principle: “There are two ways to write code: write code so simple there are obviously no bugs in it, or write code so complex that there are no obvious bugs in it.”
“Gentlemen, we are going to clean up all this crap,” Maxine says, with a level of confidence that she realizes may be foolhardy. Even Cranky Dave looks cowed. She exhorts, “This should be simple code. All we need to do is retrieve the locations from the order, right? We can do this!”
They spend two hours writing tests around the code to make sure they really understand how it works, and then they start pulling out common operations, putting them where they belong. Maxine modifies the class hierarchy to their best advantage, but adheres to functional programming principles, using modern types and their idiomatic map, reduce, and filter functions, just like the famous Google Map/Reduce paper that inspired Shannon’s Panther project.
By noon, they’ve reduced the two thousand lines of code to five hundred lines. Cranky Dave grins. “That is amazing, Maxine. This is probably the first time in over five years that anyone was brave enough to touch this code.”
“Eight years,” says the other developer. “This code is beautiful! And I think I found the problem. Here’s some code that isn’t wrapped in a try/catch block.”
Looking over at his laptop, Maxine immediately knows they’ve found the issue. “Nice work!” Now that they’ve drained the swamp of the muck, the problem is obvious.
While they go get lunch, Maxine stays behind, wanting to try an idea out. She opens up a new window on her laptop. She copies the data that the team has been manipulating all morning and starts redoing the code from scratch in Clojure.
Forty-five minutes later, Cranky Dave is back and hands Maxine her sandwich. He asks, “What are you grinning at?”
“Oh, just the results of a little experiment,” she says. “I rewrote our code in a functional programming language, using its built-in data types and standard library, to see if I could make it even simpler, smaller, and get rid of the need for exception handling.”
“And?” prompts Cranky Dave. She turns her laptop around to show him.
“Holy shit,” he says, staring at her screen in disbelief. “Fifty lines of code.”
Maxine laughs, knowing that they’ll be inspired to try to match or exceed her results. Even for her, this was an incredible display of achieving the First Ideal of Locality and Simplicity.
The work they did this morning will enable parallelized testing and make it blazingly fast, surely creating huge productivity dividends long in the future, allowing developers to keep moving fast, getting even faster feedback on any errors. It’s like the opposite of technical debt. It’s like when compounded interest works in your favor. If they could make developers a little more productive all the time, it would always pay off in spades.
Maxine smiles as she sees Cranky Dave open his laptop, still giddy with excitement over their successes. He says, “Uh oh.”
On his screen is the Unikitty CI status page. Maxine looks to see whether the fix that they checked in before lunch passed the automated tests. But instead of getting all green lights, she sees that the tests aren’t running at all. There are over fifty jobs in front of them, all waiting to start.
“This is bad,” Cranky Dave says. “The entire Unikitty CI cluster is down. Everyone’s builds are stuck.”
Annoyed, Maxine looks over at his screen. She curses. This is ruining what is supposed to be their shining moment of triumph and glory.
He says, “The #ci-unikitty channel is going nuts. No one can run their tests.”
Whenever Unikitty goes down, they have a bunch of angry customers: their fellow developers. What better proof that Unikitty is an internal platform that they must manage like a product, not just as a project. It’s never done, if they want to keep their customers happy.
They look all over for the Unikitty team. They find Dwayne, Kurt, and two other engineers in a conference room, crowded around Brent’s laptop.
“Good timing. All the Dev managers are screaming that their teams can’t get their work done,” Kurt says, looking up from the huddle. Maxine is surprised at how haggard he looks. He has bags underneath his eyes. Kurt’s having a rough couple of weeks, she thinks. “We can’t afford this distraction right now, of all times …”
“This is what we always wanted, right? Customers!” Maxine says with a big smile. “You wanted people to value the infrastructure we’re creating? Well, your wish has been granted. After all, if they didn’t care, they wouldn’t even bother complaining.”
The adoption of CI practices has been astounding, with nearly one third of all engineering teams using Unikitty in their daily work. But they’re having problems scaling it to keep up with demand.
She looks at her watch. It’s almost noon. Every developer tended to check in their code before heading out to lunch, which probably caused something in Unikitty to fall over. Unikitty is having more than its share of growing pains, Maxine thinks.
Kurt sighs. “If you look in the chat room, many of the Dev managers are saying that they’re tired of our flaky build servers and that they’re pulling their teams out of Unikitty. They’re going to go back to doing builds the old way.”
Maxine’s smile freezes on her face. “You’re kidding.” Going back to the bad old days like her first days on the Phoenix Project would be … intolerable. That wouldn’t be a setback—that would be a genuine disaster.
What was happening? It felt like all the progress they had made with the Phoenix developers and with the achievements of the Black Friday launch and the Unicorn Project were slipping away. They were slowly being pulled back into the morass, dragging all the engineers that they had liberated and made productive back in with them.
It’s almost four thirty when the Unikitty team finally gets things back up. But there are so many builds and tests backed up, it will be almost midnight before all the jobs finish running.
“I can’t believe it was a networking switch failure,” Dwayne says.
Maxine shakes her head in disbelief. Another Unikitty hardware failure is embarrassing. From the beginning, it had been cobbled together from whatever equipment Kurt and the team were able to scavenge from almost every corner of the organization.
They’ve had disk failures, power failures, and now network hardware problems. She hates seeing highly skilled engineers walking around with screwdrivers, opening up server cases, and mucking about with physical infrastructure.
Sure, she has many fond memories of working on hardware, both in her career and with her kids. When she was a young engineer, she had loved opening those huge boxes full of the newest, hottest equipment on the loading dock, and then racking and stacking them. But back then, she also loved rotating the backup tapes.
Now, this type of work seems so low-value, especially when compared to the opportunity cost of the work they should be doing, which is figuring out how to blaze the d
igital future of Parts Unlimited.
Their job was to build code, not muck with the actual hardware that the code runs on.
“I hate to say it, but I think Unikitty is on her last legs,” Maxine says to Dwayne. “We can’t keep running something this mission-critical with hardware that Brent finds under his desk. And it’s not just hardware issues—each build server is still slightly different. Like when my compile job doesn’t run on Build Server #3, it takes ten times as long. We’re spending too much time keeping it running. We’ve got to do something about this—soon.”
“No argument from me. We’re all just so busy right now,” Dwayne says, shrugging his shoulders. Maxine can’t argue with that.
As was predicted, there is hell to pay. Tuesday morning during Kurt’s team meeting, Kurt says, “Chris not only chewed me out in front of everyone at his staff meeting, but he invited Rick too. He presented his plan to create a competitive CI service to compete with Unikitty.”
“Rick?!” Dwayne asks, expressing the complete shock and disbelief that Maxine feels. “He wouldn’t recognize a CI service if it bit him in the ass!”
Kurt slumps. “Sarah has apparently started shouting from the rooftops that Unikitty is jeopardizing the entire company and that we should be shut down.”
Silence falls over the entire table.
“Amazing how people are blaming us for anything bad that happens around here. The toilets broke on the second floor yesterday, and we’re being be blamed for that too,” says Cranky Dave.
Kurt’s phone buzzes. He picks it up and stares at the screen for several moments. He looks at Maxine. “We gotta go. Bill just pulled together a meeting with Maggie. More bad news, I think.”
Maggie sees Bill look up when his assistant Ellen lets her, Kurt, and Maggie into his office. “We’ve got problems,” he says, standing up and picking up his clipboard. “We’re meeting Steve and Dick in fifteen minutes in Building 2. I’ll brief you on our walk over.”