The Unicorn Project

by Gene Kim

  “Can we just do it ourselves over the weekend?” Dwayne says. “It wouldn’t affect anyone outside the team. We could just go to an office supply store or furniture store, buy the bare minimum, and move it into the building. We can use my truck.”

  “But what happens when Facilities shows up with their badges and says that we don’t have the right permits or that we’re out of code?” Cranky Dave asks.

  Kurt bursts out laughing, “Facilities isn’t going to haul it away themselves because no one will give them the budget.” He thinks for a moment. “Let’s do it. But let’s make sure to bring in some furniture that can’t be carried away easily … like a couple of bookshelves, and we’ll fill them up with books. Maybe a goldfish tank. What do you think?”

  Cranky Dave and Shannon laugh. Adam nods thoughtfully, “Possession is nine-tenths of the law. But shouldn’t you get Chris’s go-ahead first?”

  Kurt snorts. “No way. He’d never go for it. Let’s just do it.”

  “Since we have limited space, how about we move some of the QA people to the Dev area, and move some of the developers to the QA area?” Shannon suggests.

  “Great idea, Shannon,” Maxine says. She’s delighted that the team is organizing itself, just as Erik predicted.

  CHAPTER 12

  • Monday, October 13

  Over the past week, it’s clear to Maxine that Data Hub has figured out how to deliver better value, sooner, safer, and even happier. But it’s also clear that a new constraint has emerged. The constraint used to be getting environments—no one could ever get one, and when they did, it was never quite right. Then the constraint became testing, which started only when Development was finished with all their features; finding and fixing defects would take weeks instead of the hours or days it takes now.

  Now it is obvious that the constraint is deployment—they are now able to quickly get features production-ready, but they still have to wait weeks for Ops to deploy their code into production.

  Figuring out how to get Data Hub into production more quickly is no longer an academic concern. Tom is standing in the front of the conference room with the rest of the Data Hub team. He says, “Maxine, the suspicions that you had while you were out sick were right on target. According to Maggie and all her product owners, creating effective promotional bundles is one of the most critical and urgent Phoenix priorities.

  “Kurt, the meeting we have with Maggie is scheduled for tomorrow, and you asked me to study up on this beforehand,” Tom continues. “Here’s what I’ve learned: Marketing is constantly experimenting with promotion campaigns to accelerate sales, and this is incredibly important as we approach the holidays, our peak sales season. For example, now that it is snowing in many areas, they want to create a winter promotion bundle: tire chains, ice-melting salts, and window scrapers. They also need to create a discounted price for that bundle, say twenty percent off. They also do promotions to customer segments—if you buy lots of windshield wipers, you may get offered a bundle of wiper fluid and glass defoggers, knowing that you may only need the smallest nudge to buy.

  “Conceptually, it sounds easy. But here’s all the insane steps they need to go through to get this done: First, every new product bundle needs a new SKU, just like every other item we sell. These SKUs are used by almost every application in the business: inventory tracking, in our supply chain, our in-store registers, our e-commerce sites, even the mobile apps …

  “We only create new SKUs in large batches every six weeks. After the SKUs are created, we also need to push all the application and business logic changes for these new SKUs. These are pushed to every application that needs to know about them. That’s often scores of back-end and front-end applications across the enterprise. You might have seen these go out at eight on Friday nights. And when that’s done, sometimes we even need to manually refresh certain production databases.

  “Here’s problem number one: we only create new SKUs every six weeks, which is way too slow. Thanksgiving is a month away, and we’re already in danger of not getting those product bundle SKUs created in time.

  “And the real truth is that it often takes us much longer than six weeks. We need to change so many applications during those pushes that if anything goes wrong during testing, the entire release is canceled … You can’t have new SKUs out there when some of the applications don’t know how to deal with them. There’s just not enough time to fix these things during the test period, so it’s all-or-nothing.

  “And on top of that, Promotions also needs to rapidly experiment and iterate to discover which bundles customers are actually interested in and what specific factors lead to an actual purchase. Right now, iterating only once every six weeks is not fast enough to learn and adapt—our e-commerce competitors are doing multiple experiments per day,” Tom concludes.

  “Wow, that’s really incredible,” Maxine says, looking at all the boxes he’s drawn on the whiteboard. “This is so much like the Phoenix architecture, which makes it so difficult for any team to independently develop, test, and deploy value to our customers. The architecture that supports the Promotions value stream that you’ve just drawn on the whiteboard shows how it’s almost impossible to move any work quickly to where it needs to go.”

  She gestures at his diagram. “At every step, it’s entangled with so many other value streams. We have to synchronize with everyone else’s release schedules. If any of them can’t be released, then we can’t be released … It’s just crazy.”

  “It really is. It’s frustrating that Data Hub is so tightly coupled to Phoenix and the BWOS,” Tom says.

  “What’s a BWOS?” Maxine asks.

  “Oh, that’s what we call the … you know, the big wad of … umm, crap. You know, the hundred-plus applications we connect to,” Tom says.

  Maxine laughs. “I really think if we could deploy Data Hub changes into production on demand and fully decouple them from the Phoenix release schedule, we’d be so much better off … That way, if we have to cancel a release, we could at least try again the next day. With some practice, I bet we could get SKU creation down to one or two days.”

  “I definitely agree,” Tom says. Maxine smiles, satisfied that they’re on the right track. And the value of doing this will be huge, she thinks.

  “This may not be related, but I think it’s worth mentioning,” Tom says. “We have other huge problems being connected so closely to Phoenix. It sometimes sends us tons of messages that hammer the back-end systems that we connect to. We routinely see massive waves of transactions that cause huge reliability and throughput problems, and sometimes even data integrity problems. Sometimes it’s Data Hub crashing, but most of the time, it’s the systems that we’re calling that are the ones crashing.”

  Cranky Dave piles on. “Dealing with those systems of record are a huge a pain in the ass. We don’t have any real API strategy around here. No one knows what APIs are available, and even if you do, no one knows how you get access to them or deal with their crazy authentication or pagination schemes. Everyone’s documentation is crap, and some of these teams don’t even care if their APIs don’t work as advertised.

  “And once you do get someone’s API working, they’ll break it however and whenever they want, especially since they probably don’t version their APIs. So transactions start failing for our customers, and they blame us,” he continues. “They never give you all the data you need, so when you actually have an API change you want, you have to go through all these committees to get them approved!”

  “It’s enough to drive anyone crazy,” Cranky Dave says, exhausted.

  “We can definitely stop this madness,” Maxine says with certainty.

  As promised, the next day, Kurt, Maxine, and the Data Hub team meet with Maggie. As usual, Kurt introduces all the Data Hub team members to Maggie and then asks Maggie to introduce herself.

  “Many of you already know me,” she says with a smile. “My name is Maggie Lee. I’m senior director of retail program managem
ent. What that really means is that I have the P&L responsibilities of all the products and programs behind our stores, which includes physical stores, e-commerce, and mobile. My group of product managers own strategy, understanding the customer and market, customer segmentation, identifying which customer problems we want to solve, pricing and packaging, and managing the profitability of everything in our portfolio.”

  She continues, “We bridge the business goals and everything that’s required to actually achieve them. That includes business operations, business analysts, and product managers, who work with Chris’ technical teams. I also have all the operational pieces required to deal with Sales, Finance, and Operations on my team.

  “When Kurt said that you had some ideas on speeding up how we create promotion product bundles, you certainly got my attention,” she says. “Sorry I couldn’t meet even earlier, but as you can imagine, we’re all buried with a million things right now. It’s definitely a make-or-break quarter for us. For all of us.”

  Maxine is already impressed. Maggie is in her mid-forties and has an unmistakable intensity about her. She is the same height as Maxine and obviously competent. She’s a no-nonsense type and always has a serious expression on her face. Maxine suspects that she’s Sarah’s forebrain, handling the million things required to keep a billion-dollar retail operation going.

  Kurt explains what they’ve been working on.

  Maggie looks at Kurt. “So you’re telling me that you could enable Marketing to create promotions entirely self-service, like our e-commerce competitors can? And that other changes could potentially be pushed into production on the same day?” Maggie says. “Holy shit, folks. If you can actually do what you say, this might be the miracle we’ve been hoping for. I’m not prone to overstatements, but I’m not kidding when I say that this could potentially save the quarter. And maybe even the company.”

  Maxine smiles. “From everything we’ve studied, it’s clear that it’s way too difficult and takes too long to get those promotional bundles created. We’d love to fully empower your teams to do what it takes to create new promotions anytime you want and have them pushed out to all your sales channels within hours. There’s a lot we don’t understand, but conceptually, we should be able to do it. We just wanted to explore whether this would be valuable to you.”

  Maggie nods. “Hugely valuable. Look, Steve has promised all the analysts that this holiday season we’re finally going to see an uptick in revenue. This is after years of over-promising and under-delivering. Everything hinges on Promotions being able to move the needle on sales. If you really think you can make this happen, we’ll do whatever it takes. What exactly is in the way?” she asks.

  “Who isn’t in the way?” Kurt laughs. “We’re meeting with Information Security tomorrow, who could kill this effort on a whim. But the real threat is the TEP-LARB. We’ve put together a team to create our proposal, but people usually wait six to nine months to get in front of them,” Kurt says. “Unless, of course, there’s an urgent business need with a powerful sponsor.”

  Maggie finally smiles, in not an entirely kind way. “For this, I think we need to bring in the big guns.”

  “Who’s that?” asks Maxine, curious who could possibly be a more powerful sponsor than Maggie.

  Maggie grins. “Sarah. Take it from me, there is no one more effective at busting down inconvenient barriers than she is. She’s like a chainsaw, great at cutting down trees.”

  “… and sawing off hands,” Kurt mutters under his breath.

  The next day, Kurt and Maxine meet with Ron, the security manager that Shannon introduced them to. They walk into the conference room and see that Shannon has arrived early.

  “There’s no way I’d miss this,” she says, smiling. “I should have brought popcorn.”

  Ron, who is in his mid-thirties, comes in and sits down. After introductions, Kurt walks him through their idea to decouple Data Hub from the rest of Phoenix.

  Ron says, “Interesting idea. I remember when Data Hub was still called Octopus. Why the need for such a big change? It seems to be working well enough now.”

  Kurt walks through all the reasons, and to Maxine’s surprise, Ron nods agreeably. This is going better than I thought it would, Maxine thinks.

  “That’s exciting,” he says, agreeably. But then he takes off his glasses and puts them on the table. “Look, I really want to help, but I can’t. I’m responsible for making sure applications in my portfolio meet all applicable laws and regulations and that all those applications are secure. Given how radical of a change you’re making, I’m afraid we need to perform a complete due-diligence effort. And you simply can’t jump the entire queue. You have twenty people ahead of you who would scream bloody murder,” he says.

  “But the Promotions capability is one of the most important features inside Phoenix, which is the most important initiative for the company,” responds Shannon. “Surely you see that ours should have higher priority, right?”

  “Yes, but …” Ron says, shaking his head. “I don’t set the priority or order of the applications. That comes from the business. You know, our customer.”

  “But we are ‘the business!’ And those ‘customers’ you’re talking about aren’t our customers—they’re our colleagues! Our customers are the ones who actually pay us money!” Shannon says, bright red with exasperation. “Everyone knows what the top goals are. The top priorities are what Steve always talks about in all the Town Halls. What else is more important than getting Data Hub successfully decoupled from Phoenix, so that the Promotions team can meet the holiday sales goals?”

  Ron shrugs his shoulders. “If you want to change the order, you’ll have to talk to our boss, John.”

  Kurt closes his laptop, clearly concluding that there’s nothing to be gained in this meeting.

  “Fine, fine, fine,” Shannon says, resigned as well. Then she turns up the charm, saying, “Hey, could you at least give us all of the testing procedures that you’ll use to certify Data Hub, along with a list of tools you use to scan it? We’ll do our best to replicate it in our automated test suite. Maybe we can generate security audit reports for you on-demand.”

  “That’s a great idea, Shannon,” he responds. “Come to my desk and I’ll show where all the documentation for the previous audits are.”

  Maxine loves how Shannon takes every opportunity to get people on their side.

  Watching them leave, Kurt looks at Maxine, shrugging his shoulders. “Could have gone worse, I suppose. Maybe we’ll fare better with the LARB.”

  Maxine sighs. She wonders what is required to generate a true sense of urgency. When her dad had a stroke two years ago, she had remarked on all the bewildering processes in the hospital to one of her doctor friends. Her friend responded, “You were lucky. The processes in a stroke ward tend to be superb, because everyone knows that every minute counts and waiting could be the difference between life and death.

  “The worst systems tend to be in mental health and elderly care, where there is less urgency and often no patient advocate,” she had said. “You can get lost in the system for years. Sometimes even decades.”

  Maxine remembers what it felt like to be the patient advocate for her dad, doing whatever it took to get him through the healthcare system. Now, she recommits herself to doing whatever it takes to get her teams through the company bureaucracies—the Data Hub team’s sense of mission and urgency deserve no less.

  Relentless optimism, she reminds herself.

  As Maggie promised, they are on the LARB agenda on Thursday. Maxine is amazed and wonders what strings Sarah must have pulled to get them in so quickly. Then she wonders what Maggie had to do to convince Sarah.

  Although Maxine recognizes the political necessity of pitching the LARB, she still resents all the time the team spent filling out the TEP—engineers should be writing code, not filling out forms.

  It had many valid questions about architecture and security, but some questions seemed dated, remin
ding her of TOGAF architecture diagrams from decades past, clearly written for a different era: software development and testing phase gates, datacenter specifications, HVAC specifications, Check Point firewall rules (if applicable, of course) …

  The Data Hub gang responsible for putting the proposal together is all here, sitting in the back of the room: Tom, Brent, Shannon, Dwayne, Adam, Purna, and Maxine. At one table sit all the senior Dev and Enterprise architects, and at the other table sit all the Ops and Security architects. They are all close to Maxine’s age, but mostly white males with a couple of Indian and Asian males in the mix. Maxine notices there’s not a single woman at either table.

  Data Hub is second on the agenda. First up is a group pitching to re-platform all of their applications from a commercial product onto Apache Tomcat, a battle-tested and fully open-source Java application server. A younger woman confidently presents their case, which she delivers in a very thoughtful and competent manner. But when Maxine hears that all they’re looking for is permission to use Tomcat, she’s aghast.

  Having to ask permission to use Tomcat in production is like asking permission to use electricity—maybe it was once considered dangerous, but now it’s commonplace. Worse, it’s apparently their second time pitching the LARB. Maxine’s heart sinks. If Tomcat is considered risky and controversial, their Data Hub proposal is going to get laughed out of the room.

  After twenty minutes of skeptical questions from the LARB, the young engineer throws up her hands in exasperation. “Why are we so frightened of running software we wrote? We’re a manufacturing company. We wrote our own MRP and we run it ourselves. And for Tomcat, we don’t need to rely on a commercial vendor anymore. Some of the largest companies in the world use it. We’d not only save the company hundreds of thousands of dollars a year, we could finally do things that our current vendor won’t allow us to do. There’s so many capabilities we need to better serve our customers.”