by Marty Cagan
There are three very different techniques for showing the prototype, and you have to be careful to use the right technique for the right situation.
A walkthrough is when you show your prototype to a stakeholder and you want to make sure they see and note absolutely everything that might be a concern. The purpose is to give the stakeholder every opportunity to spot a problem. The product manager usually drives, but if the stakeholder would like to play with the prototype we are happy to let them. You are not trying to sell them anything, you're not trying to test on them, and you're definitely not trying to hide anything from them.
I have seen many inexperienced product managers do a walkthrough with a prospective customer when they should have prepared a product demo. Or another really common rookie mistake is to do a product demo during a user test, and then ask the user what they think.
Be sure to be clear about whether you're doing a user test, a product demo, or a walkthrough. And, be sure you're skilled in how to do all three.
CHAPTER 57
Profile: Kate Arnold of Netflix
Netflix is one of my all‐time favorite products and companies. But back in 1999, a then very young Netflix—based in Los Gatos, with fewer than 20 employees—was on the edge of going bust. They had a couple of experienced co‐founders, including the now legendary Reed Hastings, but the problem was that they were stuck at about 300,000 customers.
They were essentially providing the same general pay‐per‐rental experience that Blockbuster provided, just an online version. There were, as always, some early adopters, and some people lived in places that didn't have a video store, but in truth there wasn't much of a reason to rent DVDs via the U.S. Postal Service when you could just stop by the local Blockbuster store on the way home from work. People would rent once from Netflix and then quickly forget about the service. They didn't seem very willing to change. The team knew that the service wasn't better enough to get people to change.
Those were the technology‐powered innovations that enabled the new, much more desirable business model.
Even worse, DVD sales were starting to lag, and a Hollywood backlash further muddied the situation. Then there were challenges with fulfillment logistics, difficulty maintaining DVD quality, and trying to figure out how to do all this in a way that covered costs and generated some cash.
Kate Arnold was the product manager for this small team, and the team knew they needed to do something different.
One of many tests they tried was to move to a subscription service. The idea was to get people to sign up for a month, and offer them unlimited movies. Would that be perceived as better enough to get them to change their media consumption behavior?
The good news was that, yes, this approach really did appeal to people. A flat monthly fee and all the videos they could consume sounded pretty great.
The bad news is that the team created some real problems for themselves. No surprise that Netflix customers wanted to rent mostly newly released feature films; yet, these were much more expensive for Netflix to stock, and they would need to stock so many copies of these that they'd very likely run out of money fast.
So, the product challenge became how they were going to make sure Netflix customers could watch a set of movies they would love, yet wouldn't bankrupt the company.
They knew they needed to somehow get customers to want and ask for a blend of expensive and less expensive titles. Necessity being the mother of invention, this is where Netflix's queue, ratings system, and recommendation engine all came from. Those were the technology‐powered innovations that enabled the new, much more desirable business model.
So, the team got to work. In three months' time, the team redesigned the site—introducing the queue, the rating system, and the recommendations engine all in support of Netflix as a subscription service.
They also rewrote the billing system to handle the monthly subscription model (a funny little side story is that they launched without this, as they had the 30‐day free trial month, which bought them the extra time they needed).
With so many moving pieces and interconnected efforts, the daily standups included just about every person in the company.
Between working with the co‐founders on the strategy, validating concepts with the users, assessing the analytics, driving features and functionality with the team—and working with finance on the new business model, marketing on acquisition, and the warehouse on fulfillment—you can imagine the workload Kate faced on a daily basis. Yet, the team got the new service up and running and used this to power and grow their business for another seven years, until they disrupted themselves again by moving aggressively to the streaming model.
Kate would be the first to credit a pretty amazing team, including some exceptional engineers, and the vision and courage of the founders. But I would argue that without Kate driving for the technology‐based solutions that could power this business, there's a good chance Netflix as we know it never would have happened.
One more interesting little aside about early Netflix—when they were struggling for cash early on, they offered to sell themselves to Blockbuster for $50 million, and Blockbuster turned them down. Today, Blockbuster is in the dead pool, and Netflix is worth more than $40 billion.
Kate is now a product leader in New York City.
Transformation Techniques
Overview
So far, we've been discussing techniques for discovering successful products. But it's important to acknowledge that getting product teams and companies to apply the new techniques and work differently is often easier said than done.
Partly this is because people are people. But mainly it's hard because the changes are so often cultural.
As a very explicit example, moving from mercenary‐style, product roadmap‐driven, output‐focused teams, to truly empowered, accountable product teams that are measured by business results, represents a major cultural shift and a substantial handoff of power and control from management to the individuals on the teams.
Believe me, this is not the type of change that happens easily.
Fortunately, there are techniques that can help the organization through this.
CHAPTER 58
Discovery Sprint Technique
I find that many teams, especially those new to modern product techniques, are looking for a structured introduction to modern product discovery. In this chapter, I describe the concept of a discovery sprint.
A discovery sprint is a one‐week time box of product discovery work, designed to tackle a substantial problem or risk your product team is facing.
A discovery sprint is definitely useful for more than just transformation. It could just as easily be considered a discovery planning technique or a discovery prototyping technique. But I find it's most helpful in bringing all these things together, so I choose to include it here.
Some people use the term design sprint rather than discovery sprint, but as the purpose of the work—when done well—goes significantly beyond design, I prefer the more general term.
Further, if your company has been struggling with the concept of MVP, this is a very good way to start getting the value from this key technique.
A discovery sprint is a one‐week time box of product discovery work, designed to tackle a substantial problem or risk your product team is facing.
I first met the Google Ventures (GV) team many years ago when they were just getting started. They are part of Google's investment arm, but even more valuable to the startup than their money, GV created a small team to go in and help the companies they invest in get their product efforts off to a good start. Their model is to typically spend a week with the startup—rolling up their sleeves and showing them how to do product discovery by doing it with them side by side.
I also know several other proven product people, known as discovery coaches, who do essentially the same thing for the teams they are helping.
In any case, during this week of intense discovery work,
you and your team will likely explore dozens of different product ideas and approaches, with the goal of solving some significant business problem. You'll always end your week by validating your potential solution with real users and customers. And, in my experience, the result is consistently big learning and insights—the kind of learning that can change the course of your product or your company.
Within this general framework, discovery coaches advocate a variety of different methods to help the team though the process and get big learning in just five days.
After working with more than 100 product teams, and refining their methods as they learned what worked well and what didn't, the GV team decided to share their learnings in a book. The book is titled, Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days, by Jake Knapp, John Zeratsky, and Braden Kowitz.
The authors lay out a five‐day week that starts with framing the problem by mapping the problem space, picking the problem to be solved and the target customer, and then progresses into pursuing several different approaches to the solution. The team next narrows down and fleshes out the different potential solutions, then creates a high‐fidelity user prototype—finally, putting that prototype in front of actual target users and observing their reactions.
And, yes, you can absolutely do this all in a week.
Sprint spells out the authors' favorite techniques to accomplish each of these steps, and if you've read this far, you'll recognize all of them. But what I like so much about the GV book is that, when teams are getting started, they often crave the structure of a proven, step‐by‐step recipe. The book spells out exactly this over nearly 300 pages, with dozens of examples from great products and teams you'll recognize.
There are several situations where I recommend a discovery sprint, starting with when the team has something big and critically important and/or difficult to tackle. Another situation where a discovery sprint helps is when the team is just learning how to do product discovery. And yet another is when things are just moving too slow and the team needs to recalibrate on just how fast they can and should be moving.
Sprint is another must read book for product managers, and I highly recommend it.
Discovery Coaches
As teams moved to Agile methods (they usually start with Scrum), many companies decided to contract with or hire an Agile coach. These coaches help the broader team—especially engineers, QA, product managers, and product designers—learn the methods and mindset involved in moving to Agile.
Discovery coaches are typically former product managers or product designers and they have usually worked for, or with, leading product companies.
This sounds straightforward enough, but many problems arise because the vast majority of these Agile coaches don't have experience with tech‐product companies, so their experience is limited to delivery. Therefore, they would more accurately be considered Agile delivery coaches. They understand the engineering and release side of things, but not the discovery side of things.
So many product companies have experienced this issue that it created the need for coaches that do have deep experience with product companies and the key product roles, especially product management and product design. These individuals are often called discovery coaches.
Discovery coaches are typically former product managers or product designers (or former leaders of these areas) and they have usually worked for, or with, leading product companies. So, they are able to work side by side with actual product managers and designers—not just reciting Agile platitudes, but showing the team how to work effectively.
Every discovery coach has his or her preferred way of engaging with a team, but they are usually engaged with one or a small number of product teams for a week or so. During this time, they help you through one or more discovery cycles of ideation; creating prototypes; and validating the prototypes with customers to gauge their reactions, with engineers to evaluate feasibility, and with business stakeholders to assess whether this solution would work for your business.
It's hard for me to imagine an effective discovery coach who doesn't have first‐hand experience as a product manager or product designer at a modern product company. That's likely one of the main reasons there's a shortage of discovery coaches today. It's also important that the discovery coach understand how to include engineering in the mix—being sensitive to their time, but understanding the essential role they play in innovation.
Discovery coaches are not unlike Lean Startup coaches. The main difference is that Lean Startup coaches often focus on helping a team discover not only their product, but also their business model, and their sales and marketing strategy. Once the new business has some traction, the discovery is usually more about continuously improving an existing product in substantial ways rather than creating an all‐new business. Because of this difference, many Lean Startup coaches don't have the necessary product experience. My view is that product discovery is the most important competency of a new startup, so I believe an effective Lean Startup coach must also be very strong at product.
CHAPTER 59
Pilot Team Technique
Earlier in this book, we discussed the technology adoption curve and how this theory describes how different people will accept change. It turns out this theory also applies to our own organization and especially to how we make changes to how that organization works.
Some people in your organization love change, some want to see someone else use it successfully first, some need more time to digest changes, and a few hate change and will only change if they're forced to do it.
If you get too excited and roll out a significant change to everyone in the organization at once, then the laggards (those that hate change) may resist or even sabotage your efforts.
Rather than fight this reality, we can embrace it. One of the simplest techniques for facilitating moving to new ways of working is the use of pilot teams. Pilot teams allow the roll out of change to a limited part of the organization before implementing it more broadly. The idea is that you look for a product team to volunteer to try out some new techniques. You let them run for a while (usually a quarter or two) with this new way of working and see how this goes.
Some people in your organization love change, some want to see someone else use it successfully first, some need more time to digest changes, and a few hate change and will only change if they're forced to do it.
Your specific success measures will depend on your goals, but ultimately, you're looking to compare the team's effectiveness in delivering business outcomes; that is, how well do the pilot teams accomplish their objectives versus the others or compared to how they did in the past?
Given the nature of the experiment, your comparisons will be qualitative, but that doesn't make them any less compelling.
If things go well, you'll likely end up with several other teams eager to adopt. If things don't go well, you might decide this technique is not a fit for you, or you might decide to make adjustments.
To maximize the chance of the pilot teams having a good outcome, we carefully consider the people involved, their location, and their degree of autonomy. Ideally, we have people who are open to new ways of working, the key people on the team are co‐located, and the team is largely in control of how they work and not so dependent on other teams that still work in the old way.
CHAPTER 60
Weaning an Organization Off Roadmaps
Many product teams want to move off product roadmaps, but their organizations are old school, addicted to the outdated quarterly product roadmap. As a result, they don't see how to transition their organizations forward.
Here's what I advocate in this case: Plan to continue with your existing roadmap process for six to 12 months. However, starting immediately, every time you reference a product roadmap item, or discuss it in a presentation or meeting, be sure to include a reminder of the actual business outcome that feature is intended to help.
The goal is that over time, the or
ganization moves its focus from specific features launching on specific dates to business results.
If the feature you're working on is to add PayPal as a payment method, and the reason is to increase conversion, then be sure to always show the current conversion rate and the result you're hoping to achieve. Most important, after the feature goes live, be sure to highlight the impact on that conversion rate.
If the impact was good, celebrate it. If the impact was not what was hoped, then emphasize to everyone that, while you did ship the feature, the result was not successful. Point out specifically what was learned, but explain that you have other ideas for ways to get the desired result.
The goal is that over time (it can take as long as a year), the organization moves its focus from specific features launching on specific dates to business results.
For this to work, it's important to acknowledge the two big reasons why stakeholders especially are so attracted to roadmaps:
They want some visibility into what you are working on and assurance that you are working on the most important items.
They want to be able to plan the business and need to know when critical things will happen.
The modern alternative to roadmaps discussed here addresses both of these concerns. Teams work on the prioritized business objectives determined by the leaders; we share our key results transparently; and we commit to high‐integrity commitments when critical delivery dates are needed.