by Marty Cagan
Second, we need to ensure we deliver a robust and scalable implementation that our customers can depend on for consistently reliable value. Your team needs to be able to release with confidence. While you'll never have 100 percent confidence, you should not have to release and pray.
So, we need to learn fast, yet also release with confidence.
It's understandable that many people might naturally view these two difficult goals as at odds with each other. We are in a big hurry to push something out to learn what works and what doesn't. Yet, we don't want to release something that's not ready for prime time and risk hurting our customers and damaging our brand.
I spend a lot of my time visiting with product teams. I have on occasion been called out for pushing hard one minute for the team to be much more aggressive in getting out to customers and getting early feedback on their ideas, and then just minutes later pushing that same team hard not to compromise their standards on releasing scalable, fault‐tolerant, reliable, high‐performance, secure software.
We need to learn fast, yet also release with confidence.
You might also recognize this problem in another guise. Many teams get into a lot of grief with the concept of a minimum viable product (MVP) because on the one hand we are very motivated to get this out in front of customers fast to get feedback and learn. And, on the other hand, when we do get out there fast, people feel like this so‐called product is an embarrassment to the brand and the company. How could we possibly consider launching this?
In this section, I clarify how strong teams work to meet these dual and simultaneous objectives of rapid learning in discovery, yet building stable and solid releases in delivery.
In general, I find that most product teams have a much better sense of how to accomplish the second goal of delivering solid software than how to accomplish the first goal of rapid experimentation and discovery. Continuous delivery is a good example of an advanced delivery technique I find in teams that understand the importance of a series of small, incremental changes to a complex system.
Part of what causes confusion is a dilution of what is really meant when we call something a “product” or “product‐quality” or “productized” or “live in production.”
I always try hard to reserve the term product to describe the state at which we can run a business on it. Specifically, it is scalable and performant to the degree necessary. It has a strong suite of automated regression tests. It is instrumented to collect the necessary analytics. It has been internationalized and localized where appropriate. It is maintainable. It is consistent with the brand promise. And, most important, it is something the team can release with confidence.
This is not easy—it's where most of the time goes when our engineers are building. As such, we try very hard not to waste this effort.
Doing all this work when the product manager isn't even sure this is the solution the customer wants or needs is a recipe for product failure and big waste. So, the purpose of product discovery is to make sure we have some evidence that when we ask the engineers to build a production‐quality product, it won't be a wasted effort. And, this is why we have so many different techniques in product discovery.
We've got techniques for getting a much deeper understanding of our users and customers, and for validating product ideas both qualitatively and quantitatively. And, in fact, most of the techniques don't require the developer's time (which is important, because we appreciate how much time and effort needs to go into creating production‐quality software in delivery).
Much of the key to effective product discovery is getting access to our customers without trying to push our quick experiments into production.
If you are an early stage startup and you have no customers, then of course this is not really an issue (and it's very likely premature to even be creating production‐quality software).
But, for most of us, we have real customers and real revenue, so we do have to care about this. Later in this section, we'll talk about the techniques that allow for rapid experimentation in a responsible way in larger, enterprise companies.
But here's the key. If you want to discover great products, it really is essential that you get your ideas in front of real users and customers early and often.
If you want to deliver great products, you want to use best practices for engineering and try not to override the engineers' concerns.
CHAPTER 33
Principles of Product Discovery
The purpose of product discovery is to address these critical risks:
Will the customer buy this, or choose to use it? (Value risk)
Can the user figure out how to use it? (Usability risk)
Can we build it? (Feasibility risk)
Does this solution work for our business? (Business viability risk)
And it's not enough that it's just the product manager's opinion on these questions. We need to collect evidence.
When it comes to how we do product discovery, there are a set of core principles that drive how we work. If you understand these, you will understand not only how to work well today but also how to easily incorporate new techniques as they emerge in the future.
We know we can't count on our customers (or our executives or stakeholders) to tell us what to build. Customers don't know what's possible, and with technology products, none of us know what we really want until we actually see it. It's not that customers or our executives are necessarily wrong; it's just that it's our job to make sure the solution we deliver solves the underlying problem. This is probably the most fundamental principle in all of modern product. Historically, in the vast majority of innovations in our industry, the customers had no idea that what they now love was even a possibility. This is only becoming truer with time.
Customers don't know what's possible, and with technology products, none of us know what we really want until we actually see it.
The most important thing is to establish compelling value. It's all hard, but the hardest part of all is creating the necessary value so that customers ultimately choose to buy or to use. We can survive for a while with usability issues or performance issues, but without the core value, we really have nothing. As a result, this is generally where we'll need to spend most of our discovery time.
As hard and important as the engineering is, coming up with a good user experience is usually even harder, and more critical to success. While every product team has engineers, not every team has the necessary product design skills, and even when they do, are they being used the way we need to use them?
Functionality, design, and technology are inherently intertwined. In the old waterfall model, the market drove the functionality (aka the requirements), which drove the design, which drove the implementation.
Today, we know that the technology drives (and enables) the functionality as much as the other way around. We know that technology drives (and enables) design. We know that design drives (and enables) functionality. You don't have to look further than your own phone to see numerous examples of both. The point is that all three of these are completely intertwined. This is the single biggest reason we push so hard for the product manager, product designer, and tech lead to sit physically adjacent to each other.
We expect that many of our ideas won't work out, and the ones that do will require several iterations. “The most important thing is to know what you can't know.”
To quote Marc Andreessen, “The most important thing is to know what you can't know,” and we can't know in advance which of our ideas will work with customers and which won't. So, we approach discovery with the mindset that many, if not most, of our ideas won't work out. The most common reason for this is value, but sometimes the design is too complicated, and sometimes it would take far too long to build, and sometimes there turn out to be legal or privacy issues. The point is we need to be open to solving the underlying problem in different ways if necessary.
We must validate our ideas on real users and customers. O
ne of the most common traps in product is to believe that we can anticipate our customer's actual response to our products. We might be basing that on actual customer research or on our own experiences, but in any case, we know today that we must validate our actual ideas on real users and customers. We need to do this before we spend the time and expense to build an actual product, and not after.
Our goal in discovery is to validate our ideas the fastest, cheapest way possible. Discovery is about the need for speed. This lets us try out many ideas, and for the promising ideas, try out multiple approaches. There are many different types of ideas, many different types of products, and a variety of different risks that we need to address (value risk, usability risk, feasibility risk, and business risk). So, we have a wide range of techniques, each suitable to different situations.
We need to validate the feasibility of our ideas during discovery, not after. If the first time your developers see an idea is at sprint planning, you have failed. We need to ensure the feasibility before we decide to build, not after. Not only does this end up saving a lot of wasted time, but it turns out that getting the engineer's perspective earlier also tends to improve the solution itself, and it's critical for shared learning.
We need to validate the business viability of our ideas during discovery, not after. Similarly, it is absolutely critical to ensure that the solution we build will meet the needs of our business—before we take the time and expense to build out that product. Business viability includes financial considerations, marketing (both brand and go‐to‐market considerations), sales, legal, business development, and senior executives. Few things destroy morale or confidence in the product manager more than finding out after a product has been built that the product manager did not understand some essential aspect of the business.
It's about shared learning. One of the keys to having a team of missionaries rather than a team of mercenaries is that the team has learned together. They have seen the customer's pain together, they have watched together as some ideas failed and others worked, and they all understand the context for why this is important and what needs to be done.
Everything that follows is based on these core principles.
Ethics: Should We Build It?
In general, product discovery is about tackling risks around value, usability, feasibility, and business viability. However, in some cases, there's an additional risk: ethics.
I know this is a sensitive topic, and I don't want to sound like I'm preaching or condescending in the least, but I personally encourage the teams I work with to also consider the question, “Should we build it?”
I encourage product teams to consider the ethical implications of their solutions, too.
You may think this is a matter of doing something illegal, but in the vast majority of cases where ethics is an issue, it's not usually a matter of law. Rather, just because we have the technology to build something, and even if it otherwise works to accomplish the specific business objective, this does not necessarily mean that we should build it.
More commonly, the issue is that our technology and design skills are such that we might come up with a solution that meets our business objectives (for example, around engagement, growth, or monetization) but can end up with a side effect of causing harm to users or the environment.
So, I encourage product teams to consider the ethical implications of their solutions, too. When a significant ethical risk is identified, see if you can't find alternative solutions that solve the problem in a way that doesn't have negative consequences.
I have one final, but critically important, note about raising ethics issues with senior management. You absolutely need to have a strong understanding of your business, especially how you make money. You need to use good judgment and be sensitive in your discussion. You are not there to try to police the organization but, rather, to identify issues and bring potential solutions.
Discovery Iterations
Most product teams normally think of an iteration as a delivery activity. If you release weekly, you think in terms of one‐week iterations.
But we also have the concept of an iteration in discovery. We loosely define an iteration in discovery as trying out at least one new idea or approach. It's true that ideas come in all shapes and sizes, and some are much riskier than others, but the purpose of discovery is to do this much faster and cheaper than we can do in delivery.
To set your expectations, teams competent in modern discovery techniques can generally test on the order of 10–20 iterations per week. This may sound like a lot to you, but you'll soon see that's not so hard at all with modern discovery techniques.
To set your expectations, teams competent in modern discovery techniques can generally test on the order of 10–20 iterations per week.
Also, realize that many iterations never make it beyond just you, your designer, and your tech lead. The very act of creating a prototype often exposes problems that cause you to change your mind. As a rule of thumb, an iteration in discovery should be at least an order of magnitude less time and effort than an iteration in delivery.
CHAPTER 34
Discovery Techniques Overview
There is no one perfect taxonomy for discovery techniques because several of the techniques are helpful for multiple different situations. Regardless, following are the key techniques in the framework that I personally use and find helpful.
Discovery Framing Techniques
Framing techniques help us to quickly identify the underlying issues that must be tackled during product discovery. If we're handed a potential solution, we need to clarify the underlying problem to be solved. We need to tease out the risks and determine where it makes sense to focus our time. We also need to ensure that we understand how our work fits in with the work of other teams.
Discovery Planning Techniques
There are a few techniques that are useful throughout the product discovery effort and help with identifying the bigger challenges and planning how you'll attack this work. We'll discuss these here.
Discovery Ideation Techniques
There are, of course, any number of ways to come up with ideas. But some sources are better than others in their potential for keeping us focused on the most important problems. Ideation techniques are designed to provide the product team with a wealth of promising solutions aimed at the problems we're focused on now.
Discovery Prototyping Techniques
Our go‐to tool for product discovery is typically a prototype. We'll discuss the four main types of prototypes and highlight what each type is best suited for.
Discovery Testing Techniques
Product discovery is mostly about quickly trying out an idea. We are essentially trying to separate the good ideas from the bad. Here we are defining a good idea as one that solves the underlying problem in a way that customers will buy, they can figure out how to use, we have the time and skills and technology on the team to build, and that works for the various aspects of our business.
It's important to recognize that many ideas don't have that much risk associated with them. They may be very straightforward. Or they might just have one area that's a risk, such as our legal department's concern about a potential privacy issue.
Occasionally, however, we need to tackle much tougher problems, and we may in fact have significant risks in most or even all these areas.
So, the way to think about discovery is that we only validate what we need to, and then we pick the right technique based on the particular situation.
Testing Feasibility
These techniques are designed for the engineers to address areas where they identify concerns. The solution being tested might require some piece of technology that the team has no experience with. There may be significant scale or performance challenges. Or there might be third‐party components that need to be evaluated.
Testing Usability
These techniques are designed for the product designers to address areas where they hav
e identified concerns. Many of our products have complex workflows and the designers need to ensure their interaction designs make sense to the user and potential sources of confusion are identified and pre‐empted.
Testing Value
Much of our time in product discovery is spent validating value or working to increase the perceived value. If it's a new product, we need to ensure that customers will buy it, at the price we need to charge, and that they'll switch from whatever they're using today. If it's an existing product, and we are improving that product (such as with a new feature or a new design), where the customer has already bought the product, we need to ensure the customers will choose to use the new feature or new design.
Testing Business Viability
Sadly, it's not enough to create a product or solution that our customers love, that is usable, and that our engineers can deliver. The product also must work for our business. This is what it means to be viable. This means that we can afford the cost of building and provisioning the product and the costs to market and sell the product. It needs to be something our sales force is capable of selling. It means that the solution needs to also work for our business development partners. It needs to work for our legal colleagues. It needs to be consistent with our company's brand promise. These techniques are about validating these types of risks.