INSPIRED

by Marty Cagan

  I would argue that this hour consistently yields a great return on your time. It's critical to learn the answers to these key questions. However, I am a big fan of taking the opportunity of a customer interview to also try out some of our product ideas. We do that after we've learned the answers to these key questions, but it's such a great opportunity I really like to take advantage of it.

  Later, when we talk about testing usability and value, you will see the techniques for this. But for now, just know that you don't have to wrap up after the interview—you can follow it up with a user test of your latest product ideas.

  This hour consistently yields a great return on your time.

  CHAPTER 42

  Concierge Test Technique

  A concierge test is one of my favorite techniques for quickly generating high‐quality product ideas and, at the same time, working on developing the customer understanding and empathy that's so important for motivating the team and delivering strong solutions.

  A concierge test is a relatively new name to describe an old but effective technique. The idea is that we do the customer's job for them—manually and personally. Just as if you went to a hotel concierge and asked if he could find you some theater tickets to a popular show. You don't really know the details of what that concierge is doing for you to get those tickets, but you do know that he is doing something.

  With this technique, you become the concierge. You do what the user or customer needs done for them. You may have to ask them to train you first, but you are in their shoes doing the tasks they would do.

  A concierge test requires going out to the actual users and customers and asking them to show you how they work so that you can learn how to do their job, and so that you can work on providing them a much better solution.

  This is similar, but not the same, as spending some time with your customer service or customer success staff. That is also valuable, and often a good source of product ideas as well, but that is helping customers once they call with a problem.

  A concierge test requires going out to the actual users and customers and asking them to show you how they work so that you can learn how to do their job, and so that you can work on providing them a much better solution.

  If you are building a customer‐enabling product, the users may be employees of your company, but the technique is the same—you go to these colleagues and ask them to teach you how they do their job.

  Like the principle of shared learning, it is most valuable if the product manager, the product designer, and one of the engineers does the concierge test.

  CHAPTER 43

  The Power of Customer Misbehavior

  Historically, the two main approaches used by good teams to come up with product opportunities have been:

  Try to assess the market opportunities and pick potentially lucrative areas where significant pain exists.

  Look at what the technology or data enables—what's just now possible—and match that up with the significant pain.

  You can think of the first as following the market, and the second as following the technology. Either way can get you to a winning product.

  However, some of the most successful companies today have taken a third approach, and while it's not appropriate for every company, I would like to suggest that this is an extremely powerful technique that's largely underutilized and underappreciated in our industry.

  This technique is to allow, and even encourage, our customers to use our products to solve problems other than what we planned for and officially support.

  This third alternative is to allow, and even encourage, our customers to use our products to solve problems other than what we planned for and officially support.

  Mike Fisher, a longtime friend of mine, wrote a book called The Power of Customer Misbehavior. This book tells the eBay and Facebook stories from a viral growth perspective, but there are several other very good examples in there, too.

  From its earliest days, eBay has always had an “Everything Else” category. This is where people could buy and sell things that we at eBay couldn't anticipate people might want to trade. And while we anticipated a lot (there were and still are thousands of categories), some of the biggest innovations and biggest surprises came from monitoring what customers wanted to do.

  We realized early on in the eBay situation that this was where much of the best innovation was happening, and we did everything we could think of to encourage and nurture customers using the eBay marketplace to be able to buy and sell nearly anything.

  While the marketplace may have been originally designed to facilitate trading items like electronics and collectibles, soon people started trading concert tickets, fine art, and even cars. Today, amazingly, eBay is one of the largest used car companies in the world.

  As you might imagine, there are some very significant differences between safely buying and transporting a car and buying a ticket that's good for one night and then worthless. But that work was only done after the demand had been established by enabling customers to transact in items and ways the team and the company didn't anticipate.

  Some product people can get upset when they find customers using their products for unintended use cases. This concern is usually tied to the support obligations. I'm suggesting, however, that this special case can be very strategic and well worth the investment to support. If you find your customers using your product in ways you didn't predict, this is potentially very valuable information. Dig in a little and learn what problem they are trying to solve and why they believe your product might provide the right foundation. Do this enough and you'll soon see patterns and, potentially, some very big product opportunities.

  The Power of Developer Misbehavior

  While the eBay example was intended to be used by end users (buyers and sellers), this same concept is what's behind the trend toward exposing some or all of a product's services via programmatic interfaces (public APIs).

  I consider developers to be one of the consistently best sources of truly innovative product ideas.

  With a public API, you are essentially saying to the developer community, “These are the things we can do—perhaps you can leverage these services to do something amazing that we couldn't anticipate ourselves.”

  Facebook's platform strategy is a good example of this. They opened up access to their social graph to discover the types of things that developers might be able to do once they could leverage this asset.

  I have been a long‐time fan of public APIs as a part of a company's product strategy. I consider developers to be one of the consistently best sources of truly innovative product ideas. Developers are in the best position to see what's just now possible, and so many innovations are powered by these insights.

  CHAPTER 44

  Hack Days

  There are many variations of hack days, but in this chapter, I describe one of my favorite techniques to quickly get a range of high‐potential ideas that are focused on solving a pressing business or customer problem.

  The two main types of hack days are directed and undirected. In an undirected hack day, people can explore whatever product ideas they like, so long as it's at least loosely related to the mission of the company.

  This is one of my favorite techniques for building a team of missionaries rather than mercenaries.

  In a directed hack day, there's a customer problem (for example, something is really difficult to learn and use, or it takes too long to do) or business objective we've been assigned (for example, “Reduce the customer churn rate” or “Increase customer lifetime value”), and we ask people from the product teams to self‐organize and work on any ideas they like that might address this objective.

  The goal is for the self‐organizing groups to explore their ideas and create some form of prototype that can be evaluated, and if appropriate, tested on actual users.

  There are two major benefits to these directed hack days. The first is practical, as the technique facilitates the inclusion
of engineers at ideation. I've mentioned several times in this book that many of the best ideas come from the engineers on the team, and we need to ensure this is happening. It should be happening on an ongoing basis, but this technique will ensure it happens.

  The second benefit is cultural. This is one of my favorite techniques for building a team of missionaries rather than mercenaries. The engineers, if they haven't already, are now diving much deeper into the business context and playing a much larger role in terms of innovation.

  Discovery Prototyping Techniques

  Overview

  Prototypes of various forms have been around for as long as we've been applying technology to solve problems. Per the famous Fred Brooks quote, “Plan to throw one away; you will, anyhow.”

  “Plan to throw one away; you will, anyhow.”

  While Fred's quote is as relevant today as when it was first published (in 1975!), many things have changed, not the least of which is that the tools and techniques we have for developing prototypes and testing them have progressed dramatically.

  That said, I continue to find teams, and even people I would consider thought leaders, that have a very narrow interpretation of what is meant by the term prototype.

  When I press people, what I typically find is that they associate the term prototype with the type that they were first exposed to. If the first one you saw was used to test for feasibility, that's what you think of. If the first one you saw was used for usability testing, that's what you think of.

  But there are in fact many very different forms of prototypes, each with different characteristics and each suited to testing different things. And, yes, some people get themselves into trouble trying to use the wrong type of prototype for the job at hand.

  In this overview, I highlight the major classes of prototypes, and in the chapters that follow, I go into depth on each of them.

  Feasibility Prototypes

  These are written by engineers to address technical feasibility risks during product discovery—before we decide whether something is feasible. Sometimes, the engineers are trying out a new technology. Sometimes it's a new algorithm. Often it is about assessing performance. The idea is for the developer to write just enough code to be able to address the feasibility risk.

  User Prototypes

  User prototypes are simulations. There is a wide spectrum of user prototypes—from those intentionally designed to look like wireframes sketched out on paper (referred to as low‐fidelity user prototypes) all the way up to those that look and feel like the real thing (referred to as high‐fidelity user prototypes), where it can be difficult to tell it's just a simulation.

  Live‐Data Prototypes

  Live‐data prototypes are a little more complicated to explain, but they are a critically important tool for several situations. The main purpose of a live‐data prototype is to collect actual data so we can prove something, or at least gather some evidence—normally to find out whether an idea (a feature, a design approach, a workflow) really works. This typically means two things. First, we need the prototype to access our live data sources, and second, we need to be able to send live traffic—in enough quantity to get some useful data—to the prototype.

  The key is that we don't want to have to build, test, and deploy a commercially viable product to do this. That would take far too long, cost far too much, and very likely yield huge waste. A live‐data prototype costs a small fraction of what it would cost to build a commercially viable product, which is what makes this such a powerful tool.

  Hybrid Prototypes

  There are also many hybrids, which combine aspects of the other types. For example, when working on search and recommendations in which we're focusing on relevance, we may need to have the prototype access live‐data sources, but we don't need to be able to send live traffic. In this case, we're not trying to prove anything, but we can learn a great deal by observing and discussing the results with the users.

  Remember that product discovery is all about coming up with the fastest, cheapest way to test out our ideas. So, depending on your particular idea and situation, you'll want to pick the flavor of prototype that best meets your needs.

  While we all might have our favorites, if you are competing against good product teams, you are going to need to be skilled at each of these.

  CHAPTER 45

  Principles of Prototypes

  As discussed in Chapter 44, there are many forms of prototypes. The best choice for you depends on the particular risk being tackled and the type of product. But all forms of prototypes have certain characteristics and benefits in common. Here are five key principles behind their use.

  The overarching purpose of any form of prototype is to learn something at a much lower cost in terms of time and effort than building out a product. All forms of prototype should require at least an order of magnitude less time and effort as the eventual product.

  Realize that one of the key benefits of any form of prototype is to force you to think through a problem at a substantially deeper level than if we just talk about it or write something down. This is why the very act of creating a prototype so often exposes major issues otherwise left uncovered until much later.

  Similarly, a prototype is also a powerful tool for team collaboration. Members of the product team and business partners can all experience the prototype to develop shared understanding. The overarching purpose of any form of prototype is to learn something at a much lower cost in terms of time and effort than building out a product.

  There are many different possible levels of fidelity for a prototype. The fidelity primarily refers to how realistic the prototype looks. There is no such thing as one appropriate level of fidelity. Sometimes we don't need the prototype to look realistic at all, and other times it needs to be very realistic. The principle is that we create the right level of fidelity for its intended purpose, and we acknowledge that lower fidelity is faster and cheaper than higher fidelity, so we only do higher fidelity when we need to.

  The primary purpose of a prototype is to tackle one or more product risks (value, usability, feasibility, or viability) in discovery; however, in many cases, the prototype goes on to provide a second benefit, which is to communicate to the engineers and the broader organization what needs to be built. This is often referred to as prototype as spec. In many cases, the prototype is sufficient for this, but in other cases—especially when the engineers are not co‐located or when the product is especially complex—the prototype will likely need to be supplemented with additional details (usually, use cases, business rules, and acceptance criteria).

  CHAPTER 46

  Feasibility Prototype Technique

  Most of the time your engineers will review your product ideas and tell you that they have no real concerns about feasibility. This is because they have likely built similar things many times before.

  However, there are several situations wherein your engineers may identify a significant feasibility risk involved in solving a particular problem they are working on. Common examples include:

  Algorithm concerns

  Performance concerns

  Scalability concerns

  Fault tolerance concerns

  Use of a technology the team has not used before

  Use of a third‐party component or service the team has not used before

  Use of a legacy system the team has not used before

  Dependency on new or related changes by other teams

  The idea is to write just enough code to mitigate the feasibility risk.

  The main technique used for tackling these types of risks is for one or more of the engineers to build a feasibility prototype.

  An engineer will create the feasibility prototype because it is typically code (as opposed to most prototypes created by special‐purpose tools intended to be used by product designers). A feasibility prototype is a long way from a commercially shippable product—the idea is to write just enough code to mitigate the feasibility risk. Th
is typically represents just a small percentage of the work for the eventual shippable product.

  Further, most of the time the feasibility prototype is intended to be throwaway code—it's okay and normal to be quick and dirty with this. It is intended to be just enough to collect the data, for example, to show that performance would likely be acceptable or not. There is usually no user interface, error handling, or any of the typical work involved in productization.

  In my experience, building a feasibility prototype requires usually just a day or two of time. If you're exploring a major new technology, such as a new approach leveraging machine‐learning technology, then the feasibility prototype could very well take significantly longer.

  The amount of time the feasibility prototype is estimated to take comes from the engineers, but whether or not the team takes that time depends on the product manager's judgment call as to whether it's worth pursuing this idea. She might say many other approaches to this problem don't have the technology feasibility risk, so she would rather skip this idea.

  While it's the engineers who do this feasibility prototyping work, it is considered discovery work and not delivery work. It's done as part of deciding whether to even pursue this particular approach or idea.