by Jake Knapp
Stitch it together
Your Stitcher will make sure dates, times, names, and other fake content are consistent throughout the prototype. Don’t mention Jane Smith in one place and Jane Smoot in the other. Look for typos and fix any obvious errors. Small mistakes can remind customers that they are looking at a fake product.
The Stitcher’s job can take many forms, but no matter what you’re prototyping, it’s a critical role. When you divide work, it’s easy to lose track of the whole. The Stitcher will be on the hook to keep everything tight. He may want to check on progress throughout the day, to see if the various parts of the prototype look coherent. And at the end, the Stitcher shouldn’t hesitate to ask the rest of the team to help out if more work is needed.
Trial run
We like to do our trial run around 3 p.m., so that we still have enough time to fix mistakes and patch any holes we find in the prototype. Have everyone pause work and gather around, and then ask the Stitcher to walk through the entire prototype, narrating as he goes.
As you go, you should double-check against the storyboard to make sure everything made it into the prototype. The trial run is also a great time to revisit your sprint questions. It’s one last check to make sure your prototype will help you get answers.
The primary audience for the trial run is the Interviewer, who will be talking with customers on Friday. The Interviewer needs to be familiar with the prototype and the sprint questions so he can get the most out of the interviews. (We’ll explain how to run these interviews in the next chapter.) But the whole team will benefit from watching the trial run. If the Decider isn’t a full-time participant in the sprint, now is another good time for a cameo appearance. The Decider can make sure everything matches what she was expecting.
• • •
In our normal work routines, there are few days where we begin with a big task, follow a precise plan of action, and end the day finished. Thursday is that kind of day, and it’s pretty darn satisfying. When you’re finished with your prototype, don’t be surprised if you start to wonder when you can do it again.
* * *
I. Software changes fast, so check out thesprintbook.com for links to the latest and greatest prototyping tools.
Friday
Sprints begin with a big challenge, an excellent team—and not much else. By Friday of your sprint week, you’ve created promising solutions, chosen the best, and built a realistic prototype. That alone would make for an impressively productive week. But Friday, you’ll take it one step further as you interview customers and learn by watching them react to your prototype. This test makes the entire sprint worthwhile: At the end of the day, you’ll know how far you have to go, and you’ll know just what to do next.
15
Small Data
One August evening in 1996, a publisher named Nigel Newton left his office in London’s Soho district and headed home, carrying a stack of papers. Among them were fifty sample pages from a book he needed to review, but Newton didn’t have high hopes for it. The manuscript had already been rejected by eight other publishers.
Newton didn’t read the sample pages that evening. Instead, he handed them over to his eight-year-old daughter, Alice.
Alice read them. About an hour later, she returned from her room, her face glowing with excitement. “Dad,” she said, “this is so much better than anything else.”
She wouldn’t stop talking about the book. She wanted to finish reading it, and she pestered her father—for months—until he tracked down the rest. Eventually, spurred by his daughter’s insistence, Newton signed the author to a modest contract and printed five hundred copies. That book, which barely made it to the public, was Harry Potter and the Philosopher’s Stone.I
You know the rest of the story. Today, there are hundreds of millions of Harry Potter books in print worldwide. How did publishers get it so wrong? Eight experts in children’s publishing turned Harry Potter down—and the ninth, Newton, only printed five hundred copies. But Alice, an eight-year-old, knew right away that it was “so much better than anything else.”
Alice didn’t analyze Harry Potter’s potential. She didn’t think about cover art, distribution, movie rights, or a theme park. She just reacted to what she read. Those grown-ups tried to predict what children would think, and they were wrong. Alice got it right because she actually was a kid. And her father was smart enough to listen.
• • •
When Nigel Newton showed Alice the Harry Potter manuscript, he got a glimpse into the future. He saw a target reader react to the book before he’d committed to printing a single copy. On Friday of your sprint, you and your team will experience that same kind of time warp. You’ll watch target customers react to your new ideas—before you’ve made the expensive commitment to launch them.
Here’s how Friday works: One person from your team acts as Interviewer. He’ll interview five of your target customers, one at a time. He’ll let each of them try to complete a task with the prototype and ask a few questions to understand what they’re thinking as they interact with it. Meanwhile, in another room, the rest of the team will watch a video stream of the interview and make note of the customers’ reactions.
The FitStar team watches customers use their prototype for the first time.
These interviews are an emotional roller coaster. When customers get confused by your prototype, you’ll be frustrated. If they don’t care about your new ideas, you’ll be disappointed. But when they complete a difficult task, understand something you’ve been trying to explain for months, or if they pick your solution over the competition—you will be elated. After five interviews, the patterns will be easy to spot.
Now, we know that the idea of testing with such a small sample is unsettling to some folks. Is talking to just five customers worthwhile? Will the findings be meaningful?
Earlier in the week, you recruited and carefully selected participants for your test who match the profile of your target customer. Because you’ll be talking to the right people, we’re convinced you can trust what they say. And we’re also convinced that you can learn plenty from just five of them.
Five is the magic number
Jakob Nielsen is a user research expert. Back in the 1990s, he pioneered the field of website usability (the study of how to design websites that make sense to people). Over the course of his career, Nielsen has overseen thousands of customer interviews, and at some point he wondered: How many interviews does it take to spot the most important patterns?
So Nielsen analyzed eighty-three of his own product studies.II He plotted how many problems were discovered after ten interviews, twenty interviews, and so on. The results were both consistent and surprising: 85 percent of the problems were observed after just five people.
Testing with more people didn’t lead to many more insights—just a lot more work. “The number of findings quickly reaches the point of diminishing returns,” Nielsen concluded. “There’s little additional benefit to running more than five people through the same study; ROI drops like a stone.” Instead of investing a great deal more time to find the last 15 percent, Nielsen realized he could just fix the 85 percent and test again.
We’ve seen the same phenomenon in our own tests. By the time we observe the fifth customer, we’re just confirming patterns that showed up in the first four interviews. We tried testing with more customers, but as Nielsen says, it just wasn’t worth it.
Remember the door frame in One Medical’s prototype family clinic? After seeing two children nearly bounce out of their strollers as they rolled into the office, the problem was obvious. The team didn’t need to gather a thousand data points before they fixed it. The same thing with crowding in the lobby and desks in the exam room. When two or three people out of five have the same strong reaction—positive or negative—you should pay attention.
The number five also happens to be very convenient. You can fit five one-hour interviews into a single day, with time for a short break between eac
h one and a team debrief at the end:
9:00 a.m.
Interview #1
10:00
Break
10:30
Interview #2
11:30
Early lunch
12:30 p.m.
Interview #3
1:30
Break
2:00
Interview #4
3:00
Break
3:30
Interview #5
4:30
Debrief
This condensed schedule allows the whole team to watch the interviews together, and analyze them firsthand. This means no waiting for results, and no second-guessing the interpretation.
One-on-one interviews are a remarkable shortcut. They allow you to test a façade of your product, long before you’ve built the real thing—and fallen in love with it. They deliver meaningful results in a single day. But they also offer an important insight that’s nearly impossible to get with large-scale quantitative data: why things work or don’t work.
That “why” is critical. If you don’t know why a product or service isn’t working, it’s hard to fix it. If One Medical had put desks in their family exam rooms, parents would have been frustrated. But it would have been difficult to pinpoint the problem. By showing families a prototype clinic and interviewing them about the experience, One Medical found out the why behind the problem: Parents needed reassurance from the doctor, and even a tiny bit of distraction was too much. When all you have is statistics, you have to guess what your customers are thinking. When you’re doing an interview, you can just . . . ask.
These interviews are easy to do. They don’t require special expertise or equipment. You won’t need a behavioral psychologist or a laser eye-tracker—just a friendly demeanor, a sense of curiosity, and a willingness to have your assumptions proven wrong. In the next chapter, we’ll show you how to do it.
* * *
I. In the United States, the book was called Harry Potter and the Sorcerer’s Stone, due to the fact that philosophers are super dorky.
II. Nielsen, Jakob, and Thomas K. Landauer, “A Mathematical Model of the Finding of Usability Problems,” Proceedings of ACM INTERCHI’93 Conference (Amsterdam, 24–29 April 1993), pp. 206–13.
16
Interview
Michael Margolis is an excellent conversationalist. He smiles easily and asks lots of questions, brimming with a natural curiosity about what it’s like to live where you live, work where you work, and do whatever it is that you do. It’s only afterward that you realize you were talking the whole time and learned little about him.
Michael’s friendliness and curiosity are genuine, but his conversational skills aren’t just a natural gift. Michael is a research partner at GV, and when you watch him interviewing customers—which we’ve seen him do hundreds of times—you realize it’s a practiced art. Everything from the structure of his questions to his body language helps people think aloud and express themselves honestly.
For more than twenty-five years, Michael has conducted research for all kinds of companies—Electronic Arts, Alcoa, Sun Microsystems, Maytag, Unilever, Walmart.com, and Google. Since 2010, he’s been at GV, working with the startups in our portfolio.
Over the years, Michael has adapted his research methods to be fast enough for startups, and learnable for the people who work there. Michael has trained product managers, engineers, designers, salespeople, and countless others in how to conduct these interviews. Anyone can do it—even a CEO.
In this chapter, we’ll let you in on some of Michael’s secrets. Back on Tuesday, you learned his shortcuts for recruiting the perfect target customers (see pages 119–123). In this chapter, you’ll learn how to interview. These interviews can teach you about the people who use your product, reveal hidden problems with your solutions, and uncover the “why” behind it all.
No matter what kind of customer he’s talking to, or what kind of prototype he’s testing, Michael uses the same basic structure: the Five-Act Interview.
The Five-Act Interview
This structured conversation helps the customer get comfortable, establishes some background, and ensures that the entire prototype is reviewed. Here’s how it goes:
1. A friendly welcome to start the interview
2. A series of general, open-ended context questions about the customer
3. Introduction to the prototype(s)
4. Detailed tasks to get the customer reacting to the prototype
5. A quick debrief to capture the customer’s overarching thoughts and impressions
Friday’s action takes place in two rooms. In the sprint room, the team watches the interviews over live video. (Nothing sneaky here. You’ll get the customer’s permission to record and play the video.) The interview itself takes place in another, smaller room—which we cleverly call the “interview room.”
There’s no special tech setup required. We use a regular laptop with a webcam and simple video meeting software to share the video and audio. This arrangement works for websites, but it also works for mobile devices, robots, and other hardware—just point the webcams at what you want to see.
Michael Margolis conducting an interview. He sits beside the customer, but gives her plenty of space. A webcam streams video of the customer’s reaction to the sprint room.
As complicated as it gets: When testing mobile apps or hardware devices, we use a document camera connected to a laptop. Video streams from the laptop to the sprint room.
Sometimes the Interviewer or the customer are in another building, another city, or out in the field (Michael has conducted interviews at hospitals, hotels, and truck stops), but since the sprint team is watching over video, that doesn’t matter. What does matter is that the Interviewer and the customer are sitting side by side, talking comfortably. The interview is a not a group exercise; it’s a conversation between two people. One person from your team can be the Interviewer for the entire day, or two people can alternate. (Since you’re looking for big, obvious patterns, you don’t have to worry about tainting the data with this kind of small change.)
Act 1: Friendly welcome
People need to feel comfortable to be open, honest, and critical. So the first job of the Interviewer is to welcome the customer and put her at ease. That means a warm greeting and friendly small talk about the weather. It also means smiling a lot. (If you’re not in the mood to smile, prepare for the interview by listening to “Keep A-Knockin’ ” by Little Richard.)
Once the customer is comfortably seated in the interview room, the Interviewer should say something like:
“Thanks for coming in today! We’re always trying to improve our product, and getting your honest feedback is a really important part of that.
“This interview will be pretty informal. I’ll ask a lot of questions, but I’m not testing you—I’m actually testing this product. If you get stuck or confused, it’s not your fault. In fact, it helps us find problems we need to fix.
“I’ll start by asking some background questions, then I’ll show you some things we’re working on. Do you have any questions before we begin?”
The Interviewer should also ask the customer if it’s okay to record and watch the video of the interview, and he or she should make sure the customer signs any legal paperwork insisted on by your lawyers. (We use a simple one-page form for nondisclosure, permission to record, and invention assignment. These forms can also be signed electronically before the interview.)
Act 2: Context questions
After the introduction, you’ll be eager to bring out the prototype. Not so fast. Instead, start slow by asking some questions about the customer’s life, interests, and activities. These questions help build rapport, but they also give you context for understanding and interpreting your customer’s reactions and responses.
A great series of context questions starts with small talk and transitions into personal questions relevant to the sprint. If you do it righ
t, customers won’t realize the interview has started. It will feel just like natural conversation.
In our sprint with FitStar, we knew it would be helpful to understand more about each customer’s approach to exercise. Michael’s context questions went something like this:
“What kind of work do you do?”
“For how long have you been doing that?”
“What do you do when you’re not working?”
“What do you do to take care of yourself? To stay in shape? To stay active?”
“Have you used any apps or websites or other things to help with fitness? Which ones?”
“What did you want them to do for you? What do you like or dislike about them? Did you pay for them? Why? Why not?”
As you can see, Michael started with generic small talk (“What kind of work do you do?”) then steered the topic to fitness (“What do you do to take care of yourself?”). As he asked each open-ended question, he encouraged answers with smiles, nods, and eye contact.
At minimum, these context questions make the customer more comfortable and forthcoming. But quite often, the answers help you understand how your product or service fits into the customer’s life—and perhaps, what people think about your competition. In the FitStar interviews, we learned about customers’ experience with workout videos and personal trainers, and how they exercised when they traveled—all useful information.