by Jake Knapp
Act 3: Introduce the prototype(s)
Now you’re ready to get the customer started on the prototype. Michael begins by saying:
“Would you be willing to look at some prototypes?”
By asking for permission, he reinforces the status relationship: The customer is doing him a favor, not the other way around, and it is the prototype that will be tested, not the customer. It’s also important to say:
“Some things may not work quite right yet—if you run into something that’s not working, I’ll let you know.”
Of course, if you built a “Goldilocks quality” prototype on Thursday, the customer will forget it isn’t real once they start using it. However, introducing it this way encourages them to give blunt feedback. Explaining that it’s a prototype also makes the Interviewer’s job easier in case something breaks or the customer encounters a dead end (both of which are likely to happen).
Remind the customer that you’re testing the prototype—not her:
“There are no right or wrong answers. Since I didn’t design this, you won’t hurt my feelings or flatter me. In fact, frank, candid feedback is the most helpful.”
That “I didn’t design this” line is important, because it’s easier for customers to be honest if they don’t think the Interviewer is emotionally invested in the ideas. Hopefully the Interviewer avoided working on the prototype on Thursday—but he should probably say “I didn’t design this” even if he actually did. Don’t worry, we won’t tell on you.
The Interviewer should also remind the customer to think aloud:
“As we go, please think aloud. Tell me what you’re trying to do and how you think you can do it. If you get confused or don’t understand something, please tell me. If you see things you like, tell me that, too.”
Thinking aloud makes the interview format especially powerful. Seeing where customers struggle and where they succeed with your prototype is useful—but hearing their thoughts as they go is invaluable.
Act 4: Tasks and nudges
In the real world, your product will stand alone—people will find it, evaluate it, and use it without you there to guide them. Asking target customers to do realistic tasks during an interview is the best way to simulate that real-world experience.
Good task instructions are like clues for a treasure hunt—it’s no fun (and not useful) if you’re told where to go and what to do. You want to watch customers figure out the prototype on their own. As an example, here’s the task from the FitStar test:
“Let’s say you came across FitStar in the App Store. How would you decide if you wanted to try it?”
Starting from this simple nudge, the customer reads and evaluates the app description, installs the app, and tries it out. The “how would you decide?” phrasing encourages her to act naturally along the way.
We learned much more from this simple task than we would have if Michael had micromanaged her at every step. (“Install the app. Now sign up. Now fill in your name.”) Open-ended tasks lead to interesting interviews. Overly specific tasks are boring for both the customer and the sprint team.
As the customer goes through the task, the Interviewer should ask questions to help her think aloud:
“What is this? What is it for?”
“What do you think of that?”
“What do you expect that will do?”
“So, what goes through your mind as you look at this?”
“What are you looking for?”
“What would you do next? Why?”
These questions should be easy to answer and not intimidating. The Interviewer tries to keep the customer moving and thinking aloud, not anxious to find the right answer.
Act 5: Quick debrief
To wrap up the interview, ask a few debrief questions. You’ll see and hear a lot during each interview, and it can be tough to pick out the most important reactions, successes, and failures. When you ask debrief questions, your customers can help you sift through everything you heard.
Here are some of Michael’s debrief questions:
“How does this product compare to what you do now?”
“What did you like about this product? What did you dislike?”
“How would you describe this product to a friend?”
“If you had three magic wishes to improve this product, what would they be?”
Don’t worry—asking the “magic wishes” question doesn’t mean you’re turning your product planning over to your customers. Instead, it helps customers articulate their reactions. It will still be up to you to decide how to interpret and apply what you learn.
If you’re testing two or more prototypes in your interviews, review each one (to refresh the customer’s memory) and ask these questions:
“How would you compare those different products? What are the pros and cons?”
“Which parts of each would you combine to create a new, better version?”
“Which one worked better for you? Why?”
And that’s it. When the interview is over, the Interviewer thanks the customer, gives her a gift card, and shows her out.
Throughout the session, the Interviewer should remain engaged in the conversation. He should encourage the customer to talk while remaining neutral (say things like “uh-huh” and “mmm hmm,” not “great!” and “good job!”) There’s no need to take notes. The rest of the team in the sprint room will take care of that for you.
Of course, we don’t expect anyone to memorize every question and all five acts. On Thursday, the Interviewer can start early by writing up a script while the rest of the team is prototyping. On Friday, he can make a printout to refer to as the interview goes along. Not only will the script make the interviews easier to run, it will also make them consistent—which makes it easier to spot patterns throughout the day.
• • •
One of our favorite stories about the power of interviews comes from our friend, a designer named Joe Gebbia. Back in 2008, Joe and a couple of friends founded a startup. They had what they thought was an amazing idea for a new online marketplace. They built and launched a website, then spent months improving it until they were pretty sure it was perfect.
But despite their efforts, the new service wasn’t catching on. They had a few customers and a little revenue, but they weren’t growing, and—bringing in just $200 a week—they weren’t even making enough to pay the rent. Hoping they could turn the business around before running out of money, the founders took a somewhat desperate measure. They stopped their engineering work, left the office, and tracked down a handful of their customers. Then they interviewed them. One at a time, face-to-face, they watched people use their website.
Joe describes those interviews as “agonizing and enlightening.” He recalls, “We were, like, smacking our heads.” Their website was riddled with flaws. Even simple issues—such as picking a date on a calendar—confused people.
When they returned to the office, Joe and his cofounders spent a week fixing the most glaring problems, and then released a new version to their customers. Revenue doubled to $400 a week, and Joe checked to make sure it wasn’t a bug in their accounting system. But the numbers were real. So they did another round of interviews, and another round of improvements. Revenue doubled again to $800, then $1,600, then $3,200 a week. That growth didn’t stop.
That startup was Airbnb. Today, the online hospitality marketplace operates in more than 30,000 cities and 190 countries. They’ve served more than 35 million guests. It turns out it was an amazing idea, but to make it work, they had to do those interviews. “There’s this gap between the vision and the customer,” Joe says. “To make the two fit, you have to talk to people.”
Airbnb’s interviews showed the founders how the product looked through their customers’ eyes, revealing problems the founders themselves couldn’t see. Listening to customers didn’t mean abandoning their vision. Instead, it gave them the knowledge they needed to combine with that vision, so they could close the ga
p and make a product that worked for real people.
We can’t promise that your interviews will make you as successful as Airbnb, but we can promise that the process will be enlightening. In the next chapter, we’ll talk about how to make sense of what you observe: taking notes, finding patterns, and drawing conclusions about next steps.
INTERVIEWER TIPS
With a Five-Act script, your interviews are sure to be effective. However, there are a few more techniques Michael uses to make them even better.
1. Be a good host
For just a moment, imagine you are the target customer who comes in for an interview. You’ve shown up to try some new product (you’re not quite sure what) in a building you’ve never been to before, and you’ll be watched by some person you just met. This encounter might have seemed like a good idea a couple of hours ago, but now you’re not so sure.
The Interviewer is the host, and the customer is the guest. Michael makes sure the customer is comfortable before the interview begins. He smiles a lot. He’s mindful of his body language. He munches on mints so his breath will be fresh. And he always starts with questions designed to put the customer at ease.
2. Ask open-ended questions
To understand what the customer thinks, you have to be careful not to ask leading questions. Some leading questions are obvious and easy to avoid (we’re sure you won’t say “You like this, right?”). But sometimes, you’ll ask a leading question without meaning to do so.
Let’s say you’re interviewing a customer who’s looking at your website—you want to know what the customer thinks, and whether she would be likely to sign up for a demo of your product.
Interviewer: “Now that you’ve seen the site, would you be ready to sign up now, or do you need more information?”
Customer: “Um, I guess I’d need more info . . . Oh, here’s the FAQ. I’ll check it out.”
This exchange looks okay at a glance, but the multiple-choice question (“ready to sign up” versus “need more information”) has influenced the customer’s response. You’re assuming that the customer wants to do one of those two options. It’s tough to do, but you should avoid asking multiple-choice questions. They’re almost always leading questions in disguise.
Now, consider what would happen if you ask an open-ended question instead.
Interviewer: “Now that you’ve seen the site, what are you thinking?”
Customer: “I dunno, I mean . . . I don’t think it’s right for my company.”
Interviewer: “Why is that?”
Customer: (Insert fascinating reason here.)
We just made this scenario up, but it’s something we’ve seen play out dozens of times. When you ask an open-ended question, you’re more likely to get an honest reaction and an explanation of why.
All of this may sound a little complicated, but Michael’s advice to avoid leading questions comes down to just two rules:
DON’T ask multiple choice or “yes/no” questions. (“Would you . . . ?” “Do you . . ?” “Is it . . . ?”)
DO ask “Five Ws and One H” questions. (“Who . . . ?” “What . . . ?” “Where . . . ?” “When . . . ?” “Why . . . ?” “How . . . ?”)
As with everything, asking questions like these gets easier with practice. One simple trick for the Interviewer: Write some sample “Five Ws” questions right into the script.
3. Ask broken questions
Michael Margolis is the master of broken questions. The idea behind a broken question is to start asking a question—but let your speech trail off before you say anything that could bias or influence the answer.
Customer: “Hmm!”
Michael: “So, what . . . is . . .” (Trails off into silence.)
Customer: “Well, I was just surprised to see that the prices were so high.”
Michael got an honest, useful response out of the customer without even asking a real question. And because the question was so vague, she didn’t feel pressure to tell Michael what she thought he wanted to hear.
In a situation like the example above, where the customer is reacting to something but not saying what, it’s tempting to ask a leading question like “Were you looking at the pricing there?” With a broken question, you can encourage people to think aloud, without leading them in any direction.
You can also learn a lot by just remaining quiet. Don’t always feel compelled to fill the silence with conversation. Stop and watch and wait and listen.
4. Curiosity mindset
Our final bit of advice on how to be a great Interviewer is not a technique, but a state of mind. On Thursday, the team has to be in a prototype mindset. On Friday, the team, and especially the Interviewer, should work hard at adopting a curiosity mindset.
Being in a curiosity mindset means being fascinated by your customers and their reactions. You can develop this mindset by focusing on the surprising details of what your customers say and do. Always ask “why?” Never assume or jump to conclusions. Before each interview, anticipate how interesting the information will be that you’ll learn from the customer. Use your own body language to make yourself friendlier and more receptive: smile, lean in, and don’t cross your arms. Curiosity is an outlook that can be embodied, and even learned.
• • •
If you’d like to learn more about customer interviews (and see a video of Michael conducting an interview), check out thesprintbook.com.
17
Learn
It was 8:30 a.m. on Friday morning in San Francisco, the final day of our sprint with Slack. Michael had scheduled our first customer interview for 9 a.m., and the sprint team was trickling in, coffee cups in hand. We rearranged the couches and chairs so everyone could face the video screen at the front of the room. Braden connected a laptop to the screen, opened a web browser, and joined a video conference that Michael had set up.
Slack started the week with a big challenge: their software was hard to explain to potential customers. Many of the benefits of using Slack (better communication, better teamwork, less stress about work) were only apparent once customers took a chance and tried it with their teams. Experimenting with a new piece of software is a lot of work, so Slack had to make the value clear from the beginning.
By Friday of our sprint, we had two competing solutions. Merci Grace, Slack’s product manager, favored a step-by-step guide called “The Tenacious Tour” that would show customers how Slack worked. Stewart Butterfield, Slack’s founder and CEO, had a hunch about an idea called “Bot Team”—a way for customers to try out Slack by communicating with a team of computer-controlled characters. You know this story, but you don’t know how it ends.
That’s what Friday is all about—finding the end to your sprint story. It’s your chance to put your prototypes in front of real customers, see how they react, answer your sprint questions, and make a plan for what to do next. On that Friday, everyone was excited and a bit nervous to see how the prototypes would do. A buzz of quiet chatter stopped when the screen flickered to life.
Over the video stream, we heard a door close. Michael’s voice began, “Thanks again for coming in today.” Then we saw the first customer sit down, nervously look right into the camera, and relax as Michael asked a few warm-up questions.
Michael introduced the customer to our first prototype. For a moment, the customer did nothing, and then she leaned forward, grabbed the computer mouse, and started talking.
• • •
Friday feels like one long mystery. Throughout the day, you’ll collect clues. Some of those clues help you crack the case, but some lead you in the wrong direction. It’s only at the end—around 5 p.m.—that everything ties together and the answers become clear.
Just like the Slack team, your sprint team will spend Friday together. While the Interviewer is testing the prototype with customers, your team will gather in the sprint room to watch and take notes. It’s the end of an intense week, and your team might feel pressure to get back to “normal work”—emails
, meetings, and mission-critical water cooler conversation. But the sprint only works if you stick together until the end.
Watch together, learn together
Everybody has a superpower. A unique strength. For software engineers, it’s writing code. For marketers, it might be designing campaigns. For us, it’s putting sticky notes on whiteboards. There’s one skill that you’re especially good at, and you probably feel most productive when you’re doing that one thing.
It’s tempting to disband the sprint group on Friday and let everyone return to his or her superpower. That way, the Interviewer can use his interviewing superpower to test your prototype with customers. We’ve tried this approach, and here’s how it goes: The Interviewer talks to each customer. Good so far. Unfortunately, he can’t talk and take detailed notes at the same time, so he records the conversations. The interviews are on Friday, so the soonest he can listen to those recordings is Monday. The interviews take all day, so it takes another full day to review the recordings and make sense of what he heard. Then he needs a few hours to put together a document or presentation with his findings. Now it’s Tuesday. (We even know some researchers who edit together a video “highlight reel” of the most interesting moments from the interviews—awesome, but time-consuming.) And once all that’s done, the Interviewer needs to schedule time with the sprint team to present and review the findings. At the earliest, it will be the Wednesday after the sprint before the team sees the results.
There are other problems. As time passes, the team’s momentum will disintegrate as everyone gets sucked into the vortex of business as usual. And there’s a credibility problem, too. Because the team didn’t witness the test, they’re stuck trusting the Interviewer’s process and results. It’s like the difference between watching a movie yourself and just having someone tell you about it.