by Bill Aulet
Moving to Quantitative: Once you have developed credible hypotheses for the small “n” sample through your qualitative methods, then you should start to test them more systematically. The scripts and the surveys now get more structured, and you are looking at scripts and surveys that will start to produce numbers, points on a cluster map, and the like. You will now get the data to validate, invalidate, somewhat validate, somewhat invalidate, or leave in the TBD column for your hypotheses.
Perceived as Opposed to Real Value: Often, as MIT Sloan marketing Professor Duncan Simester has shown in his research, there is a gap between real and perceived value. Both matter, as does the gap. In both your qualitative and quantitative research, understand the difference. Look to understand each of these and then quantify at the end as much as possible.
Don’t Always Believe What Is Said: Often people say things with the best of intent but then do different things in reality. What they do matters more than what they say. A/B testing, where customers are randomly divided into groups and are treated differently to see how the difference in treatment affects the customer’s response. Likewise, and well-designed behavioral economics experiments are good ways to see what customers will really do regardless of what they say. You will explore this topic in more detail in Step 21, Test Key Assumptions.
Results
How do you know when primary market research is complete? It never is. Entrepreneurs and good businesspeople are constantly talking with customers and seeing new opportunities and refining their offerings. That said, there are times when you have enough primary market research to continue to the next step in the 24 Steps. You’ll generally know you are at this point when your hypotheses are being validated and new customer contacts are providing little new information. If your hypotheses are being only somewhat validated, you’ll have to decide whether to keep iterating on hypotheses before moving on to the next step, or to proceed while continuing primary market research in parallel to develop better hypotheses.
As you go through the steps, you will be continually refining your hypotheses for greater specificity, and you will keep learning about the target customer, so do not think you must have the “perfect” answer before moving on. There are no perfect answers, and forward progress through the 24 Steps is the only way you will be able to rigorously test your assumptions. Sometimes you reach a limit on primary market research where you have to take more concrete steps like a prototype or even a Minimum Viable Business Product (MVBP).
As I mentioned before, people can say one thing—and really mean it—but then do something completely different, so do not expect that your interviews will accurately reflect the person’s willingness to become a paying customer. Trust but verify the results. When money changes hands, the credibility of your results increase dramatically.
Toward the end, Step 23, you will be setting up a unified full systems test for the MVBP, which will be the ultimate test for your product. It is likely you will find many surprises—or as I call them, learning opportunities!—when you build the MVBP. But your goal is to find as many of these surprises earlier, especially ones that can be fatal if caught too late in the process. That is why throughout the 24 Steps, you formulate and test hypotheses at almost every step. Good primary market research is essential to validating or invalidating hypotheses quickly so you spend more time building products that customers want.
Tools of Primary Market Research
The tools of primary market research may seem like a longer list than it actually is. At the end of the day, an entrepreneur has limited time and resources. No one is as invested in a product’s design and success as the core team is, so at the start, virtually all of the primary market research has to be done by the founding team. It is very difficult and dangerous to outsource primary market research. You would never outsource your eyes, ears, and brain to someone else, and certainly not in an innovation-driven startup like you envision creating.
The real gold in many cases is not your product, but rather the knowledge you have gained working with the new customers in this emerging market opportunity, since you will know their needs, wants, and context better than anyone else. This is why it is so crucial, when starting your company, that you deal directly with your customers. See firsthand how they like, use, dislike, and misuse your product.
I will talk more about this in Step 18, Map the Sales Process to Acquire a Customer, but I learned a very important lesson from IBM when I got trained there at the start of my career. They said, “Whoever owns the customer owns everything.” That is especially sage advice for a new product in an emerging market.
Here are several methods that entrepreneurs have used for primary market research:
Customer interviews: As noted above, this is the most common. Essential for qualitative and also good for quantitative.
Observational research: Watch customers do their work. Potentially, you would videorecord them or record their mouse and keyboard activity. You would ride with them in the passenger seat (real or metaphorically) as they do their job, carefully observing and asking questions at the right time while making sure not to change their behavior through your actions or questions.
Immersion: Do the customer’s work and fully experience all the dimensions of the job in a way that will give you an understanding that may not come from observation.
User tests: Landing pages and the like don’t just draw in candidates but also give important insights into their behavior and preferences. A/B testing is another form of this and can be very effective, especially if it can be done digitally.
Focus groups: This is another traditional tool that people refer to but has become less and less enthusiastically embraced by entrepreneurs. It can be useful, especially if done very carefully, but it can also have lots of biases and is expensive.
User-driven innovation: This is a technique described and validated through research by MIT Sloan Entrepreneurship Professor Eric von Hippel. It encourages you to look for the end user with the most acute pain from the problem and see how they are finding or developing a workaround solution.
Outcome-driven innovation: This framework is also known as “Jobs to Be Done,” created by Anthony Ulwick and popularized by Harvard Business School Professor Clayton Christensen. Outcome-driven innovation is based on the concept that customers have measurable outcomes they are trying to achieve in their day-to-day (aka when they are doing a job), and a company should link its innovation to those customer outcomes because customers buy products to get jobs done.
In the end, because entrepreneurs don’t have much time or money, they focus on 1–4. They might do some other creative things, like a mystery shop of the closest product (I want to encourage this and other creative ideas). For sure, focus groups are falling out of favor with entrepreneurs, with reason. The methodology of 6 and 7 offer interesting alternatives to creative entrepreneurs, and understanding them to make tools 1–4 better designed and more effective is time well spent.
Primary market research is an imperfect process and can be messy, but it is critically important.
References and Resources
Talking to Humans by Giff Constable: If you read one book on this topic, it should be this one. Very easy read and yet packed with practical knowledge about how to do interviews.
User Innovation edX course with Professor Eric von Hippel: Excellent, easy way to get knowledge from the originator of the concept. Great content and examples that make the fundamental concept easy to understand and apply. The online class is available at https://www.edx.org/course/user-innovation-path-entrepreneurship-mitx-uinov8x-0; he also has a website with extensive free downloadable information at https://evhippel.mit.edu/books/.
Elaine Chen’s portfolio of primary market research materials: Elaine is on the MIT faculty and our go-to practitioner for doing primary market research. She has a long history of doing ethnographic research and using it to build products, both working at startups and as a consultant to other startups. Her
e is a list of her easy-to-read yet very practical materials: “3 Go-to Techniques for Primary Market Research,” Huffington Post, http://www .huffingtonpost.com/entry/three-go-to-techniques-for-primary-market-research_us_577d4001e4b0746f5648b963
“How Startups Can Run Better Landing Page Tests,” Xconomy, http://www.xconomy.com/boston/2016/04/07/how-startups-can-run-better-landing-page-tests/
“A Primer on Primary Market Research,” http://www.slideshare.net/chenelaine/primary-market-research-an-overview-on-qualitative-and-quantitative-research-techniques
Templates and samples: Research protocol template: http://www.slideshare.net/secret/epzen6sYi1gWGr
Sample discussion guide: http://www.slideshare.net/secret/d4a2d3FzWdDnNU
Sample recruitment questionnaire: http://www.slideshare.net/secret/iIMMrhNvWJhv1h
Sample recruitment form for internal use: http://www.slideshare.net/secret/JzXpLWZmlLoAGq
Jobs-To-Be-Done: See http://strategyn.com/customer-centered-innovation-map/ as a starting point.
WORKSHEETS
The Primary Market Research Pledge
I have all of my students sign this pledge at the beginning of the semester before they start to work on their projects. You should also fully commit yourself to proper primary market research by having each member of your team sign this pledge.
Primary Market Research Worksheet I: Preparation
Make a new copy of this worksheet for each market segment you analyze.
1. Secondary market research sources and key lessons learned:
a. ____________________________________________________________________________
b. ____________________________________________________________________________
c. ____________________________________________________________________________
2. Profile(s) of the people you want to engage with (e.g., description of end user, economic buyer, champion, industry analysts, influencers; description should be enough to help you identify, find, and deselect potential candidates. Can include demographics and psychographics—see Step 3 for more info):
1st Targeted Profile Name: ____________________________________________________________________________
Description:______________________________________________________________________________________
2nd Targeted Profile Name: ____________________________________________________________________________
Description:______________________________________________________________________________________
3rd Targeted Profile Name: ____________________________________________________________________________
Description:______________________________________________________________________________________
4th Targeted Profile Name: ____________________________________________________________________________
Description:______________________________________________________________________________________
5th Targeted Profile Name: ____________________________________________________________________________
Description:______________________________________________________________________________________
3. Your general recruitment script (be clear on who you are, why you want to engage, what you are asking for): ______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
4. Initial candidate list to contact
Name & contact info Profile type Source Why you want to engage with this person plus any other info to build rapport
Primary Market Research Worksheet II: Execution
Make a new copy of this worksheet for each market segment you analyze.
1. Which profile are you engaging with:___________________________________________________
How well does this person fit the profile:_____________________________________________
Type of engagement (e.g., interview, observation, test, immersion, other):_____________
2. Your general script/framework for engagement (Guidance: open-ended ➜ qualitative insights/hypotheses ➜ [if appropriate] quantitative insights/hypotheses and data) (approximately 5 key items):
a. ________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
b. ________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
c. ________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
d. ________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
e. ________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
3. What did you learn?
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
4. What surprised you?
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
5. Which hypotheses did you seem to confirm? How and why?__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
6. Which hypotheses did you seem to invalidate? How and why?__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
7. Which hypotheses were you unable to reach conclusions on? Why?__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
8. What new questions were raised in this engagement?__________________________________________________________________________________
__________________________________________________________________________________
____
______________________________________________________________________________
__________________________________________________________________________________
9. List of additional future candidates obtained from current candidate
Name & contact info Profile type Why does the current candidate think we should engage with this person, plus any other info to build rapport
10. What changes should I make for the next primary market research engagement?
Profile changes: __________________________________________________________________________________
__________________________________________________________________________________
Qualitative insights/hypotheses updated (could be more or less than 3):
a. __________________________________________________________________________________
__________________________________________________________________________________
b. __________________________________________________________________________________
__________________________________________________________________________________
c. __________________________________________________________________________________
__________________________________________________________________________________
Quantitative insights/hypotheses updated (optional—only if appropriate and you are far enough along) (could be more or less than 3):