7 Rules of Marketing that Get Results
Page 21
In this approach to post-test research, because the ad copy is shown and all the brand clues are first introduced and then interrogated, unsurprisingly the advertising performance of brands is outstanding. By using this research methodology, marketing directors may prove both to themselves and to their CEOs that their advertising has been a smashing success. In addition, their advertising agencies may be delighted to hear just how good their own work is.
Most research companies, while they conduct post-test research, also measure brand perception by asking people what they think about the brand and its rivals. And their questioning doesn’t end there; they also ask whether or not participants will buy the brand. When the answers to the questions about “perception” and “purchasing intent” are thrown in, the researchers gather results that are beyond imagination. If you show people an emotionally resonant brand advertisement and then ask them several questions about the brand for ten to fifteen minutes, how could the results be anything but 100% positive when it comes to their perception and likelihood of buying the brand in the future?
This post-test methodology may make everyone happy, but it’s little more than self-deception and a waste of the company’s limited resources.
As discussed in chapter 69, the average success of an advertisement is as low as 16%, and the job of marketing directors and advertising agencies is to raise ad performance above that threshold. Their job is to improve the performance, not to engage in biased research design that inflates the numbers. (See Byron Sharp, How Brands Grow.)
Biased research design is a real problem in market research. I try to put a stop to this terrible habit in every company I work with, but the condition is apparently difficult to cure. Changing a system like this one that keeps both sides happy is a challenge.
What is a better, less biased way to conduct this research? The correct way to conduct post-test research is to show people the advertisement without the logo and then ask them the following questions: whether or not they’ve seen the advertisement, what they understood from the advertisement, whether or not they liked the advertisement and, finally, whether or not they remember which brand it belongs to.
Advertising research should under no circumstances include perception questions or issues related to intent to purchase. In this context, such questions lead the participant responses and skew the research, yielding unusable biased data that doesn’t help the company to get an accurate sense of where it stands.
Customer satisfaction research Companies improve products and services they market by measuring whether or not they’ve met users’ expectations.
NPS (Net Promoter Score) is currently the most widely used method for measuring customer satisfaction around the world, although, as I mentioned earlier, it provides absolutely no benefit when it comes to measuring satisfaction and serving as an indication for future purchase. Marketers like to use NPS simply because a single question is used to make the measurement, ensuring ease of calculation.
I do my best to discourage the companies I work with from using the NPS method, because I don’t believe it’s possible to measure something as multifaceted as customer satisfaction with a single question. Scientific studies confirm my convictions (details in chapter 51). Instead of relying on NPS, I argue that companies should have unique studies done that are appropriate for the scope of the different services they provide.
In addition to customer satisfaction research, companies also conduct mystery shopper research (sending a secret shopper to a store) to measure service quality of sales staff. As with all research, if this project is well planned and conducted by qualified personnel, the information gathered will be beneficial to the company.
Demand analysis Before companies roll out a new product, they conduct research on demand potential; they study what people think about the product they’re thinking of releasing. They also investigate the price of the new product, the concepts that will be used in ads and whether or not people will actually buy the product.
Unfortunately, the currently available research toolbox doesn’t have an accurate tool that serves this purpose. In fact, in my 25-year research career, the most misleading results I have obtained were from this kind of research (details in chapter 89). When people are shown a new product, they simply don’t know whether or not they’ll buy it in the future. To make accurate predictions based on their responses is hardly possible.
Obviously, one may design a beneficial research study that will help to discover the unmet needs of people, but these researches are easier said than done.
Custom research In addition to all of the studies I’ve talked about here, companies can conduct ad hoc research about any marketing activity. Because of the wide range of marketing activities that a company needs, and the information needed about these activities, it isn’t difficult to see that custom research constitutes a large portion of the research industry. On request, companies that conduct research can design custom research for almost any area. The research company will choose a sample from the target audience—usually between 300 to 1,000 people—and use quantitative methods to analyze this sample.
In some situations, research companies will also use qualitative methods, such as focus groups or interviews.
Qualitative research This type of research most often takes the form of a focus group. Focus groups consist of six to eight people gathered together in a room with a moderator. This moderator asks questions in the natural flow of the discussion to obtain the participants’ opinions. The most important advantage of this type of research, compared to quantitative research, (which provides statistical results from a large number of people), is that it allows the company to delve into the details. Qualitative research also provides marketing directors with the flexibility to amend the research (to ask additional questions) even while it’s being conducted.
This method provides results much faster and at a lower cost than quantitative research. It accounts for approximately 10% of the total research market.
But there are many problems associated with this methodology. First, the small number of participants may not represent the population. Second, people in general don’t think too much about their own purchasing or consumption habits in their daily life. But when asked questions about their habits by a moderator in a focus group that lasts at least two hours, usually people will fabricate answers. Without any bad intentions, they invent misleading opinions about the product and/or the brand. Third, if one of the participants is too knowledgeable about the category and brand, this knowledge can taint the group dynamic and affect the other participants.
If such biases can be avoided, companies may benefit from qualitative research.
Neuroscience market research This research measures human brain activity using magnetic resonance (MR) and electroencephalogram (EEG) machines. Neuroscientific research studies are usually conducted on ten to twenty people. Every participant (subject) is hooked up to a machine while they’re shown a visual, or asked to listen to music or taste something related to the product or brand in question. During this exposure, the researcher monitors what parts of the brain are activated and reports what people’s true responses are to the stimuli. The value of this method is that instead of evaluating people’s spoken or written responses to what they’ve seen, heard or tasted, researchers monitor their brain activity. This methodology ensures no distortion in the responses of participants—the ideal scenario every researcher wants to achieve. This relatively new method is developing and advancing every day. Marketers and researchers are only just beginning to understand what we can accomplish with the understanding gained from this technique.
Ethnographic research This research observes humans under the real conditions in which they live. The anthropologist who directs ethnographic research will prepare a report based on their observations, providing brands with depth and a level of detai
l they can’t possibly obtain from any other kind of research. Essentially, ethnography provides expert insight. It’s not a prevalent type of research because it’s difficult to conduct, takes a long time and is naturally more expensive than other research. However, it’s a technique that large international corporations employ.
These are the major types of market research that companies use today to understand consumers and the market. Each type, used alone, has its strengths and its shortcomings. To benefit from research, marketers will need to use a custom mix of methodologies tailored to each company’s needs. Most importantly, companies need the right research, conducted in an unbiased and cost-effective manner, and it must measure the things they actually need to know so they can run their businesses well.
89. Und
erstanding the Nature of Research
Length is measured with a measuring tape, weight with a scale and temperature with a thermometer. In marketing, however, the researcher’s unit of measurement is the questions they ask.
Researchers measure people’s emotions, thoughts, behavior and tendencies by asking questions. However, the people who answer the researcher’s questions can sometimes—totally unintentionally—be misleading with their responses. The answers that people give to some questions in some surveys don’t reflect the truth. This bias is an intrinsic part of research. It’s the fundamental difficulty that every researcher should overcome.
Here’s how it happens:
People tend to give the “right” answers to research questions. Everyone knows that they should brush their teeth twice a day, read books, exercise and eat right, so when a researcher asks them questions about these sensitive issues, their answers are generally what “should be” instead of what “is.”
Issues such as faith and values are also sensitive matters that people don’t want to share with strangers. Similarly, people are rarely transparent about how much money they make or how much tax they pay. Because research on these types of issues doesn’t yield accurate results, experienced researchers approach these matters cautiously.
Remember, people don’t generally know why they do what they do. The decision to buy some brand in a certain category from a supermarket is not very important for most people. As Andrew Ehrenberg and Byron Sharp succinctly explain, they already buy several brands on a regular basis, and they buy one of those brands every time they go shopping. Their choice of any one of the brands from their repertoire that day might be due to any of a number of reasons: maybe the fact that it was displayed better on the shelf, or because they saw a poster for it in the store or because the woman in front of them chose it. No one lives their life by analyzing the reasons behind all of their decisions; thus, they don’t know exactly why they do what they do. If the researcher asks the shopper why they made a particular choice, they’ll give generic answers, such as “the quality is better,” “the price is reasonable,” or “this is the brand my friends use.” These answers may not be the real reasons that the shopper purchased the brand (details in chapter 24). People follow the “law of least effort” when they’re making daily decisions. They make almost all of their decisions in the manner they know best, trust most and are most familiar with. Yet people also intellectualize when given the opportunity to reflect. When researchers ask people the reasons behind their habitual behaviors, people prefer to answer in a way that makes them look intelligent, rather than saying, “I don’t know.” Daniel Kahneman’s descriptions of decision-making apply here. A person living their life according to System 1 (using intuitive, “fast” thinking) and a researcher thinking and working according to System 2 (deliberate, data-based, “slow” thinking) are people from totally different worlds (details in chapter 11). In the moment, it’s like they’re even speaking different languages.
Researchers need to remember this: people make decisions using System 1 thinking, but they’re not aware this is how they make decisions. In studies, consumers say that they make brand selections with their own free will, uninfluenced by advertising, but all of the products in their house are brands that engage in extensive advertising.
For all of the above reasons, it’s critical that researchers avoid, as much as possible, any “why” questions when conducting their surveys.
People don’t usually know exactly what they need, but they’re also unaware of their own ignorance of the fact. In 1990, before cell phones were available, I did a study for Nokia to estimate cell phone demand. Our goal was to calculate how much demand there would be for this innovative product that no one knew anything about. Using the most advanced methods, we determined that there would be no demand for cell phones. Consumers didn’t want to use cell phones! If a decision had been made based on this research, none of the cell phone operators we have today would have invested in the cell phone market. Significant biases exist in new product research and demand prediction research, because people don’t know what they’ll need in the future.
Such is the nature of research. I view it as a diamond hidden deep in the earth, and once extracted, it can only be seen after all the dirt is rinsed away. To reach this diamond, both the researcher and the company commissioning the research must
get basic knowledge about human nature (details in chapters 3–14),
learn the marketing laws (details in chapters 26–35),
know which types of research provide reliable results, and which are the source of major biases,
know the areas on which not to conduct market research, and
know what conditions to fulfill to get proper research.
No research is 100% accurate, but it’s possible, obviously, to do research that measures reality and that will help executives make the right decisions. Good research doesn’t give marketers ready-to-implement decisions. Good research gives executives insights that dispel the clouds so they can see a little further away.
This is why good research is invaluable.
90. Bad
Research Habits
Most companies have developed many bad habits in commissioning market research. Just as they attach to the marketing myths mentioned throughout this book, companies get attached to doing research incorrectly: requesting the wrong type aimed at the wrong audiences, and often at the wrong intervals.
From what I’ve seen, companies need to break the following research habits:
Conducting too frequent research on matters where no change is expected is a bad habit. As Byron Sharp explains, when a brand has significant market share, this success has a positive effect on all other performance metrics (details in chapter 26). Most of the companies that are leaders in their markets today conduct research in several different areas, but in every study, all the findings are affected by the size of the brand in the market. So, in fact, every study measures the same happening in different aspects. Brand image research is the chief culprit here. The perception of every brand is directly proportional to its market share. Unless the number of users reached by a brand (penetration) changes, no change in perception should be expected. What’s interesting about this, Sharp points out, is that even if companies conduct this research every month, they obtain different results, because results vary over time because of sampling errors and many other factors. Because the human mind is programmed to create meaning, when confronted with these essentially meaningless fluctuations, every executive will reach conclusions that have no basis in fact.
Another bad habit is the infatuation that company executives have with research models that have interesting or snazzy names, when, in fact, they don’t exactly understand what these models measure or how they’re conducted. A perfect example of this is the phrase “brand health” (mostly based on loyalty pyramid analysis—which is a myth—as described in chapter 50). Many executives have no clue what these types of models are measuring or what they’re used for, but most of them think they are “a good thing.” I can confirm f
rom my own research experience that very few of the research models used in the research sector are of any real use.
It’s also bad for companies to attach too much importance to statistical techniques when they don’t grasp their true meaning. While presenting the results of brand image studies, researchers summarize their findings on a “perceptual map” (details in chapter 35). These maps are produced by using multivariate statistical techniques. But as Andrew Ehrenberg pointed out, these kinds of analyses provide no tangible benefit to marketing staff; rather, they create an unnecessary complexity that overshadows conclusions that even an executive with no knowledge of statistics could make with the naked eye. The complexity makes it difficult for executives to see the heart of the matter.
Case in point: Such brand perception (image) maps exaggerate the small differences between brands. When interpreting these maps, executives end up attributing some special meaning to the fact that the brands are grouped in one or another part of the map. They easily deduct that clusters of brands are competing among themselves, having no relationships whatsoever with other brands located in other parts of the map. This is an incorrect conclusion. Byron Sharp recommends that every executive who interprets this type of research also look at who is buying the brand. If they do this, it will be easy for them to realize that every brand shares customers with every other brand in the market (details in chapters 31, 33 and 35).
Research doesn’t have to have a snazzy name or include maps and complex statistics. The best research approach is to always use the simplest form possible. Andrew Ehrenberg recommends this approach, one that I adopted after many years of experience in research.