What Makes a Pretotype a Pretotype
As you have seen, pretotypes can take many forms. But to be worthy of the name, a pretotype must satisfy three key requirements:
A pretotype must produce YODA with skin in the game.
A pretotype can be implemented quickly.
A pretotype can be implemented cheaply.
But even if you stick to these requirements, you will have a large number of pretotyping techniques and combinations to choose from. Which raises a few questions:
How do you choose which pretotypes to use?
How many different experiments do you need to run?
How much data do you need to collect?
When can you stop testing?
To help you answer those questions, you need a final set of tools, analysis tools, which are the focus of the next chapter.
6
Analysis Tools
You have learned how thinking tools, like the XYZ Hypothesis and hypozooming, help you transform a fuzzy and poorly articulated idea into a clear, objective, and testable set of hypotheses. You have also learned how pretotyping tools, like the Mechanical Turk or the Fake Door, make it possible for you to collect data about your idea by testing those hypotheses quickly and inexpensively. Now you will learn how analysis tools can help you make sense of the data you collect, so you can make that important transition from data to decision.
The Skin-in-the-Game Caliper
I’ve already introduced and casually used the term skin in the game a number of times. Now it’s time for a more thorough introduction and discussion of this key concept along with examples of how to put it into practical use.*
It’s not clear who is responsible for coining the term skin in the game. Some people attribute it to legendary investor Warren Buffett; others have found no evidence to support that claim. For the longest time, I thought it was a phrase used by fur trappers playing poker and betting animal skins instead of dollars. (Not a bad guess given that the slang term bucks—as in dollars—comes from buckskins, which were considered a form of currency at some point.) Despite a lack of consensus about the origins of the term, most people agree on its meaning. Generally speaking, in the expression skin in the game, the word game is a metaphor for some activity with a possibility of winning or losing, and skin is a metaphor for having something of value at risk in that game (e.g., money, time, or reputation). In the context of this book, the game consists of bringing a new idea into the market, and winning or losing is determined by whether that new idea succeeds or fails in that market.
Unlike games such as chess or football, our game is not a zero-sum game: one new product’s loss (market failure) does not necessarily mean that another, similar product will win (market success). It is also extremely rare for any one product to capture 100% or even 90% of any one market and continue to dominate that market for decades. In other words, it’s a tough game to win and, as we’ve seen, most people lose at it most of the time. If you lose (your idea fails in the market), you lose all or most of your investment (skin). But if you win (your idea succeeds in the market), you get your investment (skin) back along with a few—and sometimes a lot of—extra pelts. And it’s those extra pelts that make the game and its risks worthwhile and fun for many of us.
Entrepreneurs, inventors, and investors in a new idea automatically have skin in the game and typically lots of it. If Mark quits his good job, takes on a second mortgage, and works eighty hours a week to start a new business, he’s putting all kinds of skin in the game, including money (giving up a salary, taking on a loan) and time (eighty hours a week)—he’s risking a lot.
When Mary convinces her company to invest in her new invention to let her lead its development, she may not be using her own money and the company is paying for her time, but she’s investing (and risking) her reputation, future promotions, and earning potential if the invention fails to meet the expectations she set for the company. When venture capitalists or angel investors decide to invest in an idea, they are investing and risking not only money and time, but also their reputation in the VC community.
As an entrepreneur, inventor, or an investor in a new idea, you have no choice but to have some and often a lot of skin in the game. Everyone knows and expects this—and it’s the way it should be. Having skin in the game means that you have something at risk, something significant to gain or lose, depending on the outcome. This skin is an indication that you are serious about the idea, that you have done your homework, and that you will deal with challenges as they arise rather than abandoning your idea at the first sign of trouble. Skin in the game indicates commitment, seriousness, and thoughtfulness.
But make sure that you don’t put too much skin in the game to develop an idea unless and until you can demonstrate that the market is interested enough in the idea to put up some skins themselves. Not opinions, not predictions, but skin!
Until and unless individuals put some of their own skin in the game, you can’t count those people as likely customers or users. Rather, they are merely spectators—they’ve got nothing to lose. Heck, they might even enjoy watching you and your idea crash and burn.
When it comes to gauging an idea’s market potential and likelihood of success, our deliberation must be based on hard data, and the data must come with some skin in the game. I’ll say it again, because this is so important: before you put a lot of skin in the game on your idea, make sure you can get some skin in the game from your target market.
But what qualifies as skin in the game from your target market? And how much of it do you need?
To help you answer those important questions, I’ve developed a Skin-in-the-Game Caliper that you can use to assign skin-in-the-game “points” to different types of target market responses. The caliper can be customized for specific products and markets.
Example: The Tortell-o-matic
Let’s assume that my idea is the Tortell-o-matic, a $250 automated tortellini maker for the home. Fill the Tortell-o-matic with flour, water, eggs, salt, and your choice of filling, and two minutes later this amazing machine starts to pop out perfectly shaped, freshly made tortellini. Yum!
In Thoughtland (and tummyland) the Tortell-o-matic scores great. Many of my friends and colleagues love the idea and say that they would definitely buy a Tortell-o-matic for their home (yeah, right!). Unfortunately, designing, developing, and manufacturing the Tortell-o-matic will require a big investment. Even a one-off prototype is likely to take months and cost tens of thousands of dollars. That’s a lot of my own skin in the game. So before I undertake that investment, I need something more tangible than opinions and promises from my target market. I need to see some YODA with some of their skin in the game in it.
I can use one of my favorite pretotyping techniques, the Mechanical Turk, to set up an experiment and collect that YODA. A dummy Tortell-o-matic sits on a table. Under the table and hidden by a tablecloth, an accomplice holds a bucket of premade tortellini. When I push a button on the nonfunctioning machine, the accomplice plays a recording of mechanical noises and begins to push tortellini through the machine. I can use this pretotype to collect YODA in a live demo, an online demonstration, or both.
The question is, after people get to see the Tortell-o-matic in action, how do I assign skin-in-the-game points to the responses I get from the market? That’s where a Skin-in-the-Game Caliper comes in handy. As an example, here’s the Skin-in-the-Game Caliper for my Tortell-o-matic:
Type of Evidence Examples Skin-in-the-Game Points
Opinion (expert or nonexpert) “Great idea.” “Nobody will buy it.” 0
Encouragement or discouragement “Go for it!” “Keep your day job.” 0
Throwaway or fake email address or phone number [email protected], (123) 555–1212 0
Comments or likes on social media “This idea sucks,” thumbs-up or thumbs-down, Like 0
Surveys, polls, interviews online or off “How likely are you to buy on a scale of 1–5: ___.” 0
A validated email address with the explicit understanding that it will be used for product updates and information “Give us your email to receive updates about the product: ____.” 1
A validated phone number with the explicit understanding that you will be called for product updates and information “Give us your phone number so we can call you about our product: (__) __-___.” 10
Time commitment Come to a 30-minute product demonstration 30 (1pt./min.)
Cash deposit Pay $50 to be on the waiting list 50 (1pt./$)
Placing an order Pay $250 to buy one of the first 10 units when available 250 (1pt./$)
As you can see, I am brutally selective when it comes to assigning skin-in-the-game points. For example, I ignore not only random opinions, but also so-called expert opinions. Why? Because we know from experience that all too often experts don’t know any better than nonexperts. For example, the people at Ford who came up with the idea for the Edsel, one of Ford’s biggest commercial failures, were experts in the automotive market with many previous successes under their belt. Similarly, few companies have more expertise in developing internet products than Google, and yet Google failed with Google Wave, Google Buzz, and many other internet-based ideas. At the same time, many ideas that experts considered to be hopeless turned out to be huge successes. Author J. K. Rowling, for example, received many rejections by the world’s most experienced publishing houses before an editor took a chance on the first Harry Potter book. That’s why opinions—of any kind and from anyone—rate a zero on the caliper.
I also assign zero points to online likes, thumbs-ups, thumbs-downs, tweets, retweets, comments on social media, polls, and surveys of all types. It’s too easy to mindlessly click on a like, make a comment, or write a tweet. No skin = no points.
The smallest piece of evidence I consider data in this example is a validated email address given willingly to you by its owner with the understanding that it will be used to contact the person for offers, updates, or information about your specific product idea. By validated, I mean that it must be a bona fide email account that the person actually checks and uses regularly, not the kind of throwaway email address people often give when they are forced to do so. Based on my experience (and probably your own) most people are protective of their main email address and cautious about whom they give it to, which means it has some value to them. Because of that, I consider a validated email address—as long as it’s knowingly and willingly* given to you for the purposes of being informed about your specific product—to be skin in the game, but the very smallest possible unit of it: 1 point.
A validated phone number, on the other hand, I value at 10 points. Why so much more than an email address? Because most people are at least an order of magnitude more protective and careful about sharing their phone number than they are about sharing their email address.
I value a person’s time at 1 point per minute. If someone is willing to invest 30 minutes to listen to a new product presentation, they get 30 points of skin in the game, because it’s a reliable indication that they have a bona fide interest in the product.
Finally, we come to the ultimate type of skin in the game: buckskin—good ol’ money. As the saying goes, “Money talks, opinion walks.” All right, that’s not exactly how the saying goes. I’ve taken the liberty of replacing bovine dung with something even less appealing and useful: opinions. At least manure can be used as fertilizer. When it comes to money, I keep things simple: $1 counts for 1 skin-in-the-game point.
Feel free to substitute your local currency and adjust accordingly for your own idea or market, but don’t try to be too precise (e.g., an email is worth .4 points and a phone number 3.7). Instead, think in terms of orders of magnitude (e.g., 1, 10, 100) and use your intuition and experience to guide you in making those determinations. The distinction and clear separation between no skin (zero points) and skin is far more important than being precise with the points. Just make sure you are honest, objective, and reasonable in assigning relative values: for example, a $50 deposit should count much more than a validated email address.
Example: A Tale of Two Teams
I use the Skin-in-the-Game Caliper all the time when students or clients come to me with what they think is data. Here’s a sample scenario.
Imagine that two teams, Team A and Team B, come to you with two different opportunities to invest in a new product idea. Team A makes their case by showing you a flashy YouTube video demo of their future product in action. Then they flaunt a bunch of skinless metrics:
In one week we got 140,000 views, 20,000 thumbs-ups, and only 100 thumbs-downs. And look at these comments: “Awesome,” “This will be great!” “Amazing idea” . . .
Team B also has a video, but instead of using total views, or thumbs, or comments to make their case, they present their data as follows:
We made a two-minute video to demonstrate what our product does and spent $400 on online ads to draw traffic to it. In one week, our video was seen in its entirety by 8,000 people. At the end of the video we gave people an opportunity to give us their email address to be informed when the product will launch. We received 120 email addresses, 40 of which were not confirmed, so we only counted 80 as valid. A week later we sent a follow-up email to those 80 people giving them the opportunity to buy a prerelease (handmade) version of the product for $125 (a 50% early-adopter discount), and we received 20 orders.
Using the above Skin-in-the-Game Caliper, how many points does Team A get? That’s easy. Zero! Goose egg! To an untrained eye, the absolute numbers presented by Team A may be more impressive than those of Team B (i.e., 140,000 views vs. 8,000), but they are not good YODA. Team A did not even tell us how much time or ad money they spent to get those 140,000 views.
Team B, on the other hand, gave us not only numbers, but also the context in which to evaluate them and, most important, their YODA comes with some skin in the game:
$400 in ads → 8,000 views = 0 skin-in-the-game points (no points for views, but an interesting first estimate for how much it will cost to get a view, i.e., $0.05/view)
8,000 views → 80 validated email addresses = 80 skin-in-the-game points
80 emails → 20 orders for $125 each = 2,500 skin-in-the-game points
It’s entirely possible that Team A’s idea is The Right It and that it will be a smashing success. They have a lot of impressive-looking numbers to share, but to be convinced I’d need to see real data—YODA with skin in the game. I’d give them a copy of this book and ask them to come back to me with some pelts.
On the other hand, Team B has given me real and useful data: a $400 investment in ads resulted in 80 validated email addresses ($5 per address) and $2,500 in orders. I would, of course, want to run a few more experiments (I’ll show you why and how in a bit), but it’s a promising beginning.
All data is not created equal—even YODA. Team A focused on quantity, while team B focused on quality. Quality of data beats quantity of data, and there’s no better indication of data quality than data that comes with a lot of skin in the game attached to it.
The TRI Meter
Collecting YODA with skin in the game to validate our Market Engagement Hypothesis is a necessary first step, but raw data by itself is not sufficient. In order to extract value out of data and use it to make rational and well-informed decisions, we need a way to interpret it, put it on a scale, compare it, and combine it with other relevant data.
The data from a cholesterol test, for example, is just a ratio of two values, the number of milligrams of cholesterol per deciliter of blood. Let’s say you go for your annual physical exam, take a blood test, and learn that your total cholesterol is 300. That number by itself is not very meaningful, but when the doctor pulls out a chart and shows you that, statistically speaking, people with a cholesterol level of 300 are 4.5 times as likely to die of heart disease than those with a cholesterol level of 200, you may decide to lay off cheeseburgers for a while.
The Right It Meter, TRI Me
ter for short, is a visual analysis tool I developed to help you interpret the YODA you’ve collected as objectively as possible. More precisely, the TRI Meter is a gauge to help you estimate how likely it is that an idea will succeed in the market, but it’s a gauge that is not too technical, complicated, or confusing—which can easily happen whenever probability and statistics are involved.
First let me show you what the TRI Meter looks like in action. Then I will explain how to use and interpret it. The image below shows a TRI Meter after four pretotyping experiments (represented by the four white arrows on the right). As you can see, the TRI Meter scale is partitioned into five likelihood-of-success categories ranging from Very Unlikely (10% chance of success) to Very Likely (90% chance of success), each representing the likelihood that your idea is The Right It.
Why only five categories and not seven or ten? I’ve debated whether to add more categories (e.g., Extremely Unlikely and Extremely Likely), but that would only add complexity and suggest greater precision than our experimental tools and subjects (i.e., people) can provide. It would also bring us uncomfortably close to a feeling of certainty in an environment in which there are no guarantees. That’s why the scale goes from 10% to 90% in broad 20% increments instead of 0% to 100% with finer increments. I am a big fan of “say it with numbers,” but asking for too much precision or too much confidence in these kinds of predictions reminds me of the following lines from Dr. Seuss’s wonderful 1990 book Oh, the Places You’ll Go:
And will you succeed?
Yes! You will indeed
98 and 3/4 percent guaranteed.
Those lines always bring a smile to my face. Unfortunately, I can’t give you anything as precise as that 98 and 3/4 percent guarantee—nobody (with the possible exception of Dr. Seuss) can. And I am pretty sure that the good doctor himself could not have predicted that Oh, the Places You’ll Go would sell over 5 million copies and still be a bestseller two decades later—talk about a book that is The Right It.
The Right It Page 13