This list of painful failures included my own, those of my friends and colleagues, and many that were reported in business articles and the news. (One of the good things about researching market failures is that there is never a shortage of examples.) After a few weeks of being on the lookout for examples of pretotypes and market failures, I had collected a short list of pretotyping techniques and a long list of The Wrong It failures. And that’s when things got really interesting.
I took each of those failures and asked: Could this market failure have been prevented by one or more pretotyping techniques? To put it another way, could we have learned, before getting in too deep, that the premise for this product was wrong (the product was The Wrong It), using some creative pretotyping techniques?
In almost every case the answer was a clear and resounding yes! Most of those painful and costly failures could have been easily prevented with well-planned and -executed pretotyping experiments. No approach or set of tools can give you a 100 percent guarantee, but if properly used, pretotyping tools will help you determine if an idea is The Right It or The Wrong It faster and more reliably than any market-research approach based in Thoughtland.
If you think that this sounds too good to be true, I don’t blame you. That was my initial reaction. I am a skeptic by nature; but after several years of using, coaching, and teaching these tools and techniques, I am convinced that they work. But don’t take my word for it. After all, I am biased, and my experience is at best OPD and at worst anecdotal—two types of data I’ve warned you not to depend upon. So I say, “Don’t trust me. Test me!” The best way to convince yourself of the logic and power of pretotyping is to experience it firsthand. Go get Your Own DAta.
In the following pages I will introduce you to pretotyping techniques that, alone or in combination, can be applied to collect valuable YODA to help you validate any new product idea. If you’ve ever worked on a new product that failed in the market because it was not The Right It, you will probably discover one or more techniques that might have prevented that failure.
The Mechanical Turk Pretotype
The Mechanical Turk pretotype borrows its name from the famous Mechanical Turk chess-playing “machine” that toured the world in the late eighteenth century. People were led to believe that the “Turk” was a mechanical contraption (an automaton) programmed to play chess. In reality, however, the box concealed a small expert chess player making the moves by manipulating the mannequin.
A Mechanical Turk pretotype is ideal for situations where you can replace costly, complex, or yet-to-be-developed technology with a concealed human being performing the functions of that supposedly advanced technology.
Sound familiar? It should. The IBM speech-to-text experiment with which I started this chapter is a great example of a Mechanical Turk pretotype in action. Developing a good enough speech-to-text engine would have taken years and a huge investment. But a human typist, hidden in another room the same way the chess player was hidden inside the Mechanical Turk device, easily simulated that complex function and allowed IBM to collect the YODA it needed.
Let’s look at another example where a Mechanical Turk pretotype can help us validate our idea.
Example: Fold4U
Most coin laundries have machines to wash clothes and machines to dry them. But at the end of the drying cycle, we have to first sort out a jumbled pile of assorted garments and then fold and stack them by hand. We have self-driving cars, but we still have to fold clothes manually? That’s unacceptable! Okay, perhaps that’s a bit of an exaggeration, but wouldn’t it be great if there were a clothes-folding and -stacking machine to take care of that last step?
Ivan the inventor believes that he can build such a machine, and he’s convinced that, by leasing it to coin laundries for a fixed monthly price plus a per-use fee, he can turn piles of clothes into piles of cash. All he needs is $50,000 and about six months to build a proof-of-concept prototype. Ivan does not have the money, because the invention from his previous venture—the RoboDogWalker—did not sell as well as he had anticipated. So he offers Angela, a friend who also happens to be an angel investor, a 25% equity stake in his new company, Fold4U, in exchange for the $50,000.
Angela has full confidence in Ivan’s technical prowess. She knows that if Ivan says that he can build an automatic clothes folder and stacker, he will. But she is not at all confident in Ivan’s business model and financial projections for Fold4U. Ivan’s plan is based on the premise that most coin laundry customers would be willing to pay an extra $2 to $3 to have their clothes folded and stacked.
When Angela calls Ivan’s Market Engagement Hypothesis into question, Ivan gets defensive: “It’s not an assumption, Angie. I’ve done my market research. I’ve interviewed 632 coin laundry users while they were folding their clothes. And 421 of them told me that they hated doing it and would gladly pay a few extra dollars if there were a machine that could do that for them.”
“Isn’t that the same kind of survey you did for RoboDogWalker?” Angela asks.
Ivan’s face turns beet red. He responds, “Look Angie, if you are not interested in investing in Fold4U, just tell me. There’s no need to humiliate me. I know that RoboDogWalker was a failure, and the unfortunate accident with the poodle didn’t help. But this is a much better—and less risky—idea and a completely different market.”
“Ivan, I am interested,” replies Angela. “Quite interested, in fact. I can definitely see the potential for Fold4U. But before I invest $50,000 to build a prototype, I need stronger evidence that all those coin laundry customers who say they’d pay for the service will actually pay for it. I want to see those people put their clothes in the machine and pay for it.”
“That’s why I need the money to build the prototype, Angie. How can we test that people would pay for the Fold4U if we don’t have one available?”
Before you read on, spend a couple of minutes thinking how you would reply to Ivan if you were in Angela’s shoes. Would you give Ivan the $50,000? How could you use a Mechanical Turk pretotype to validate Ivan’s MEH? C’mon, give it a shot. It’s simple, and I am confident you’ll get it right.
* * *
All right. I hope you tried the exercise. If you did, compare your answer to what Angela and Ivan came up with.
Angela shares with Ivan the IBM speech-to-text story. By the time she is done, Ivan’s face is still red, but for a different reason. His irritation has changed into excitement. “That was so clever of those IBM folks. I can’t believe I never heard this story before. I think we can use this prontotyping thing to test Fold4U.”
“It’s called pretotyping,” says Angela with a laugh, “but considering how quickly it gets you results, prontotyping would also be a good name.”
After some discussion, Ivan and Angela come up with the XYZ Hypothesis:
At least 50% of coin laundry customers will pay $2 to $4 (depending on location) per load to have their clothes folded and stacked.
Then they hypozoom into and decide to test the following xyz hypothesis:
At least 50% of Lenny’s coin laundry customers will put their clothes in a Fold4U machine and pay $2 to have them folded.
And now the fun starts. Ivan meets with Lenny (a local coin laundry owner), explains to him the Fold4U idea, and offers him $200 to let him run a pretotyping experiment in his shop. Lenny agrees to the deal, and since he’s just as excited and interested in the idea as Ivan, he also agrees to help Ivan set up and run the experiment. He even gives Ivan an old broken clothes dryer—a perfect prop for the experiment.
Ivan modifies the old dryer by replacing the drum with a compartment that has a hidden door in the back. This way, after people put their clothes in the machine and pay, Ivan can open the hidden back door, pull out the clothes, fold them by hand, and put them back. To complete the illusion, Ivan makes a recording of mechanical noises that he has playing inside the machine while he manually folds the clothes. At the end of the “fold cycle,” he rings a litt
le brass bell from inside the machine to alert the customer that the clothes are done.
The pretotype works so well that none of the users suspect a thing. They all believe their clothes are being folded by some kind of robot. But although most of Lenny’s coin laundry customers are intrigued by the new machine, few choose to use it. And most of those who do admit to doing so out of curiosity. The initial pretotyping experiment falls well short of expectations:
xyz hypothesis: At least 50% of Lenny’s coin laundry customers will put their clothes in a Fold4U machine and pay $2 to have them folded.
YODA: 12% of Lenny’s coin laundry customers paid $2 to have their clothes folded by a Fold4U.
Just to make sure, over a period of two weeks, Ivan runs a few more experiments using different prices and different coin laundries. Unfortunately, the results don’t change much—even when the price is dropped to just $1. In Thoughtland, people said (and probably believed) that they would pay $2 to $4 for such a service, but when the time came to actually put some skin in the game (in the form of coins in the clothes folder), very few did.
Does this mean that there is no chance for Fold4U? Not necessarily. But Ivan’s plan and business model was based on 50% or more of coin laundry users paying for the machine; if the actual number is less than 15%, he will have to revise many of his assumptions if he hopes to convince investors like Angela to back him.
Ivan may be disappointed that Fold4U is unlikely to be The Right It, but he’s also relieved that, unlike RoboDogWalker, he did not waste two years of his life as well as a big chunk of money to learn that lesson.
What do you say, Ivan?
“Thank you, pretotyping!”
* * *
Did I hear some of you say that, after all this talk about failure, you would like a happy ending once in a while? Well, I might argue that preventing Ivan from another business disaster like the RoboDogWalker is a happy ending, but I know what you mean; you want a proper, Hollywood-like happy ending. No problem. You deserve it. Here you go.
Alternative Ending: Not only are most of Lenny’s coin laundry customers intrigued by the new machine, they are lining up to use it. In fact, the new Fold4U is quite the sensation. People applaud every new batch of folded clothes that comes out. Everybody wants to see and use the new machine. But, of course, it’s not a machine doing the folding, but poor Ivan. The next day, just as his arms were ready to fall off from all that folding, Ivan ends the experiment. He puts an “Out of Order” sign on the pretotype and goes to meet Angela. This time, instead of Thoughtland surveys, he presents her with fresh YODA:
xyz hypothesis: At least 50% of Lenny’s coin laundry customers will put their clothes in a Fold4U machine and pay $2 to have them folded.
YODA: 78% of Lenny’s coin laundry customers paid $2 to have their clothes folded by a Fold4U.
Angela is thrilled. To make sure the great results aren’t a fluke, she and Ivan agree to hire part-time assistants to help with the folding and to run additional experiments. As the novelty wears off, market engagement drops a little (it turns out that some people were more curious to see if and how the machine worked than in using it regularly). But the average market engagement (the number of coin laundry patrons who pay to have their clothes folded by the Fold4U after drying them) stays at a healthy 62%—providing strong confirmation for Ivan’s 50% or more projection in the XYZ Hypothesis.
People said that they would pay for such a service, and when the time came to actually put some skin in the game, they did. In this case, YODA matched opinions. It happens. Just not as often as we’d like, and that’s why we need to test the ideas.
Angela decides to invest in Fold4U, and by using the impressive YODA from the pretotype, Ivan is able to increase his company valuation and recruit additional investors. Not only that, but when the time comes to market and sell the Fold4U, Ivan can provide coin laundry owners with a compelling business case: “Our data shows that over 60% of your customers will pay an extra $2 to $4 per load to have their clothes folded. This will increase your total revenue and profits by at least 20%.” Sold!
What do you say, Ivan?
“Thank you, pretotyping!”
As these two different endings show, investing a little time and resources to pretotype an idea is a win-win tactic:
If YODA from the experiments does not validate your hypotheses, pretotyping will save you from a very likely failure.
If YODA confirms your hypotheses, you will be in a better position to recruit partners, secure investors, and convince potential customers.
Every idea deserves to be pretotyped, and there’s (at least) a pretotype for every idea—so let’s give the Mechanical Turk (and Ivan’s arms) a well-deserved rest, and let’s explore a few more pretotyping techniques.
The Pinocchio Pretotype
The Pinocchio pretotype is named after the beloved fictional character Pinocchio, the wooden puppet who dreamed of becoming a real boy. You will understand why I picked this name after I share with you the example that inspired it.
Example: The PalmPilot
In the mid-1990s, brilliant innovator and entrepreneur Jeff Hawkins had an idea for the personal digital assistant (PDA) that would eventually become the PalmPilot. But before committing to it and investing in building an expensive prototype, which would have required a full team of engineers and a lot of time and money, he wanted to validate some of his assumptions about the device. He knew he could build it, but would he use it? What would he use it for? And how often would he use it?
His solution was to carve a block of wood to match the intended size of the device, whittle down a chopstick to make a stylus, and use paper sleeves to simulate various user screens and functions. He carried the block of wood in his pocket for several weeks and pretended that it was a functional device in order to get insights into how he would use it. If someone asked for a meeting, for example, he’d pull out his wooden block and tap on it to simulate checking his calendar and scheduling a meeting reminder.
The PalmPilot wooden model on display at the Computer History Museum, Mountain View, CA.
With the help of his pretotype, Hawkins collected valuable YODA. He learned that he would actually carry such a device with him and that he would be using it mostly for four functions: address book, calendar, memo, and to-do lists. His simple experiment provided him with enough YODA to convince him that he would love to have a working version of the device. He knew, of course, that a sample size of one (himself) was not sufficient to determine if other people would respond to the Pilot the same way he did. He would have to follow up this test with additional experiments to validate the rest of the market. But the idea passed an important first test: its own inventor found it useful. This may seem a trivial threshold to pass, but you’d be surprised how many people bring to market a product without first validating that they themselves would use it.
The data collected from the simple wood and paper pretotype helped to guide and justify the much greater investment needed to develop a proper working prototype. Not only did the PalmPilot become incredibly successful, it also paved the way for smartphones and established a form factor (i.e., shape and size) for most portable electronic devices that continues to this day. Pinocchio was a wooden puppet who dreamed of becoming a real boy. The PalmPilot pretotype was a wooden PDA that Jeff Hawkins dreamed might one day become a real product. Both dreams came true.
In addition to being a great example of a powerful pretotyping technique, the PalmPilot story also illustrates some of the key concepts I’ve been emphasizing. Below is how TIME magazine reported it in March 1998. I’ve marked some of the key points in italic:
Hawkins, 40, Palm’s chief technologist and Pilot’s creator, designed one of the first handheld computers, the GRiDPad, a decade ago. It was an engineering marvel but a market failure because, he says, it was still too big. Determined not to make the same mistake twice, he had a ready answer when his colleagues asked him how small their new device should be: �
�Let’s try the shirt pocket.”
Retreating to his garage, he cut a block of wood to fit in his shirt pocket. Then he carried it around for months, pretending it was a computer. Was he free for lunch on Wednesday? Hawkins would haul out the block and tap on it as if he were checking his schedule. If he needed a phone number, he would pretend to look it up on the wood. Occasionally he would try out different design faces with various button configurations, using paper printouts glued to the block.*
This story embodies the core motivations for and principles of pretotyping:
Hawkins’s painful experience of spending years and millions to produce the GRiDPad, a product that turned out to be “an engineering marvel but a market failure.”
His realization that the mistake was not that he built It wrong, but that he had built The Wrong It.
A commitment “not to make the same mistake twice.” In other words, he told himself something along the lines of, “Next time, make sure that you are building The Right It before you build It right.”
The creation of a first pretotype—not to test if the Pilot could be built, but to test if, how, and how much one would actually use it, by collecting firsthand YODA to inform design decisions for the actual prototype and eventual product. For example: — Carried the device in my pocket 95% of the time.
— Pulled it out to use it an average of 12 times a day.
For scheduling appointments: 55% of the time
To look up phone numbers or addresses: 25% of the time
To add to or check a to-do list: 15% of the time
To take notes: 5% of the time
Using his imagination (i.e., pretending) to fill in the missing functionality using a dummy of the envisioned product as a prop.
The Right It Page 8