By now, we all know that it is crucial to validate the assumptions of your innovative business idea through experimentation – before you make large investments in production and infrastructure. The “experimentation” we are talking about here is early experimentation – when the business model is still somewhat fluid and the critical questions of the customer’s desire for the product and his willingness to pay are still paper hypotheses. The idea is to learn rapidly and cheaply and not to design the experiments to produce reams of statistically valid data. At this stage it is all about testing hypotheses, being open to learning and to surprises, and then quickly incorporating the learning into your business model rev 2.0.
Coming up with clever, cheap, fast experiments to test the assumptions of your business model is not easy. There is really no prescription for what to use when or where. This is one of the more “creative” exercises within the innovation journey. But the good news is that there are few design principles to guide you and some examples that can hopefully spark some ideas as you design your own experiments.
Experiment Design Principles
Identify assumptions and test independently
The most important thing NOT to do is to build a “scale model” of the whole business concept to see “if it works.” If you only measure success of the “scale model,” how can you figure out what parts worked and what parts didn’t? The first step in experimentation, and the most crucial, is to clearly identify the most important assumptions baked into your business concept and devise a way to test each of this independently. This shouldn’t be a long list – what are the 2 or 3 most important assumptions that the success of your business depends on? The first assumption you need to focus on involves the customer need and her willingness to pay. If the customer doesn’t want what you are offering, there is no point in testing other assumptions until you can provide an offer that customers want or even desire.
Make your customers pay – something
The best experiments require an exchange of value between you and the customer. We all know of the unreliability of focus group participants stating that they intend to buy something. A more reliable indicator is when the customer has to “pay” something for the product. The payment doesn’t need to be money. Customers can “pay” by providing personal information (like their email address), with reputation (liking your product on Facebook), with time (attending a demo of your product), or with other forms of currency. An interesting experimental tactic to reliably gauge willingness to pay is to have online customers begin the checkout process for your item, only to find out that your item is out-of-stock.
Fast and cheap … and faster and cheaper
Experiments should be designed for learning, not for success. Your objective should be a laser-like focus on validating (or not) your most crucial assumption, and not on designing in a lot of other “nice to have” features. Once you have an initial experiment design, ask yourself: How could I make it 10 times cheaper? How could I accelerate it? Keep in mind that the goal is not statistically valid data at this point. (This will come later as the business concept evolves and the experiments grow into more formal pilots or test markets). Limit your investments in infrastructure – manually perform the tasks you will eventually automate. An appliance manufacturer that was testing the customer appeal of “customized” refrigerators at a big box store used their own employees to customize the refrigerators in the store’s back room with a small set of customize-able features like colored door panels, different size storage bins, and others. Use a “human backend processor” rather than build the software. Zappos, in its earliest days, simply purchased shoes at local retailers to offer online rather than invest in their own inventory.
Examples
A few examples highlight these principles and hopefully, provide some ideas to design your own experiments.
Breakfast Bars
A large food company was testing the idea of oatmeal squares as a breakfast alternative to a bowl of oatmeal. One of the key assumptions being tested was that a square was a more appealing format than a round breakfast treat. The team sold the product, unpackaged, at deli counters in three grocery stores in the Chicago area. They offered two different shapes (square and round) for 30 days and conducted follow-up interviews with buyers. What they found was that the square was better than the round (because it looked less processed) – and that consumers thought of it more as a portable breakfast food rather than a “hot meal replacement.” So, without investing in packaging the product, branding the product, or securing shelf space, the team validated an important hypothesis quickly and cheaply at the deli counter.
Best Buy’s Technology Trade-In
An innovation team at Best Buy believed that one of the barriers to buying new electronics was the consumer’s need to somehow dispose of the old TV’s, stereos, etc. Their idea was to offer a Technology Trade-In program where customers could cash in their old electronics for Best Buy store credit. To determine if this idea was even appealing to customers (and whether it led to increased sales); the team designed a fast and cheap prototype of the concept. The Technology Trade-In program was staged as tent event in a Best Buy parking lot (to take advantage of the foot traffic); it was advertised through “door knob hangers” printed at a local copy shop in local neighborhoods; the products were valued instantly (by looking up comparables on eBay) and traded in for store credit. All of the electronic products turned in were sold on eBay. So with no infrastructure (other than borrowing eBay’s) and little cost, their original assumption was validated and the concept eventually evolved into Best Buy’s online tech trade-in program.
Meal Kits
A food company wanted to test “meal kits” that were intended to teach consumers how to prepare ethnic meals. An important assumption in this product concept was that consumers were willing to invest the time to learn to cook. To test this assumption, the team rented out a cooking school kitchen and invited target customers for a relaxing evening of food and fun. When the customers entered they had to choose between 2 different cooking experiences. In Class A, the customers watched a demo of a chef preparing an ethnic meal using the meal kit. In Class B, the customers actually cooked the meal themselves using the same ethnic meal kit. The number of customers opting for the “cook it yourself” class was an indicator of the customers’ willingness to invest the time in learning to cook. As an added twist, the customers in Class B were asked about the items currently in their freezers, and if they would trade those frozen items for meal kits. In this example, the exchange of value was time (spent at the event) and the trade of items in their freezers.
In all of your experiments, you are seeking deeper understanding of the assumptions and of the overall concept. It is not useful to simply conclude that the assumptions are invalid and customers don’t like the concept. Don’t simply collect the data – be on the lookout for the unexpected and for any surprises that provide insight and clues for the next iteration of your concept.
