How to increase the accuracy of consumer experiments

    Strategos

    Why is it that test markets for a new product concept are often very enthusiastic, but that enthusiasm fails to materialize at launch? As entrepreneurs, marketers, and product developers know, consumers are notoriously fickle in their responses to new products and often are not able to give accurate feedback regarding their reactions and purchase intent. Far too often, we hear about wildly successful experiment results followed by pilot and launch phases that fail to substantiate the excitement of the initial testing.

    So, what went wrong? We’re all familiar with the standard modes of testing with customers – observing in-context usage, filling out questionnaires, conducting a focus group. Essentially, these forms of engagement require that users interact with a product, and then provide their initial feedback and reactions. The problem is that a verbal commitment is not much of a trade-off. It’s much easier to give approval through a survey than to actually pull out your wallet and part with your hard earned cash.

    An element that we work with our clients to instill in their experiments is the concept of Value Exchange. Essentially, this is an embedded step in the test that asks users to go beyond verbal responses to something a bit more substantial. By asking users to trade something of value, we get a more accurate picture of their response to the concept. Let’s look at a few examples:

    Put your possessions on the line
    Want to see if a consumer truly enjoys your new product? Challenge them to trade something they currently own for one that you’re offering. One frozen foods maker ran a focus group to demonstrate their new concept. At the end of the discussion, participants who said they liked the product were then asked to exchange the current contents of their freezer for the new item. The client made an offer to come by their house and make an exchange. In the end, the new item was offered for free, but the simple act of asking for an exchange peeled back the true intent of these users.

    Vote with your feet
    A company testing new concepts in ready-made foods wanted to test whether consumers would truly be interested in a new DIY concept. They set up 2 environments for participants to choose from – (A) watch as a celebrity chef prepares the meal, or (B) participate in a walk-through to create the meal at your own cooking station. This simple act of voting by participants helped gauge the initial reactions of participants, in addition to the observations noted throughout the demonstration itself.

    Invest with funny money
    An old standby that essentially leverages the wisdom of the crowd, we’ve seen several companies leverage both internal and external markets to garner votes on product concepts. The idea is simple, distribute a set number of dollars, votes, or points to a group of people and ask them to place their assets on the concept they feel has the most legs.

    We’ve shown just a couple of examples of how the accuracy of your experiment results can be raised by asking participants to interact and make a trade-off beyond a verbal response. These can range from simple to elaborate trade-offs but in all cases there needs to be an exchange of value for the experiment to be meaningful, provide more accurate results and ultimately increase the chances of launching a successful product or service.