Pre-cooked Experimentation Environments: How to accelerate experimentation through bundled infrastructure

    Strategos

    Several years ago, while living in New York, I turned to a convenient method of grocery shopping – FreshDirect’s online service.  One great feature I found was the ability to peruse a recipe, view all the necessary ingredients for that particular recipe, and add them all to my shopping cart with 1-click.  Ready to cook meals, bundled ingredients, and pre-seasoned entrees are not new concepts.  But considering my hectic work schedule and novice cooking skills, I was more than happy to take advantage of these delicious prepared bundles that still allowed me to partake in the preparation of the meal.

    A client we have been working with recently instantiated a pre-cooked set of “environments” for rapid experimentation of new business ideas.  Essentially the organization acted as the prep cook, making sure all the raw materials for the experiment were available for use.  The beauty of this system is that teams of brand managers and product developers can focus on the design of their experiments rather than constantly reinventing the wheel by having to gather resources from scratch for every experiment.  Large corporations have an advantage in this respect – they can commit resources to building infrastructure that teams can leverage and re-use quickly and easily.  Any business unit was welcome to use these experiment environments to test, learn, and validate a business concept they were exploring.

    Strategos’ role was to help the organization think through the key elements of setting up a range of proper environments for rapid testing of ideas.  This included the infrastructure as well as the knowledge management and governance required to give teams the ability to use these effectively.  Here are some of the key elements needed to set up experiment environments.

    The infrastructure:

    • Context and ambience – Whether online, in a focus group, or with a retail partner, you’ll need some way to put customers in the mindset of the situation they’ll be facing.  When do they face the need you are targeting?  Make sure each environment can flexibly communicate a few critical cues to set context for participants.
    • A ready source of test participants – Each environment needs to be ready to tap into a source of users or the ability to gather them quickly. Often, there’s a temptation to grab any available person, but keep in mind that to start, it’s best to find participants as close to your intended target as possible.
    • Designers and prototypers – Unless it’s a thought experiment, you’ll need a couple of creative designers that can whip out a mockup or prototype of what you intend to test.  Depending on your concept and type of experiment, you’ll want designers that can deploy a wide range of medium, e.g. virtual concept, mockups, simple mechanical prototypes, etc.
    • Expedited governance and administration – Of course the more resources there are the more controls that exist.  To be effective, your teams will need access to expedited approval processes as well as a safe haven to take risks and learn from failure.  To ensure these pathways, Strategos deploys a set of tools to identify potential barriers and help institute a governance framework.

    The know-how:

    • Design principles of experimentation – We found it helpful to provide a set of templates and guidelines that emphasized key experiment design principles.  For example, identifying and prioritizing critical assumptions, emphasizing learning before earning, and employing a value exchange with participants.
    • A recipe book of case logs – To spark some thought starters and ramp up the learning curve, we suggest providing analogues and case studies, as well as more mechanical examples of measurement types and observation results.  These items are helpful in sparking experiment ideas and familiarizing teams with the process.
    • Experimentation sherpas – Of course, it’s helpful to have a few people that are knowledgeable and experienced to help guide teams through questions and obstacles they may face. Often, teams will benefit from design guidance, how to interpret quantitative and qualitative results, as well as navigating buy-in from senior managers.

    Ultimately, having a base set of 5-7 experiment environments allowed us to spend 2 days designing actual experiments with our clients, and it allowed the managing director to promise the results of these rapid experiments within 1 week.  The speed at which we were able design and execute these experiments set an example and allowed the organization to be proactive in their quest to obtain rapid, validated learnings for new business concepts.

    Reference: Here’s a sample set of the types of experiment environments Strategos has created with clients in the past:

    • Attribute(s) out of context – Testing a specific solution attribute in a proxy environment
    • Minimal prototype – Deploy a basic with key functionality to test value with customers
    • Split-test – Offering the participant two or more variations of the hypothesis at the same time
    • Person behind the curtain – Utilize a polished front-end with a manual back-end to emulate the value proposition
    • Mock-through – Using a mock-up to walk through the concept with potential customers
    • Simulated test market – Prototype testing on a “safe” audience – employees, family, friends
    • Crowd market – Tapping into a larger audience for ideas, feedback, and support
    • Thought experiment – Techniques for stretching and validating without necessarily requiring user interaction