Big Data Is Not the Same As Big Insight

    Michel van Hove

    The promise of a (Big Data) revolution

    Not a day goes by without an article in a magazine or online publication about the promise of Big Data. The topic of big data has pre-occupied businesses for the past few years and many see it as an under-leveraged source of value. In a recent CEO survey by the Economist Intelligence Unit 23% of CEOs believed Big Data is going to revolutionize businesses and 42% believe generating customer insights is priority number one.

    Is Big Data the treasure trove of information that companies can leverage to develop more personalised services, cross sell, create greater customer loyalty and innovate? While big data may hold big value we believe it is a step removed from true insight that drives real and impactful innovation.

    Reacting to what’s obvious

    Today every person generates vast quantities of data and more and more of that is captured. The data describes “what people do” and with proper analytics this can help companies improve and optimize their operations.

    Let’s take a simplified example of a typical CPG company that captures transactional data of all its SKUs across multiple channels and notices that it’s sales of a particular SKU has spiked in a chain of stores in the New York area.

    The conclusion is that the product fulfills a particular need and so we provide more of it or develop line extensions to saturate the channel and make sure there is enough to satisfy demand. For a while sales increase and you consider following the same tactic for other channels in similar geographies. You also include special promotions to boost sales. For reasons unknown to you sales in these other markets do not go up and you stop the promotions. And to add to the problem the sales in the New York area drop dramatically but you have no way of telling why this has happened. Sound familiar?

    The issue here is that you are not leveraging your data assets enough because you still have no idea why sales have gone up in the first place (and why it didn’t elsewhere). What’s hidden behind what you can clearly see? Most managers would react based on their experience or intuition but real innovation that has a lasting impact often challenges conventional and widely shared beliefs in your industry. But to understand what those are you’ll need to dig deeper. You still need to find the customer’s frustrations, workarounds and delights that you could develop into a great customer insight, i.e. understanding the true nature of a situation or behaviour in a way that has not been obvious before.

    Leveraging Big Data to drive innovation

    We believe companies can leverage Big Data by using it to drive discovery activities around customer and consumer behaviours. If the patterns in Big Data reveal “the what”, discovery focuses on revealing “the why”. What you are providing, a product or service may address a need or solve a problem that you were unfamiliar with when you first launched the product. But because circumstances change this problem may have emerged without you being aware other than noticing the spike in the dataset. Understanding the nature of the problem will help you potentially identify other consumer groups who are faced with the same or similar challenges. Or you may find that this learning would drive the development of a product or service that would even meet the consumer need even better and in a differentiated way. You may have to change your value proposition to accommodate specific circumstances but in the end you stand a much better chance to create a lasting impact. Learning is what counts, wanting to know the why behind “the what” drives deeper discovery. It takes time but the learning is valuable.

    Unlocking the value

    Big Data can be a source for great innovation and businesses are right to give it the attention it deserves. But to unlock the value from big data beyond optimization you will need to a) identify meaningful patterns b) develop hypotheses about why they are present in the data and c) conduct your own discovery to identify the real customer needs and uncover the beliefs you have that blind you to new opportunities.

    Developing these insights will provide you with new perspectives that can be used to drive real innovation and unlock value from Big Data.