On the Tough Question of Contradictory Data in Market Research Methodologies

It’s great when social supports an initial theory proven by traditional market research data. Wonderful, in fact. But what about when it…doesn’t?

We’ve already explained why you need to embrace the “just do it” mantra when it comes to incorporating social data into your research mix. We also shared several ways to blend different data sets with social. And now it’s time to get into the nitty gritty examples I’m sure you’ve all been anxious to know.

Can you share any examples of a situation where social intelligence insights reveal contradictory findings to other research?

 

The ultimate purpose of any digital intelligence technology is to optimize experiences, influence customer behavior, and drive better business decisions.

Social data and analytics tech is no different. But right now, many social insights teams are stuck in listening and reporting mode, despite their potential to gain deeper and more actionable insight.

 

As practices and tech mature and become more business-, user-, and action-friendly, and integration with other data insights processes increases, expect to see growth in the use of social insights across the enterprise in parallel with increased ROI. But for today, expect to see isolated teams within many firms just sitting and listening to their customers socialize on their brands and offerings.

Posing some tough questions to our Research Services Project Manager

There may be many reasons and examples of why social research insights could be contradictory to other research. Here are two key ones that come to mind.

Traditional research methods, for example focus groups, provide a limited sample.

When you broaden your research to a wider audience via social, you’re not necessarily getting contradictory data but rather you’re getting new data.

In a focus group you’re asking specific, pre-determined questions, which can limit the responders’ answers. You’re setting them up to answer one way or another. The goal is to avoid that in focus groups, but alas bias creeps in despite all efforts to the contrary. You introduce bias simply by being in the room during a focus group.

Social data might seem contradictory, but it’s a new view on what the data might be and can take you down roads and surface topics you didn’t expect to encounter when you embark on a research project. It reveals new areas you’re not usually exposed to with traditional research in the form of surveys or focus groups. Social research allows you to select a very general topic without having to determine the parameters of how they may be discussing that topic – be it product, industry, trend, or even a specific spokesperson. Social then does the work of gathering and showing you what everyone is saying on the topic which brings up many different ways they’re discussing it.

Traditional research or syndicated data is inherently outdated.

We use syndicated data such as that from MRI, Simmons or Nielsen, as truth because it’s what we readily have access to in our research teams. And in fact, traditional and syndicated data is a good source of data to extract insights, but we have to keep in mind that by the time it gets into our research mix and our collective systems, it can be a year old. A lot changes and quickly in our world, especially given the socially-driven consumer environment.

One thing I’ve discovered is that social is not a solution to this problem of stale data, but another way of looking at it. When you find something contradictory by comparing social to other research, it’s an impetus to dig deeper. It sparks curiosity and questions about why the discrepancy exists to begin with and what other factors may be in play.

Applying social in the real world (of market research)

I asked Evelyn if she could share with us some more tangible examples to help bring to life how social data isn’t necessarily contradictory, but revelatory for market research teams.

For one alcohol brand I was working with, we found through syndicated data that younger people were not drinking as much beer and the consumption volume had decreased.

We had initially based our insights on syndicated data which limits the options to select to respond. Frankly, this type of research is always a little bit behind the trend. With social listening you have the luxury of simply searching for the term “beer” and seeing what’s trending, how people are discussing this topic, what other tangential topics and phrases might be coming through in this conversation.

So why was consumption of “beer” down? It’s possible that it wasn’t down. Social introduced the possibility that it’s not that they were drinking less beer, it’s that the brands and types of beer they are drinking are changing.

When you’re doing social research you also can’t see how much they’re consuming – that’s a sales data question. But we were able to see on social that they were still talking about drinking beer. The brands in the discussion were simply changing as newcomers in the beer manufacturing industry prevailed in social conversations.

We could use social to pinpoint specific demographics discussing beer, and then more specifically craft beer. These demographics could be as general as simply looking at “women” or as specific as Hispanic millennial women under 30, or men in finance under 35 in the 10 most populated cities in the U.S.

Social data gives you the liberty to broaden the questions that you ask from your research and gives you the ability to intake the data in a new and fast way.

The important thing when you find contradictory data is to determine the why.

So what are the key differences between the two forms of research? Using both sets combinedly to inform your research allows you to understand why one set of research has a specific outcome. In the alcohol brand’s case, you can learn from one piece of research that the volume is decreasing and then use social insights to determine the why.

Here’s another example from the cosmetics industry.

Let’s say a cosmetic brand is interested in learning how African Americans use cosmetics.

If you were to begin your research by looking at sales data you could assume the use of cosmetics by this demographic is less than the general market if sales volumes are low. However, if you were to broaden the scope of your research to social, you would learn that this specific demographic discusses using cosmetics extensively online. But then why are sales low? Where is the discrepancy between sales data and social discussion?

In a particular research project I worked on to dig deeper into this topic, syndicated research didn’t tell us what social revealed, that in fact limitations of color matching was a major issue this demographic encounters. And they were taking to the web to discuss. Unless a brand specifically asks a question about color matching limitations in a traditional research survey or focus group, they would not hear about this product limitation that was affecting sales for a large demographic.

When you see people are talking about makeup on social media, but the numbers are saying they’re not consuming this CPG (Consumer Product Good) product, you then have a starting point to address this through product R&D.

What you’re likely to find through social research is that this specific demographic of consumers is discussing the limitations of the product online, and simultaneously praising the brands and products that they are in fact using to fulfill their product needs by offering a wider range of shades to match skin tones.

All of these findings hold valuable insights. These insights can be the catalyst to product feature change (such as offering a wider variety of skin tone matching options in various cosmetics) increasing sales, building customer loyalty, and addressing a consumer need that will elevate your brand in the eyes of the consumer.

Any last advice for us, Evelyn?

Don’t ignore data because you find it contradicts other data points you have previously found or are more comfortable with. Take it as an opportunity to dig deeper and determine the key points of differentiation between the two data sets. As described above, this can help you garner insights that you would not have previously encountered.

It is also important to keep in mind that you aren’t always able to use all data sources at once.

Taking the time to learn more about the different insights you can garner from different data sets will also help inform what methods you need to utilize in certain scenarios. For example, when gathering consumer insights, social and syndicated research might be the best combination. But if you are looking to learn more about product development you may want a blend of focus group and social data.

Compiling the notes

The two examples Evelyn shared in her real world examples come from two very different industries (alcohol and cosmetics, respectively) and really highlight the potential contradictions and correlations social research can provide to complement more traditional research data.

Here’s my quick summary based on all this excellent knowledge sharing from James and Evelyn:

1. Set out specific KPIs based on your goals as a marketer
2. Utilize social as a complementary research data set to confirm, question and dig deeper into insights revealed from traditional data
3. Blend data, and avoid using more than two data sets except in certain use cases (such as advertising)
4. Never stop asking questions
5. Rinse, and repeat.