Interview: How To Ask The Best Questions That Social Intelligence Can Answer, With Paul Siegel

Friday afternoon isn’t usually my favorite time for philosophical discussion (unless I’m in the pub), but I was on the phone to my esteemed colleague, data scientist Paul Siegel and I knew there was no way this was going to be basic.

The reason for our call was to discuss questions – which questions are the best kind of questions for social data to answer? Is it possible to get too caught up in the question formulation and not do enough analysis? Is there room for creativity?

A lot of questions were asked about questions. The result is this article, in which we get to grips with how analysts can get the most from social intelligence by building strong foundations in the question formulation stage.

Asking good questions

So what makes a good question for social intelligence? Paul says:

“Good questions for Brandwatch and our customers to ask are questions that strike a balance between two factors – a question that addresses an issue close to the business and a question that has an actual, tangible answer – something that tests and experiments can validate.”

In a sudden flashback to my journalism training I remember the four Ws and a H – who? what? when? why? and how? – which of these are the best questions we can ask of social data?

There are three main components to social data, Paul says:

1. The information element – the text and words we post
2. The human element – who we are and what we do
3. The time element – the fast paced nature and our ability to update the world instantly on what’s happening

When we think of social data as the interesting intertwinement of all these things, the who, the what and the why seem like the most pertinent question formats to start with.

This makes an interesting comparison to search data where the “what” seems to be the primary question starter. I’ve often heard the differences between search and social data as search giving the analyst the what and social giving them the why.

Paul agrees, using the natural example of cheese.

“Social data can show that there’s an uptick in people talking about cheese, just like search can. But social gives you that extra layer of why cheese conversation changes. Perhaps cheese conversation has risen in a particular demographic, perhaps it’s a particular type of cheese that people are talking about at a certain time of day.”

The two can clearly go hand in hand when asking questions relevant to our industry.

Getting appropriate questions from non-analysts’ requests

I put a scenario to Paul – you’re an analyst going about your business when you’re accosted by a member of a different team asking if some social data could be gathered around a particular topic. How can an analyst take the ramblings of someone who doesn’t work with social data and turn them into questions that can be answered?

“It seems sensible to ask about topics that are big in conversations around your industry, but even nailing down what a topic is is difficult…When you’re getting started, the majority of the work (60%) is finding the right question. Then 40% is analysis.”

This breakdown surprised me – as someone who often tends to take a more exploratory approach to social data by looking around at different elements of the conversation before crafting a story around that data, spending 60% of my time wondering what questions to ask seemed like a lot.

“Over time as an analyst I’d make it my priority to get that initial time down,” said Paul. He’s clear that putting together processes that answer predictable questions on a regularly basis is a good way to set up your system so you’re not spending all your time pondering.

What doesn’t help is that there’s no one-size-fits-all approach to categorizing social data in all the ways we want it to be broken down. An example Paul gives is sentiment analysis. “The reality on the ground is that one-size-fits-all sentiment models aren’t always fit for purpose – they have a hard time with language which is idiosyncratic to a brand, an audience, an industry, or an event.”

Paul says that the teams that get the most out of sentiment classifiers are those that invest time in monitoring the kinds of words people use in their own industry or area and making an effort to maintain the classifiers they set to ensure they’re consistent and accurate. For example, if you work in an industry where your audience often use slang, creating rules that categorize the latest positive and negative words correctly will help ensure nothing is missed.

“In other words, most of the work is in precisely formulating the question ‘what is sentiment?’ in a way which meets their business needs,” Paul says.

Creating customized sentiment rules with a tool like Brandwatch Analytics can help automate the monitoring of positive and negative sentiment around your brand that suits the unique conversations that go on around it.

Over- and under-thinking

Given the large amount of time Paul suggests for piecing together the best research questions, I ask if there’s a danger that you can get a bit too philosophical about it all.

He laughs.

“You don’t have to be like ‘what even is language, man?’”

Instead, he discusses the damage that can be done by ill-thought out questions when going into an analysis:

“The risks of over-thinking questions are much smaller than under-thinking questions. If you rush it there’s no scenario where you won’t pay for that mistake.”

At the same time, nailing down questions shouldn’t mean there’s no room for exploring the data and getting a feel for it, and even using it to inform more questions.

“Spending time reading through mentions and letting your brain make connections to form questions, getting to know the data, is cool. There should be a part of getting into social that’s exploratory or open minded,” Paul says.

Are you asking the right questions?

My main takeaway from my chat with Paul was the very serious attitude he and his team have towards creating excellent questions.

In previous meetings I’ve had with the Data Science team at Brandwatch I’ve come away with a sore head, wondering how on earth I’ll be able to get started on a topic I want to write about, knowing that we’ve just totally pulled apart what that topic is and the assumptions that went into the questions I’d originally started with. It was almost reminiscent of a philosophy class when my lecturer challenged us to prove the table in front of us was real and then proceeded to rip our arguments to shreds.

That starting point can be frustrating, but like Paul said, if you’re debating the very nature of language you’ve probably gone too far with pulling apart your questioning. Striking the balance between over and under-thinking is not easy, but remember the dangers of jumping into a lengthy analysis without a question you can straight-forwardly answer.

Hopefully you’ll come away from this article with less questions than answers but, then again, what even is a question?