Understanding Sentiment Analysis: What It Is & Why It’s Used

Sentiment analysis – otherwise known as opinion mining – is a much bandied about but often misunderstood term.

In essence, it is the process of determining the emotional tone behind a series of words, used to gain an understanding of the the attitudes, opinions and emotions expressed within an online mention.

Sentiment analysis uses

Sentiment analysis is extremely useful in social media monitoring as it allows us to gain an overview of the wider public opinion behind certain topics. Social media monitoring tools like Brandwatch Analytics make that process quicker and easier than ever before, thanks to real-time monitoring capabilities.

The applications of sentiment analysis are broad and powerful. The ability to extract insights from social data is a practice that is being widely adopted by organisations across the world.

Shifts in sentiment on social media have been shown to correlate with shifts in the stock market.

The Obama administration used sentiment analysis to gauge public opinion to policy announcements and campaign messages ahead of 2012 presidential election. Being able to quickly see the sentiment behind everything from forum posts to news articles means being better able to strategise and plan for the future.

It can also be an essential part of your market research and customer service approach. Not only can you see what people think of your own products or services, you can see what they think about your competitors too.  The overall customer experience of your users can be revealed quickly with sentiment analysis, but it can get far more granular too.

The ability to quickly understand consumer attitudes and react accordingly is something that Expedia Canada took advantage of when they noticed that there was a steady increase in negative feedback to the music used in one of their television adverts.

Sentiment analysis conducted by the brand revealed that the music played on the commercial had become incredibly irritating after multiple airings, and consumers were flocking to social media to vent their frustrations.

A couple of weeks after the advert first aired, over half of online conversation about the campaign was negative.

Rather than chalking up the advert as a failure, Expedia was able to address the negative sentiment in a playful and self-knowing way by airing a new version of the advert which featured the offending violin being smashed.

Contextual understanding and tone

But that is not to say that sentiment analysis is a perfect science at all.

The human language is complex. Teaching a machine to analyse the various grammatical nuances, cultural variations, slang and misspellings that occur in online mentions is a difficult process. Teaching a machine to understand how context can affect tone is even more difficult.

Humans are fairly intuitive when it comes to interpreting the tone of a piece of writing.

Consider the following sentence: “My flight’s been delayed. Brilliant!”

Most humans would be able to quickly interpret that the person was being sarcastic. We know that for most people having a delayed flight is not a good experience (unless there’s a free bar as recompense involved). By applying this contextual understanding to the sentence, we can easily identify the sentiment as negative.

Without contextual understanding, a machine looking at the sentence above might see the word “brilliant” and categorise it as positive.

How we do sentiment analysis at Brandwatch

Remember the scene in Terminator 2, when a young Jon Connor teaches the T-800 hip 90s phrases like “no problemo”, “eat me” and “hasta la vista, baby”?

That’s not entirely dissimilar to how a linguist expert would teach a machine how to conduct basic sentiment analysis.

As language evolves, the dictionary that machines use to comprehend sentiment will continue to expand.

With the use of social media, language is evolving faster than ever before. 140 character limits, the need to be succinct and other prevailing memes have transformed the ways we talk to each other online. This of course brings with it many challenges.

Machine learning techniques and the field of natural language processing both have their role to play in the future of sentiment analysis. There’s a lot of work to be done, but improvements are being made every day.

At Brandwatch, we employ a rules-based process to help our software better understand the ways context can affect sentiment.

We take all the words and phrases that imply positive or negative sentiment and apply rules that consider how context might affect the tone of the content. Carefully crafted rules help our software know the first sentence below is positive and the second is negative.

“I want a burrito so bad”

“I just had a burrito. It was so bad.”

We’re also working a more detailed analysis that looks at emotions, including happiness or anger. This reveals even more context to the conversation happening around a brand or event.

You can also use our positive and negative classifications to create a sentiment score for conversations. As you can easily see the percentage of mentions that are positive, negative ,or neutral, you can get a picture of how your brand is performing over time. For example, if positive mentions of your products has gone up by 15% year-on-year, it’s clear you’re doing something right.

The caveats of sentiment analysis

The above examples show how sentiment analysis has its limitations and is not to be used as a 100% accurate marker.

As with any automated process, it is prone to error and often needs a human eye to watch over it. At Brandwatch, we give users the opportunity to redefine sentiment if they believe that it has been wrongfully categorised.

Beyond reliability, it’s important to acknowledge that human’s expression doesn’t fit into just three buckets; not all sentiment can be categorised as simply as positive, negative or neutral.

Predictions for the future of sentiment analysis

While it’s difficult to speculate how a relatively immature system might evolve in the the future, there is a general assumption that sentiment analysis needs to move beyond a one-dimensional positive to negative scale.

In the same way that politics cannot always be reduced to a position on a left to right scale, there are other kinds of sentiment that cannot be placed on a simple barometer.

For the future, to truly understand and capture the broad range of emotions that humans express as written word, we need a more sophisticated multidimensional scale.

Can text analytics measure skepticism, hope, anxiety, excitement or lack thereof? Yes! And we’ll be making this technology available soon.

Organisations will certainly become more aware of the applications of sentiment analysis within their marketplace, fuelling the growth of sector specific services and technology delivering sentiment specific use cases – for example, intelligence tools that aid decision-making for financial traders and analysts.

We will see a shift in perception of the reliability of sentiment analysis. Users will become more comfortable with the idea that the automatic analysis of individual text material is hard to match human performance.

The insight that can be gained from large datasets (millions of Tweets) will overshadow the concerns about reliability at a granular level (a single Tweet).

Instead, the focus will be on how to make results interpretable and actionable. In the meantime, we’ll be ensuring we are working at making sentiment analysis as accurate and easy to understand as possible.

If you want some more advice on sentiment analysis in Brandwatch, head over here for some tips.