The Signals Journey: Building a New Kind of Intelligence

Last month, we announced Signals, our smart new feature that alerts you to emerging crises and trends.

Signals is our latest foray into the world of more sophisticated automated intelligence, building on top of extensive work we’ve already done to collect, clean and analyze mountains of social data.

It’s a feature with an interesting history – it’s been in development for over 18 months, and has had many iterations and challenges along the way.

Our goal was to build a new kind of intelligence that could detect trends in your data for you, in real-time – allowing analysts and marketers to get insights and updates on their data without being logged in to the platform 24/7.

We played around with different formats for Signals, all the while seeking feedback from a select group of clients on the early beta.

Early phases

Somewhat inspired by Google Now, we initially started with a sort of streams view that would show different events as they happened.

Then, we moved to a ‘bubbles’ interface that showed different events as bubbles that moved and grew, because the streams view didn’t give users an overview of how issues were growing or changing.

Both of these looked cool, but we found they weren’t actually that functional – they meant that a user had to keep checking to see if anything new had happened and didn’t help them understand which events needed further attention.

We like pretty things, but we don’t believe in just making things that look nice; they also have to work.

A breakthrough

In the end, we had a rethink and went back to the initial problem and use case: if there is a crisis, a user wants to know straight away.

We decided to drop the interface altogether and instead focus on the most important part of Signals – the delivery of the information. What’s one of the fastest ways to deliver information? Email.

Our goal was to make the emails as useful as possible – including not only the reason for the Signal (i.e. what triggered it) but also other useful information such as volumes, related topics and example mentions – in order that someone could receive it and instantly be able to evaluate whether it needed escalation or further investigation.

We spent a lot of time thinking about what suitable triggers for Signals would be.

Some we experimented with and eventually dropped as they resulted in too much noise and too little Signal. The ones that remained were tweaked and honed over time. The Signals architecture was a herculean task; for each one of the Queries with a Signal set up, it keeps track of hundreds or even thousands of possible triggers, watching each and every potential trigger closely to see if the volume has exceeded a critical threshold.

This means less interruptions for you whilst still allowing you to keep your finger on the pulse.

A Signal about Signals! Just one of many different types of Signals emails, which show different metrics and data depending on the trigger.

The challenge of Signals blindness

There were of course challenges along the way, and will be as we continue developing Signals.

One of our biggest challenges was ‘Signals blindness’ – if Signals was too sensitive, it would send too many emails, and then users would start ignoring them (which is obviously the opposite of what we wanted!). At the same time, we have to balance that with the risk of missing something if we make it less sensitive.

One of the ways we worked around this to improve it was to add another layer to the algorithm that would ‘cluster’ events together – so that it recognizes when several triggers (say, a hashtag increasing, an increase in volume from a specific country and a popular retweet) are all related to the same event.

This helps us avoid sending you multiple emails about the same thing. We’ve worked on this algorithm for a long time, using advanced machine learning and some good old fashioned ‘eye balling’ to make sure it’s at the right level. Sometimes, you may still get multiple Signals for the same event; but we hope it’s rare and worth the payoff for not missing anything.

We want feedback!

Now, as we continue to develop Signals, we’re working on adding new types of triggers, as well as exploring more ways to improve the user experience and delivery methods for that data, including mobile, and ways of tracking an issue or trend in a more advanced dashboard.

As we continue developing Signals we want your feedback – are the Signals you’re getting useful? Did we miss anything? Find anything really important? Do the dashboards work? What else could improve Signals?

Let us know your thoughts – via email, support, your AM, in the comments below, on Twitter – whatever works for you! We really do want to hear them so we can continue building on Signals, so get in touch.

And, if you want to find out more about Signals join our webinar with VPE Product Strategy Glenn White on Tues 7th April – sign up here.