How the Food Standards Agency used Pulsar to help issue early warnings for Norovirus

Using social media data to help predict, rather than just report on diseases such as the Norovirus is something the Food Standards Agency (FSA) can now do at an 80% accuracy level.

CHALLENGE

Most researchers working on health and disease topics rely on information they receive from labs, hospitals and General Practitioners (GPs). For example, collating lab reports of quantities of people getting a virus at a given time will give a good understanding when outbreaks occur, after they’ve happened. Useful? Absolutely. But how can you wrangle data to help you predict, rather than just report, on outbreaks of diseases in order to inform the wider public?

SOLUTION

Sian Thomas, Head of Information Management of the Food Standards Agency, started looking into the true potential of different data streams around people’s illnesses in 2014. After testing a variety of methods and data sources, she came to Twitter, a key platform when it comes to sharing spur-of-the-moment thoughts and feelings such as feeling rather unwell. Zooming in on the highly contagious (and equally notorious) Norovirus, her team could soon identify correlating words users shared to describe the disease, such as ‘puke’, ‘vomit’ and ‘diarrhoea’.

Monitoring this manually at first, the team decided to start using Pulsar as one of the tools to keep track of activity around the disease online due to its flexibility when it came to the range of needs expressed by the FSA (LINK TO any USP talking about slicing data in different ways / the only platform combining keywords topics and people) and its fundamentally real-time approach. Says Thomas:

“The problem with lab reports is that they only give you the ‘hindsight’ view. They are useful, but you are working with old information which obviously doesn’t help much in terms of predicting to a certain level outbreaks of the Norovirus. Plus, it’s often elderly people who go to the GP with these types of diseases. Younger people are more likely to deal with it on their own terms and might not even visit the GP. So in that sense, the lab reports give a skewed view. It’s just very hard to say how skewed.” (this might just stress the fact that social media in this case also gives a skewed view… elderly people are not on Twitter)

Sian’s team of analysts then created an algorithm by using historical data from Twitter – one, should be added, the FSA Analytics team has won the inaugural cross-government Data Science competition for in November 2015 when presented by David Millson, Operational Research Analyst at FSA – and started comparing the volume of mentions to Norovirus lab reports reporting confirmed cases of the disease in the UK. They’ve been doing this for over a year now, and the overlap and timing turned out to be even better than the analytics team had hoped for. Using their custom algorithm, the FSA can now predict outbreaks of the Norovirus at an 80% accuracy rate – which is incredibly high.

OUTCOME

Using Tweets to communicate early warnings to the general public about the Norovirus is now something the FSA can do, thanks to their approach to different types of data. (Social) data science can definitely make a difference in health care, as Sian Thomas and her team can attest to: “More people turn to the internet than ever to find information about disease areas, and they are open about sharing information too. Mining this information in the right way can help us raise awareness with the general public – which in turn can result in disease prevention. There is huge scope for using social data this way, we’ve only scratched the surface. I’m excited about the opportunities it offers for researchers, health care professionals, and ultimately, the wider public.”

If you you are interested in learning more about how we can help government organizations or healthcare providers don't hesitate to send us an email at [email protected], or book a demo by filling out the form below. 

[gravityform id="3" title="true" description="false"]