Big and Open data presents many opportunities for the food sector. Louise Marston, Director of Innovation Policy and Futures at Nesta, assesses the potential and pitfalls.
Some of the earliest uses of big data in business were in food: Walmart and Tesco pioneered the use of supplier and customer data to improve their businesses. But new tools, new sources of data and new technologies mean that all sectors can improve their use of data, and be aware of the risks and opportunities it provides.
Nesta research shows that even amongst online businesses, only 18% use data to make decisions. Businesses that do collect, analyse and use their data are more productive and profitable.
Food and farming businesses deal with data from farm machinery, processing plants, vehicles, stores and customers. Here are five ways that other sectors have addressed the potential and pitfalls of big, personal and open data.
Improve data for everyone
The Fukushima nuclear disaster provided a testing ground for a new generation of open sensors and open data. Whilst very little official data was available on radiation levels in the area, and as the world supply of Geiger counters ran out within 24 hours, the founders of Safecast made their own Geiger counters, then started recording and publishing data in great detail. The Safecast project attached detectors to cars and bikes, and travelled around taking readings every 5 seconds, massively improving the available data.
By bringing a group of people together to solve this problem, and by publishing the data openly, a very detailed map was created, which was constantly updated, for anyone to access. In time, it should be possible to study the ongoing effects on health and other factors using this data. Similar efforts have assembled open data on air quality in cities like Barcelona, and similar sensors can be used to measure weather, soil conditions and even the health of livestock.
Precision Agriculture is starting to see these sorts of sensors appear on UK farms and tractors, especially through the big suppliers. But sensors don’t have to be sold as part of an elaborate suite of tools. They can be made very cheaply, and by combining data, they can be very powerful.
This summer, sensor company Senseye announced a trial with Riverford farm to monitor soil and air conditions, and combine the data with open data such as weather forecasts to improve the monitoring of current and future crop conditions. There will be lots of opportunities in future to share sensors and data with others.
Combining data for innovation
Bringing data from many sources together can create new opportunities for innovation. This is made easier when the data is open, or at least exportable for the individual. The government’s midata project looked at the potential of combining data from companies about an individual. The midata Innovation Lab project produced a number of prototype apps for monitoring health, older people, finding energy savings and tracking finances.
By getting people to sign up and authorise the data to be combined, much more interesting services can be provided. Open transport data is a great example of this. Transport for London publishes a huge amount of live transport data, which is accessed by 5,000 registered developers to create hundreds of transport apps.
Nesta and the Open Data Institute ran a food open data challenge earlier this year to demonstrate the potential of combining open food data. The winning entry from FoodTrade brought together government open data with data from food producers and simplified the process of identifying allergens on menus.
Some data has to be protected and secured, but combining it with data from other businesses or government can still be done securely, and can produce much greater innovation.
Personal data can be too personal
Collecting personal data can create problems. There are legal protections to consider (which differ between countries, adding complexity). I believe there is also an ‘uncanny valley’ for personal data. Using someone’s personal data to make predictions that are too accurate feels ‘spooky’ – almost as if you are being followed.
The classic example of this is the Target pregnancy story, as told by Charles Duhigg in his book ‘The Power of Habit’ and republished in the New York Times. Target allegedly sent offers for pregnancy and baby goods to a teenage girl. Her father complained, although he later acknowledged that she was pregnant, but he hadn’t been told. Although there are some doubts about how accurate the original story really is, the popularity of the myth is that companies can and do get very personal insights about you from your data.
Food retailers have been collecting personal data for a long time, but consumers are increasingly aware that their data has value and that they can trade it for benefits. Even small companies can now benefit from this sort of data collection. The key to this appears to be being more transparent, not less, about how you use people’s data, and what it is for. The midata programme found that agreement to share data nearly doubled to close to 90% of participants when a specific service was being offered in exchange.
Keeping up with the Jones’ data
Research in behavioural economics has demonstrated that people are influenced by information on what most other people do. The Behavioural Insights Team improved tax payments by 5 per cent by including a message on the tax demand that indicated that most people pay their tax on time.
This can have a troubling effect too. There is evidence that the weight at which we think of ourselves as ‘overweight’ has been rising, partly as an effect of more people around them being overweight too. It could make us more likely to disregard health guidelines if we think everyone else is doing it. When something seems normal, there’s less incentive to change. Prompting in other ways, such as food labels or ‘five-a-day’ messages can help create more positive expectations, although beware of information overload.
Merely publishing data can change behaviour – if done with care – and is a powerful tool.
Beware the biases
Big data has huge potential, but also presents risks. Capturing data that is finely grained in time or geography or records every transaction can reveal previously hidden patterns and correlations. But every data set and data collection method has some biases, and these can be well hidden.
Google Flu Trends was hailed as a major milestone in big data use. By monitoring search terms, Google was able to track flu incidence in near-real time. Calibration with the official CDC data that was available later suggested it was accurate. Papers were published, keynote talks were given. But four years after it was unveiled, the data departed from reality, and the model no longer worked.
There are different explanations for why this happened. Some think that the introduction of Google’s auto-suggest feature started skewing the data – you are perhaps more likely to search for flu if you see it suggested in the search box. Others say that what Google built was part flu detector, and part winter detector. Like correlating shark attacks with ice-cream sales, there wasn’t ever a real link between the data and the event it was tracking.
It’s tempting to see a ‘big’, comprehensive dataset as more ‘truthful’. But just because the data is large, it doesn’t mean that it isn’t biased. Examine what it’s actually telling you, and test your predictions to see if they hold.
Food and farming are being transformed by data technologies: sensors on farms and distribution vehicles, consumers tracking their health, retailers examining sales, websites looking at their traffic. There are pockets of the future everywhere, but taking advantage of these opportunities will need careful judgment as well as new skills.
Louise Marston is a Director of Innovation Policy and Futures at Nesta, and is leading a new project on the future of food. Nesta is an innovation charity with a mission to help people and orsanisations bring great ideas to life.