Although it is hard to tell what portion of data is dark just like dark matter, it is okay to assume that it is a considerable portion.
According to Gartner, a renowned IT consultancy, dark data entails the information assets that organizations gather, process and store throughout normal business tasks, but mostly fail to utilize it for other things.
Kirsten Gillon, ICAEW IT Faculty’ technical manager, also cautioned that the operational expenditure of dark data is increasingly becoming a drag on the business, mainly if it is left unmanaged.
She reiterated that this effort is aimed at assisting organizations to think about what they can achieve with all their untapped data.
One of the creative uses or purposes can be to use unstructured or historical data for predictive purposes by utilizing artificial intelligence (AI) or machine learning. In fact, the National Business Research Institute has already reported that 61% of enterprises asserted that they had applied artificial intelligence (AI) in 2017.
Dark data startup Datumize approximates that two-thirds of all the data in a company are dark. On the other hand, IBM further estimates that 80% of all the data is unstructured or dark, and is expected to rise to 93 percent by 2020.
Carlota Feliu, a marketing director, said that one of the applications of the unutilized data is to optimize performance and control risk through tracking the movement of assets and people in their premises.
According to her, a warehousing company with all the data regarding the movement of its assets and workers can assist in improving its spatial organization.
More often than not, dark data may be noisy and unstructured, for instance, data from social media.
It possesses limited value for undertaking predictive analytics since the valuable meaning or sentiment is mostly drowned in noise. As such, most organizations find themselves focusing on what is said about them on social media or social media data only for customer service.
Steve King, Black Swan Data’s CEO, has created a social forecast application for PepsiCo, which is fed with 50 million pieces of data daily from Instagram, blogs, reviews, forums, Twitter data and sales data.
It utilizes the data in predicting the future data for over 1,000 ingredients through the use of customer views of 52 themes and 72 benefits in five Pepsi markets.
King offered three recommendations to help you to determine which dark data is useful as well as the data that should be ignored, archived or deleted.
To begin with, you don’t require to get rid of your dark data unless you are confident that it is incomplete or inconsistent to be used later. Thanks to the declining costs of cloud storage, it can be stored in the cloud.
His second recommendation urges you to be disciplined about the data collection and storage process in a bid to make it accessible. Some data stays dark mostly because its quality is considered too poor to train an AI or build useful insight.
Lastly, King suggested that in a bid to create value, you need to shift your focus to the most vital business issues rather than the most exciting AI applications.
