It’s sensible to think that dark data which is out of sight and out of mind, could contain unknown risks. If effectively managed or analysed these risks would be mitigated, and costs could be reduced. Without knowing what dark data is present in an organisation, it’s not possible to realise its value. This means people are reluctant to get rid of the ‘dark data’ believing there may be some unknown value in it.
To offset this reluctance, organisations need to understand the costs and risks associated with keeping the data. Typically, it is unencrypted and not considered sensitive (even though businesses often have no real idea of its content). Disparate pieces of data can be pieced together by others – and aggregated data could still be valuable to criminals. This is extremely serious.
1. Time to change your process
The first question to ask is: Does every form and document used by your organisation belong to a business process with a responsible owner? If not, then it’s time to change your information management processes.
Some of the answers may come from the past. Record Managers are considered old fashioned but they were experts; knowing what should be kept and where, for how long and how accessible it was. Then computers came along and the role disappeared – with no-one around to stay on top of the data. Is it time for Record Managers to return?
There needs to be one person in every department who is in control of the data it produces – even when it only has short-term value. Not having an information management policy is pretty bad in this modern age of data – but having a policy that is not followed or which no-one takes responsibility for is even worse.
2. The forgotten paper files
It is easy to forget about paper, too. Not all dark data is digital – people leave paper everywhere and the ultimate dark data may well be sitting in a third party warehouse that businesses are billed monthly for but have almost forgotten. Perhaps the equation to consider is whether the potential value of data is worth more than its risk. If the risk is higher than the potential return then consider deleting it.
The promise of the golden nugget hidden away in data is a compelling vision of “Big Data” and Business Intelligence vendors, and the reason many businesses refuse to destroy it. But someone has to be asking the right questions of the data for that to happen – insight doesn’t happen by chance and so data that is not managed effectively is unlikely to provide the hoped for commercial breakthrough.
So now really is the time to consider: Is the risk worth it?
Or does some of your dark data need to be destroyed?
3. Dark data megaphone
The dark data megaphone – or black hole if we are sticking to the astronomical analogy – provides an insight into the risks of keeping unnecessary data. Pulling your future budgets towards it like a black hole, it will swallow storage budget and potentially increase risk at the same time.
4. Own it, manage it – or dispose of it
If data is growing exponentially, it’s time to consider whether the only answer is to store ever more – or to manage it more appropriately. Not all data is equal, not all of it provides insight. Dark data needs to be identified and brought into the light, it needs owning and managing. Or it needs to be disposed of as part of a sustained governance framework.
Part of the problem is that the cost of storage is reducing year on year. As the cloud becomes increasingly popular and secure, it’s easy to think that hiding information in the ether is the answer to all problems. But that under-estimates just how quickly the volume of data is expanding.
5. Managing information, saving costs
If data storage costs fall at 20 percent per year but growth is at 40 percent, overall there is still an increase in direct costs. Indirect costs will probably increase too as businesses consider how to house more hardware and pay for increased administration and management. Either way costs will only go up unless serious attention is paid to information governance and data management.
6. Information overload
Typically IT will respond to the business ‘needs’ by putting an ocean of data in more and more buckets of storage, and patching leaks at every juncture. Instead, IT departments need to help the business understand its information. If not, we have forgotten what they are there for and they might as well drop the ‘I’ from IT.