So far, I have written several posts on why data quality is important and the impact it has on an organization. In this post, I will highlight how bad data impacts an organization's bottom line, quantify the costs and discuss some of the root causes.
Cybersecurity and hackers get lots of press and therefore making a strong case for investing in Cybersecurity is easy and straightforward. Budgets for Information Security are growing, since senior leaders feel an immediate impact from customers and key stakeholders - when their organisation's data is compromised. Loss of personal and financial data is tangible and so is the exposure to those whose data is compromised. This results in reputation loss, financial loss and a loss of credibility with customers and results in long term damage. The Target Corporation hack is a great example. The data breach and subsequent compromise of 40 million debit and credit card numbers as well as personal information for another 70 million people resulted in the firing of the CEO and CIO and a significant drop in revenue. Target has had to beef up its data security and data governance processes, compensate the victims and introduce new and improved credit cards to mitigate any future risks.
The cost and impact of data quality on an organisation's bottom line is the same or could even be higher than that of cyber crimes, but it seldom gains much publicity and to my knowledge no C-level executives have been fired as a result of producing or consuming bad data. This is primarily because there is a lack of awareness about the impact of bad data among the leadership team, impact and cost of bad data on an organisation's bottom line are hard to quantify, organisation's aren't measuring them and they are usually absorbed by individual departments, as part of their routine operational processing activities.
Let me re-iterate a very important point - data quality is a nebulous term and is hard to quantify, since data is pervasive, most companies do not have a mechanism to quantify data quality across their Information Supply Chains (ISC) and hence are unable to determine the impact of bad data on their bottom line. However, most business and operations staff can provide anecdotal evidence and specific examples regarding the impact of bad data on their organisation's bottom line.
In 2002, The Data Warehousing Institute (TDWI) published a report titled "Data Quality and the Bottom Line: Achieving Business Success through a Commitment to High Quality Data.". This report showed that there is a significant gap between the perception and reality regarding the quality of data in many organizations, and that data quality problems cost U.S. businesses more than $600 billion a year. The report's findings were based on interviews with industry experts, leading-edge customers, and survey data from 647 respondents and primarily focused on the impacts of bad customer and address data. Imagine what the current costs are, 13 years later, since data volumes have exploded due to social and mobile applications and data is far more complex, with respect to variety.
My highly conservative projection puts the current impact of bad data and data quality issues on US businesses at One Trillion Dollars. To put this in perspective, the US Gross Domestic Product (GDP) is approximately 17 Trillion Dollars. So, at a minimum, bad data is costing the U.S. 6% of its GDP. The worldwide impact may be in the Trillions. These metrics should get the attention of Policy Makers, Shareholders and Boards of Directors!!
In a blog post regarding the cost of data quality, Jim Harris states, “It always seems crazy to me that few executives base their ‘corporate wagers’ on the statistical research touted by data quality authors such as Tom Redman, Jack Olson and Larry English that shows that 15 to 45% of the operating expense of virtually all organizations is WASTED due to data quality issues".
I recently read a well written post titled "The Causes, Cost and Consequences of Bad Government Data authored by Katherine Barrett and Richard Green", in which they highlight this issue and discuss it in great detail.
The good news is that a significant portion of the wasted operating expenses can be avoided, with a relatively small investment in data quality best practices, tooling and some training.
Here are the seven root causes that I have identified for the high cost of bad data:
1. Data Processing
Inefficient processing and project delays, resulting from poor data quality, impacts time-to-value,
Data quality checks and data cleansing performed by each data consumer within individual departments is expensive and resource intensive,
Organisations that aren't mature from a data quality perspective, typically identify and address data quality issue manually. This is error prone, time consuming and expensive. Lack of automation and repeatable processes impacts the bottom line,
Redundant data reconciliation checks and remediation is performed across departments, since they operate in silos, and
Root cause analysis and due diligence performed to address data quality issues is expensive.
2. Decisioning
Analytics performed on bad data results in bad outcomes with serious consequences,
3. Financial
Redundant data quality checks that occur across data silos as it is being transported across the Information Supply Chain (ETL Processing) is expensive and error prone,
4. Reputation and Brand Impact
Bad customer data impacts reputation, frustrates customers, impacts decision making, doesn't give you 360 degree view of customer. This is true of other data domains such as product.
5. Regulatory Compliance
Bad data complicates and makes compliance activities and processes expensive. Therefore, companies are being forced to invest in proactive data quality management.
6. Risk Exposure
Risk Management is based on data and facts. Using Bad or low quality data to make risk management decisions is not desirable and can have serious ramifications for organisations.
7. Development Costs
The development costs incurred by organizations, to work around all their bad data.
In the post titled "5 Data Quality Best Practices" I provided some thoughts on how organizations can address the above challenges. I provide further insights in "This Data Quality Rule Will Save Your Organization Millions".
Go forth and conquer!