Data Quality – How Much is Enough?

I read Henrik Liliendahl’s blog post today on “Turning a Blind Eye to Data Quality”.  I believe Henrik and those that commented on the post have some very, very good points. For data quality professionals, the question might be “How much is enough?” when it comes to data quality.  And the answer to that question really depends on the nature of your business and how the leaders in your organization view the value that data quality can bring them.  The question we will most often be asked is “how does DQ help my bottom line?”  If we as  data quality professionals can’t tie DQ initiatives directly to bottom line impact, it will be hard to get serious attention.  And believe me, we want serious attention or our own value to organizations will be questioned.

This means we probably need to change the conversation away from DQ as a means to its own end and towards a conversation about how selected projects can have a positive impact on the bottom line. That conversation may be happening within many organizations at levels above our pay grades. Our first goal should be to ask our direct managers if the conversation is going on and how can we help in real, meaningful, tactical, financially relevant ways.  If that conversation is not happening, we need to ask what can we do to get the conversation going in the same real, meaningful, tactical, financially relevant ways.  We must abandon the mythical goal of a single version of the truth for all attributes.  Our goal needs to be about making the business more successful in incremental tangible ways and thus making ourselves more successful in incremental tangible ways.  After all, as Henrik points out, our businesses are being successful today despite bad data.

In comments to Henrik’s post, Ira Warren Whiteside mentioned that “As with everything else in order to convince an executive to “fix” something it has to be really easy to do, not involve a lot of collaboration and be cheap”.   I would argue with this perspective.  I think that the decision to go forward with the project should be easy, not necessary the project itself.  This means clear, unquestionable value to the business (most likely directly to the bottom line) is needed.  And I think such a project should show impact quite quickly.  You can’t have a 12, 24 or 48 month project without value being realized within, say, the first six months or so.  Thus, it is best if the initiative can be absorbed in bite-sized chunks so initial benefits can be quickly realized to help reinforce a culture of DQ. Don’t wait too long to deliver or you will lose your audience and your chances for any future projects will diminish – greatly.

In the retail supply chain, suppliers end up paying, on average, 2% of gross sales in penalties to their retail customers. This is 2% that is taken directly from the bottom line and is usually tied back to inaccuracies in delivering orders – problems with being “on time”, “right quantity”, “right product”, “right location”, “broken products”. Each supplier has people (often a whole team) focused on resolving these issues.  There are two key ways they tackle these problems.  The first is that they identify the dollar amount below which it is too costly for them to address the problem.  In short, it costs them more to fix it than it does to just pay the penalty.  The second is deep root cause analysis of those issues that are too costly not to fix – either in aggregate (ie. the problem occurs frequently) or as stand alone problems.

I think DQ practitioners could learn a lot from what is done to address these retail supply chain problems.  The first is to identify what is ostensibly “noise”.  The data that costs too much to fix based on its low impact on the business.  The second is to take that bad data that is too costly to ignore and identify initiatives to resolve the DQ problems – and tie that back to tangible business benefits.  The challenge is easy with “perfect order” delivery problems in the retail supply chain.  Companies know that the penalties are directly impacting the bottom line.  Fix a problem and the resulting money goes straight back to the bottom line.

I’m not so sure its that easy for DQ professionals, at least not with all our DQ problems.  But I’ve been around long enough to know that there are some low hanging fruit that exists in just about every business.  As a hint, if your company or customers are suppliers to retailers, you might start your quest for a promising DQ project with your compliance team since often supply chain problems can be tied back to DQ problems.  Imagine that!

 

 

 

 

Advertisements

One comment on “Data Quality – How Much is Enough?

  1. Dylan Jones says:

    Good post, thanks for sharing.

    I think a big problem is that a lot of companies don’t connect the dots between data and function enough. If understanding around data is poor I often find that knowledge of business functions is nowhere in sight.

    I’ve seen a number of clients in the past cleansing and improving data for no tangible gain, they were just doing whatever came easiest with their tools of choice.

    I think the better route is to understand the functions that are critical to the business now and in the future, map those to the data and then draw conclusions on where to focus.

    I find that if you’re having a hard-time linking your data quality initiative to some tangible gains then you’ve effectively missed off a major piece of your strategy.

    Interesting read, thanks for continuing the debate.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s