Social Networks, Big Data, Data Quality and You!

Not only is social data Big Data, its Quality Data, too!

For the better part of the last two decades I have advised my clients and employers to push hard on their suppliers to adopt either EDI (Electronic Data Interchange) or, in certain more recent cases XML, instead of trading using phone, fax, email or snail mail.  The costs – both in time and in bad data – are too great to continue executing business transations in a completely manual way.  I remember years ago when the web first made it viable, I also started recommending that when suppliers would not trade electronically with them, companies should push the use of web portals my clients could off-load the data entry – and responsibility for bad data – onto their suppliers.  Unfortunately many companies (not my clients, mind you) abandoned electronic trading altogether when their IT shops found supporting portals was easier than supporting electronic trading…but that is a completely different story.

The thing here is the idea of having your supplier handle the entry and validation of data.  They have a vested interest in making sure the invoice information they provide you is complete and accurate or they might not get paid.  In the same way, I see the the social networks, and other companies, that are aggregating information on all their users leveraging the same model…but it is even better for them.

Every one of us must fill out web forms when signing up for almost any service on line.  Information we share includes name, address, email address, age and many other personal tidbits.  The information becomes the basis of the “consumer as a product” business that these companies have adopted where our virtual selves are sold to the highest bidder – either directly for cash or indirectly through targeted advertising and routes.  In a March 11th column for the Huffington Post, former Federal Trade Commissioner Pamela Jones Harbour wrote “To Google, users of its products are not consumers, they are the product.”

Imagine having a “self-building” product.  It is self-aware and concerned about itself so it naturally wants to make sure the information about it is accurate so on a daily basis it makes itself better.  From a data quality standpoint, there are few better ways to make sure your data is accurate.  Make sure the products – er, those that enter the data – have a vested interest in it.  But the data quality aspect of social data is even better!  For the most part, the data collected from on-line searches, click-throughs, web browsing and everything else you do online – whether collected overtly or covertly – will pretty much be accurate – it is near impossible not to be.

Social data, then, is a data managers dream.  The real challenge is what to do with all the data.  With few data quality issues, data managers responsible for social data are left with working with their business counterparts to figure out the best ways to exploit the data they collect.  Which leads me to an old quote from Google’s ex-CEO Eric Schmidt when speaking with The Atlantic: “Google policy is to get right up to the creepy line and not cross it.”  With Chloe Albanesius at reporting that at least one recent company defector thinking that Google+ has ruined Google, perhaps they’ve stuck their nose just across that line afterall.  Quality Big Data can be a scary thing.

All I can say is “Stay behind creepy line, please!”


Size Doesn’t Matter to Donkeys

Should it to you?

When it comes to treats, my donkeys don’t seem to care how big the cookie is – just that they are getting something.  Thus, I keep my donkeys lean and my costs low by breaking their special horse treats in two before letting the boys scarf them down.  Likewise, I break our dog treats in 2 or 3 before dispersing to the pack.  Polly in particular has benefited from a lean diet supported by pieces of biscuit as opposed to the whole thing.  Of course, I could just purchase smaller treats, and sometimes I do.  But what about your company?  When it comes to your EDI and B2B programs, do  you go for the small provider or the big one?  For you, does vendor size matter?

In recent articles in EC Connexion and the VCF Report, I outlined my thoughts on the pending GXS/Inovis merger.  In the former I focused more on the B2B practitioner and what it might mean for them.  In the latter I leveraged some of the first and provided a higher-level view focused on the business and managerial reader.  In both I pointed out that what you get out of the merger depends on how you and your company reacts.  There is enormous potential in the merger, but companies will only benefit if they hold the new company accountable for brining the right mix of solutions to the table – a mix the merged company will have access to, but might not fully exploit.

However, this merged company will be best positioned of all extant B2B players to provide the full end-to-end services on a global basis.  In fact, others who claim to be global will – when you look under the hood – only have one or two people in emerging markets like China, India and Brazil where the merged company will have significant resources in those locations.  In this case, size does matter.  If you have global operations – whether it be your own enterprise or companies in your extended supply chain (supplier, supplier’s supplier, customer, customer’s customer) you must consider the ability of your B2B partner to help you manage that ever changing supply chain as you move production from country to country and from far offshore to a mix of far and near shore manufacturing, and as you change your mix of carriers as production changes. 

Successfully implementing, maintaining and managing a global business requires that your partners be there – and that they have been there doing what you need done for a while.  You need them to be smarter than you when it comes to new countries and new regions.  There is no use partnering if the partner can’t bring something to the table.  There isn’t another B2B player that has the global reach within their own organization than the GXS/Inovis merger will bring.  Much of that comes from GXS, but with the addition of Inovis’ unique solution offerings, the global capabilities of the merged company will be significant. 

No matter the size of your business, if you do business globally, size should matter to you.  There are other fine players in the space – Tie Commerce is one smaller player with strength in Europe, the US and Brazil – but if you truly need round-the-world visibility, accountability, presence and capability, your partner’s size will matter.  Don’t underestimate the knowledge that doing business in a country can bring, or that being able to do business in nearly 20 languages can get you.  Legal, business and technical knowledge are all there for leveraging.  The question is, if size does matter and you choose the merged company, will you demand that they leverage their size (both global presence and overall solution set) to help you do business better with enhanced global visibility, data quality and synchronization, compliance management and full, uninterrupted end-to-end automation?  Or will you just ask them to do what you’ve always done – automate purchase orders, invoices and ship notices and call it a day.  If that’s all you do, your stakeholders won’t be happy and, when it comes to your B2B partner, size really won’t matter after all.

Data Quality Doesn’t Get Any Respect

Halloween Costumes have some fuming.  But should stakeholders be the most upset?

David Loshin of DataFlux recently asked “is Data Quality Recession Proof?“ I think the answer is “yes”.  There is no doubt that data quality is still a problem in an economic downturn. In fact, I think it is recession irrelevant.  It continues to have a bad impact and is demanding attention, but I wonder if it is getting any more attention than it has in non-recessionary times.  I believe it should because – as Charles Blyth points out in a comment on David’s blog – organizations are more efficient with greater data accuracy. The fact that so much attention has been paid to new ways to move data (see The Long Tail of B2B Standards by Steve Keifer) that we miss the need to get the data right! 

Is Good Data Really Such an Alien Issue at Target?

I’d say it is time to stop focusing on XML and other means of moving data and spend more time getting the processes, culture and technology in place to assure accuracy of the data.  It does little good to move it quickly or in a new format if the data is bad. 

So, any time is a good time for data quality.  The problem is getting enough enterprise focus on the issue.  Yet when you do get corporate attention, is it for the wrong reasons?  Take Target, for instance.  They were the first to respond to the controversy regarding the “illegal alien” Halloween costumes they (along with Wal-Mart, Walgreens and others) sell.  This weekend some people started suggesting the costumes are not “politically correct” and that caused the risk-adverse retailers to pull the web pages for those products from their online stores.  In Target’s case, the fact that they are (or were) selling the costumes was explained as a “data entry” error (Hey, they got first dibs on this excuse – wonder what Wal-Mart and Walgreens will say?). 

If a data entry error (meaning data quality) is really the cause, was that error in the ordering of the multiple SKUs – like “We didn’t mean to enter in these SKU numbers and quantities along with distribution centers to deliver them to and the dates on which to deliver them”?  Or did they want to order them but not post them on the web site or put them in the store for sale?  In this case was the data entry error in revealing they had a horde of the products that they weren’t planning to sell?  I can see the CEO in a staff meeting:  “Let’s order these things and not sell them, thus keeping everyone from getting them because we think they are not politically correct!”

Let’s face it: Data Quality becomes a convenient scapegoat even when it probably isn’t the problem.  Yet, when it really is a problem it is too often ignored.  I know how bad the data is because in my last role I was able to analyze the quality of data being sent to retailers from their suppliers.  The data crossed numerous industry sectors so we know the problems are persistent and numerous.  We know the companies using the GXS Product Data Quality tool are experiencing better data, however, I suspect that only the retailers are really making use of this data.  Most suppliers don’t take the cleansed data and refresh their own systems with it.  Thus, retailers are ordering from good data and it won’t line up with the data in supplier systems.  The suppliers have to know their data is bad.  Do they just not care? 

In the end, a data quality program that fixes data on one end but not the other of a strategic relationship really is little better than having the same bad data on both ends.  The only difference is that retailers can wield a heavier compliance stick when they know their own data is good.  I think it is time to divest from those suppliers and their retailers.  Since supply chains with divergent data on each end are destined to be less competitive, they aren’t a good place to invest nor a good place to continue holding stocks.  Unfortunately most retail supply chains are like that. 

Getting back to the costumes, these retailers have sophisticated, automated systems.  Even for seasonal products Wal-Mart has decided against seasonal suppliers so they don’t have to manage occasional, manual business relationships.  They want automation with repetitive, reliable and accurate processes with correspondingly good data.  If the information about the costumes in Target’s systems  was wrong I could see it being a data quality problem.  But the fact they were sold in the first place?  Don’t blame it on errors in data – keystroke or otherwise.  There are enough of those without adding additional, fictitious fault.

Data quality just doesn’t get any respect.