Data Hygiene Woes Never Faced Before
By Barbara Love
Ten years ago, there was a new net person joining our population every 22 seconds. That has increased to one every 11 seconds, according to Rod Ford, CEO of Cognitive Data, which has an advanced data hygiene technology used by catalogers, financial services, retailer and non-profits, as well as publishers.
The list hygiene problem has accelerated because today's consumers are more fluid in the identities they use and move more frequently than ever before, Ford explains.
"We process 4 to 5 billion records a year, so we see lots of consumer data and are able to recognize the trends and patterns of error that is not being corrected by normal data quality methods," Ford explains. "The problem is much larger than most people think."
"There is an explosion of marriages [and] divorces;7,000 marriages a day and 3,500 divorces a day. And hyphenated names are a growing problem. Never before have so many females used so many varying name combinations to identify themselves.
"Women generate 75 percent of the consumer purchase activity in this country, he adds. "If a woman changes her last name, 60 percent of the time she will also change her physical address.
"And what people don't realize is that 15 percent of the time she changes her last name, she will also change her first name;maybe Elizabeth to Beth or Beth to Elizabeth. Women who get married want to change their lifestyle and their look.
"None of the traditional hygiene methods were designed to follow that person's new first and last name changes and new address."
Additionally, the explosion of multicultural names is also making it difficult to identity customers, donors and subscribers. There are typically several correct spelling representations of these names as well as challenges created in the accurate capture of the name in the form of typos, misspellings and mispronunciations.
Another challenge to existing list hygiene methods is increased mobility in our society, he points out. "We have never had a more mobile populace. People are moving more often. The population densities are changing through urbanization.
"120,000 people move every day. Mainly, we are no longer rooted in our work force;flexible jobs, access to job opportunities through web job portals and the flood of consumer credit that provides a cushion between jobs results in people no longer being as tied to a geographical location as before.
Everybody faces the same challenge in keeping their data accurate. These social issues are generating data error at an alarming rate. The question is what is the error in your file and what can you do about it?
"Direct marketers are using the same basic processes for list hygiene they used 10 years ago," Ford points out. "They are simply comparison based and have no native intelligence of their own. Frankly stated, these processes were never designed to identify the kind of name deviations and mobile geographical preferences that consumers have today."
In general, you have to have a better technology to track the location and identity of subscribers, he says. "Publishers have to use some referential sources of data to match all of the identities and names and the geographical locations from which they are purchasing."
"Publishers have to think beyond the classic hygiene processes. There are advanced data processing technologies that allow them to compare their subscriber files against a referential knowledge base of the identities consumers utilize and where they live. The old processes are no longer adequate for a mobile society and a fluid identity society."