I’ll give you all fair warning. We’ve been building up to this, and it will be the heaviest of the series.

I know that a number of you will have cause for concern, when it comes time to “cleanse” your database of duplicates. It’s an exercise that requires a delicate hand, and skilled knowledge of the existing database structure. To this end, it should only be a task that true professionals handle.
So, what are the issues? Well they’re relatively simple to understand, but I’ll let this diagram explain them in part;

In the above diagram, we have “Table1” as the primary table of the database. Shooting off from this, there are two “reference” tables. These tables could contain anything, such as contact information or historical records of transactions. The deduping that we’ll be undertaking will be on the primary table, or in this case, “Table1”. Within this table we’ll setup “Parent to Child” relationships, or “Chaining”. The idea behind this is to mark the most “valuable” record to the business, and keep this as the only remaining record in that chain. How to find this record is a business decision, and whether or not to enhance that record with “child” information is also a business decision.

Once the “parent” record has been determined, all child records within the relative “reference” tables will need their respective ID’s updated to point to the “Parent” record. It’s not an overly complex process, yet the implications of erroneous work are devastating.

Imagine for a moment, that you’re in the business of selling infants clothing. And during the course of your business, through either multiple touch points or mismanaged data, you’ve created duplicates in the system. Further to this, the “Table1” is the parents, and the reference tables contain the children in one table, and the articles of clothing in another. So, you’ve updated your data, and you send out your first Direct Marketing piece to Jeff and Di Jones and their child Alexander who just turned three. Imagine that the updating of the records was incorrect, and that Jeff and Di don’t have a son Alex, yet have a daughter Renee. Where’s your credibility just gone? Yep, thrown out with last year’s growth spurt clothes!

So as I’ve said, it’s not a hard process to undertake, but it is a process that requires a great deal of attention and quality control. If this story sounds all too familiar, let me know, and we’ll see what we can do to help.