Free Services Now Available

In-line with TDC’s commitment to supporting Small Businesses, we’re offering an opportunity to tap into our list cleaning resources for free.  There are of course conditions that apply, but this will not negate the benefit for the majority of Small Businesses.  We hope that as many of you as possible, can take advantage of this limited offer!  More information can be found here.

QLD Flood Victims

The Data Company has all of it’s thoughts and prayers at this time, with all the families dealing with the tragedy of the QLD Flood event.  As TDC is a SE QLD based company, we’re well and truly aware of the devastation that this has caused.   I’ll personally be up in Brisbane helping clean-up my parents place.  TDC is still fully operational however, and has not been affected by this event.

I’d like to take this opportunity to urge all persons affected by this event to remain safe, and ensure that your electricity is off and that under no circumstances do you consume contaminated water.  Be aware of mosquito’s, and wear the appropriate repellent(DEET) to ensure you remain safe.

Best luck to all of those involved, and I’m sure that if we all pull together, we can bring Brisbane back to life quickly!

Why do you need Data Management?

How do you convince someone that they need your service or product, when they have no real idea of what it is you’re offering or what it is your product/service can do for them?  That’s quite often the biggest challenge for “Data Management” companies.

Data Management is a broad term, which encompasses the general concepts around how data is handled.  This is from creation, through to the final use of the data.  During the life cycle of a lead/prospect/customer, multiple events occur which impact the data present and the subsequent quality of the information at hand.  Positive interactions, are those when you have direct contact with the lead, and are able to update/enhance the record in question.  Yet even these so called “positive” interactions, can create a management overhead if not done properly.  No interaction, is certainly a negative, and will lead to your data becoming stale and unproductive.

At the end of the day, it all comes down to rules;

  • Do you have them?
  • Are you able to enforce them?
  • Are your staff aware of them?
  • Are the constraints within the GUI?

If you answered “No”, to any of the above, then you’re in need of some sort of “Data Management”.  Your needs could be as simple as normalising your data or  maybe you need to validate what has been done, or perhaps the effort that has been done has now created duplicates within your system.  It doesn’t matter what needs to be done, The Data Company can help.

Where’s the value?  or, How do I justify the cost?  There are a number of ways to look at this;

  • Reduction in effort, due to no duplicates
  • Greater performance from marketing campaigns, due to your increased ability for contact
  • Reduced costs from wastage, such as Return to Sender (RTS)

So don’t put it off any more and get in contact with us today to find out how we can help you, it’s never too late and it’s always just in time.

TDC and DataTools P/L

We may have been quite lately, but we’ve been quietly working! A lot of great things are happening, and I’m please to formally announce an alliance between The Data Company P/L and DataTools P/L. As of late last week, these two organisations have joined forces to offer up a complimentary solution for all tastes. Coupled with the high strength and versatility of the DataTools suite of products, and the engineering and solutions orientated ability of The Data Company, we’ll conquer the world! J Well, perhaps not the world, but we’ll start with you!

This alliance, will ensure that The Data Company is able to offer the best possible solutions for Address Validation, Data De-dupe and ongoing Data Management. The solutions are really only limited by our imagination! We can create solutions to handle the point of data entry, backend processing or scheduled batch processing, all coupled with fantastic reporting opportunities.

DataTools offers us an unique opportunity to provide solutions for the smallest company looking to keep a couple of tables “clean”, using a standalone desktop product, all the way to the largest corporates looking for a fully automated and integrated solution. DataTools’ unique range of products is able to handle all scenarios.

We don’t just recommend DataTools, we USE DataTools products in our own environment! So you know, that you’re getting a Tier One solution when you choose DataTools and The Data Company.

For more information, contact me directly and check out the DataTools website by clicking on their logo below!

Press Release: TDC now a proud supporter of RACQ CareFlight Rescue

Well, what an exciting day it is. Today, we officially became a proud partner of RACQ CareFlight Rescue!

This is quite a momentous day for us, as I am sure you can imagine, and we’ve already highlighted the first project to be tackled by TDC for CareFlight. More on that later though…

For now, a bit about CareFlight;

The CareFlight Group Queensland collectively represents Australia’s largest air medical retrieval operation, carrying out more than 3000 domestic and international emergency rescues each year. CareFlight is one of a select few services in the world to integrate its own full-time critical care doctors, air crews, dedicated medical helicopters and jets to dramatically reduce response time.

Once again, we’re truly excited to be partnered with CareFlight Rescue, and we’re sure that this will be the start of an very powerful and long term relationship!

How to minimise the impact of data de-duping within a relational database

I’ll give you all fair warning. We’ve been building up to this, and it will be the heaviest of the series.

I know that a number of you will have cause for concern, when it comes time to “cleanse” your database of duplicates. It’s an exercise that requires a delicate hand, and skilled knowledge of the existing database structure. To this end, it should only be a task that true professionals handle.
So, what are the issues? Well they’re relatively simple to understand, but I’ll let this diagram explain them in part;

In the above diagram, we have “Table1” as the primary table of the database. Shooting off from this, there are two “reference” tables. These tables could contain anything, such as contact information or historical records of transactions. The deduping that we’ll be undertaking will be on the primary table, or in this case, “Table1”. Within this table we’ll setup “Parent to Child” relationships, or “Chaining”. The idea behind this is to mark the most “valuable” record to the business, and keep this as the only remaining record in that chain. How to find this record is a business decision, and whether or not to enhance that record with “child” information is also a business decision.

Once the “parent” record has been determined, all child records within the relative “reference” tables will need their respective ID’s updated to point to the “Parent” record. It’s not an overly complex process, yet the implications of erroneous work are devastating.

Imagine for a moment, that you’re in the business of selling infants clothing. And during the course of your business, through either multiple touch points or mismanaged data, you’ve created duplicates in the system. Further to this, the “Table1” is the parents, and the reference tables contain the children in one table, and the articles of clothing in another. So, you’ve updated your data, and you send out your first Direct Marketing piece to Jeff and Di Jones and their child Alexander who just turned three. Imagine that the updating of the records was incorrect, and that Jeff and Di don’t have a son Alex, yet have a daughter Renee. Where’s your credibility just gone? Yep, thrown out with last year’s growth spurt clothes!

So as I’ve said, it’s not a hard process to undertake, but it is a process that requires a great deal of attention and quality control. If this story sounds all too familiar, let me know, and we’ll see what we can do to help.

Fuzzy Logic, the benefits and pitfalls.

Article Four: Comparing data sets with fuzzy logic, the benefits and pitfalls.

Welcome to week 4, the 2nd last article in this insightful series.

Fuzzy logic, what is it? Essentially fuzzy logic is a process. It uses matching algorithms to determine if two or more records are the same, when they are not. So if we had two records with the same last name, but one record was at 1/1 Apple St and the other at 1 Aple St, then depending upon the algorithm, Fuzzy matching would be able to determine that these two records are actually the same. As you can see, the advantages of fuzzy matching come into play when either a data entry, or other issue, reduces the completeness/accuracy of a record. In these cases, fuzzy matching is able to still perform a result that the business finds satisfactory.


Well the benefits certainly outweigh the pitfalls, but no system that I have worked with is 100% perfect and as such, you only have two options. You either “Over” dedupe your data, or “Under” dedupe your data. And this decision would be solely based on the data you were deduping, and why you were deduping it. For example, if you’re looking to send out a direct mailer to your existing customers, then you could accept the small loss of records and “Over” dedupe. This will represent a higher level of professionalism, to ensure no customer receives two mailers. Conversely, if you were to look at your customer database and merging this to remove duplicates for the sake of a data migration, you’d make the decision to “Under” dedupe your data, as you’d not want two distinct customers to be linked together.


As already stated, no system is perfect! You WILL have to make a choice between “Under” or “Over” deduping. But I don’t see this as negative. It’s more important to understand the weakness of a process and deal with it, then to just blindly use and trust it. When it comes to deduping data sets, it’s not just “Plug and Play”, you have to work with the system to develop the rules inline with the business and the respective data sets. I can honestly say that so far, no two companies have been the same. And the reason for this, each respective company has had their own procedures employed to derive/key/import/purchase data.

There are of course examples where fuzzy logic is not required. This is generally the case when the data that you’re manipulating is of such a high quality already, that there is no chance of erroneous entries. Electoral Data, Council Data and some other government datasets spring to mind. They are however, few and far between.

Excel and CSV files…

How to open a csv file in excel, without loosing the Zero.

Seems basic?  You’d be surprised just how many people get stuck on this.  The fundamental issue is that excel recognises the phone number as a “number” column.  Which it is.  But how many numbers do you know that start with ZERO?  That’s right…NONE!

So, how do you trick, Excel into letting you see the zero?  Read on;

Step 1 ) Right click on the file and rename the file extension from .csv to .txt.  (if you can’t see file extensions, go to: Control Panel – Folder Options – View – Un-tick the box “Hide extensions for known file types”)

Step 2 ) Open Excel, and make sure that it only takes up half of one screen.

Step 3) Drag the text file, with your left mouse button, into the Excel sheet and let go of the mouse button.

Step 4 ) Select column “A” in excel.  You should now see the entire column highlighted in blue.

Step 5 ) Go to the menu and click on “Data”, then “Text to Columns”. At this point, a new window will pop up.

Step 6 ) Ensure that the radio button is on “Delimited” and click “Next”.

Step 7 ) It’s a “Comma” separated file, so make sure “comma” is ticked.  Click “Next”

Step 8 ) Scroll across to the phone field and select the phone field(the phone field will now be highlighted in black), and ensure that it’s “Column Data Format” is “TEXT”.  Click “Finish”.

Ensure that the phone column is selected=

Step 9) Auto calibrate your column width by clicking here once, then double click in-between column A and B.
Clicking here will select your entire sheet.


Just a little “Gotcha” to be aware of…If you open a CSV file, and then save it (no matter what file type you save it as) you WILL LOOSE THE LEADING ZERO of ANY number!

5 Part Weekly Series, Part 3

Article Three: Why haven’t you tried MS SQL Express yet?

Last week we touched on the value of leveraging MS Access when using MS SQL.  At TDC, we’re using full blown versions of MS SQL, but that doesn’t mean you can’t get a taste as well.  MS is generous enough to allow everyone access to a completely free version called, MS SQL Express.  With the latest iteration (2K8 R2), we’ve seen an increase in the overall DB size allowed, grow from 4gb (2K3) to 10gb (2k8 R2). You can download the runtime and management interface from here.

So, what’s stopping you from jumping with both feet in?  Well, you do need to have a bit of an idea on how to write Transact Structured Query Language (TSQL).  Without this, it’s pretty hard to get anywhere.  SQL 2k8 R2 has become a lot more intuitive, as in it will actually help you write the query, but it’s still reliant on initial user input thus, initial user knowledge.  But fear not all you novices out there, there is a simple “cheat” you can employ.  Within Access, when you’re putting together a query in design view, you have the option to also write that in SQL.  If you’re lucky enough to have access installed, then this is the least path of resistance.  Just write put your query together in design view, then click on the design button to drop down the “SQL” item.

If not, then simple commands like “SELECT * FROM [table]” will get you started.  After that you starting putting variables into the “WHERE” clause, such as, “SELECT * FROM [table] WHERE like ‘%variable%’ ”

Limitations???  Sure, nothing that is free in life is perfect!  And the fundamental missing element from the “Express” version, is the ability to schedule jobs.  It is possible, with 3rd party software, but it’s not a supported feature.  As such, user beware!

So you want to dip your toe in, but unsure how to really get started.  It’s pretty basic, just download the corresponding 32/64bit package from the above link, and install.  If you need help in putting together your SQL code…you know where I am! J

Happy SQLing!

Missed the first two articles???  We can’t have that!

Article 1

Article 2

Services Addition, finding lost customers.

TDC has just added to it’s services, a process that will allow stale data sets to be “refreshed”. So, if you’ve got an old customer data set that you haven’t touched in years, this is the service for you! Head on over to here, and check out what we have on offer. If you’re trying to figure out how to update that old data set, this is how you do it!