Dec 18 2008

Clean up your CSV!

Have you ever heard of the phrase ‘garbage in, garbage out’?

It’s used in the field of computer science and ICT to refer to the fact that you’ll only get useful results from a computer program if you give it ‘correct’ data to work with in the first place: for instance, you’ll only get meaningful results from a Google search if you give it a half-decent description of what you’re actually looking for.

Google, however, can at least gain some results from your query (albeit irrelevant ones), no matter how terribly you may have written it; but other computer programs simply won’t function if they have ‘bad’ input. One such example, used by many of our clients, is the mailing list (used to send out e-mail campaigns). Mailing lists need to store a large number of e-mail addresses, names and other personalisation data. Typically, this data will be exported from CRM software to a .CSV file, and then imported into an email broadcast tool. These tools require you to import data that’s formatted just right, or else it won’t work. A CSV export may contain extra data we don’t need, or user’s names might not be in the correct format. This typically requires some amount of ‘manual handling’ to sort out.
For instance, imagine that your e-mail campaign software requires that we import data of the format:

Bradley Ford, bford@abccompany.co.uk
Joe Middleton, jmiddleton@abccompany.co.uk
Bethany Brookes, bbrookes@abccompany.co.uk

Which is fine and dandy, until you get handed a file of customers which looks more like this:

FORD, Bradley D., +441234 567894, BFORD@abccompany.co.uk
MIDDLETON, Joe P., +441345 365927, JMIDDLETON@abccompany.co.uk
BROOKES, Bethany J., +441735 294754, BBROOKES@abccompany.co.uk

And with over 1,000 subscribers to your mailing list, this is going to take a while to sort out… isn’t it?

Perhaps not! The guys behind online database tool Dabble.db have just released a new free product, called Magic/Replace (tagline: clean up data – no magic wand required!) to cater for just this situation. And it really is just like magic.

It’s a super-simple process to paste in or upload your problem CSV file (Magic/Replace also accepts .XLS and .TSV files):

Importing data into Magic/Replace

Magic/Replace will then show you an example record from your data. You can then copy and paste data between the various fields (and you must copy and paste, rather than re-typing, due to the way Magic/Replace does its stuff). You can change case, add punctuation, delete data and fields… anything you need to do to get that record into your ideal format.

The Magic/Replace editor

Finally, you click preview and you should see the sample data transform before your eyes – as if by magic – and contort itself into the format you need. Magic/Replace will e-mail you the data once it’s done converting it all, voila! You just update the one record to show how it’s done, and the rest will be changed to match. Job done.

Magic/Replace results

This is a really smart, very well implemented little application (our usability gurus approve!) and should save a lot of time and effort for a lot of people. For more information, take a look at the Magic/Replace website at http://cleanupdata.com/. They have a video which shows the magic in action, along with some sample data for you to use to try it out.