Find Similar Duplicate Records

July 2, 200322 yr

Hello,

I am merging some mail lists for a customer of mine and we are running into some difficulties. We are trying to compare a list of 15,000 names and addresses to about 12,000 names and addresses in his current database. We are updating his database with the NEW list. Everything works fine when the records match. However, sine sometimes the company name or addresses may be spelled differently then there is no way to match and import these records. Is there a way to import "similar" records together or remove the "similar" records? I have a remove duplicates script that works great when the data matches but it isn't much good when:

customer A is in the old database as: 125 N. 6th Street

customer A in the new database is: 125 North 6th St.

Any ideas?

Thanks in advance.

July 2, 200322 yr

This really comes down to an unsolved problem in AI. For every example like the one above, I can give you one that the smartest clerical in your office can't solve. About the best you can easily do is to use a REALLY unique field such as phone number or email address to make the comparison (and hope they didn't change in the time span between the two entries).

Names and addresses have too many variations to be useful without VERY complex software. Software used by mailing houses might be able to convert your entries to standard addresses. If they would do address conversions on both files to the standard addresses, this might be a start.

-bd

July 2, 200322 yr

Hi Pmmarketing,

You might take a look at a demo by CobaltSky Find Similar Values

It's structured to be used comparing a main and related db (it's a self-join) so you won't even have to import the two dbs. It won't solve your problem, but may provide a list of items that are close enough to narrow them down for you.

You may have to tweak it a bit since it depends upon User input. But I believe that you might be able to use it for your situation with some modification.

It's probably worth a looksee, anyway!