De Dupe

[Hoya]

I have been having problems de duping my database. I want to find the duplicate records but I only want the duplicates to be found, providing me the option to delete the found set. I have tried to use a script with a global and mark field however, this is not bringing up all the duplicates. For example, when de duping by email, not all duplicate records are found. It appears that even though I speciffy to search only the email field, it searches the entire record. And then if the rest of the record information is different, both records are not recognized as duplicates even though the emails are exactly the same. Please help!!!

[Chuck]

In the FileMaker manual there is an algorithm for writing a script that finds the duplicate records. Look in the index of the manual for duplicate.

Chuck

[Hoya]

I followed the manual to write the script, word for word, however the script does not find all the duplicates. For example. If I'm de duping by email, it only brings up the records that have the same email as well as the rest of the contact information. It does not find other records that have slightly different contact informaton but the same email. So even though I specify to only de dupe by email, it still goes throught the whole record and misses some duplicates...

[Kurt Knippel]

quote:

---

Originally posted by Hoya:

I followed the manual to write the script, word for word, however the script does not find all the duplicates. For example. If I'm de duping by email, it only brings up the records that have the same email as well as the rest of the contact information. It does not find other records that have slightly different contact informaton but the same email. So even though I specify to only de dupe by email, it still goes throught the whole record and misses some duplicates...

---

Hmm, duplicate checking...well first you need to CLEARLY and COMPLETELY define what is a "duplicate". Is John Jones and Jon Jones duplicates? How about Bill Abbot and William Abbot? Is [email protected] and [email protected] the same?

Once you clearly define what is a duplicate you can start by looking at candidates records and them comparing them by all of your criteria.

For instance maybe you start by saying that anyone with the same Last Name, First Initial, and Company Name is a candidate duplicate. So you setup a calculated field to concatenate all those fields and then use the ! as your find criteria which will find all duplicates in that calc field. Then you can loop through the records and perform a more detailed check to see if they are indeed duplicates based on your CLEAR and COMPLETE defination of a duplicate.

[Zipididodad]

A note about the "find duplicates" script in the FM manual,...it only finds duplicates if the duplicates are next to each other in the recordset.

It will not find duplicates which are scattered thru the database...

FYI

[Zipididodad]

Hi Steve,

You are absolutely right. The script works perfectly now that I sort the records prior to doing the loop. My mistake. Sorry everyone!

[spb]

Not true. The example script includes a Sort step that takes the scattered dupes & organizes them next to each other first. The example script I'm looking at is on page 10-13 of the FMP 5.0 User's Guide. Possibly you are looking at the manual for an earlier version?

I have adapted this script for my own db and it works like it oughta. It will find dupes on a single field and flag them. If other fields differ, it still flags them. You can add script steps to confine the search to multiple fields if needed, though. In the original email dupe problem, it will find records with duped email addresses and different info in other fields.

By flagging, instead of automatically deleting them (NOT recommended), the user can scroll through & look at them. If any are deemed not to be deleted, uncheck the flag field. Then do a manual refind on the flag field & delete found set.

Steve Brown