Removing all duplicates to only have unique records

[Nestor]

Hi,

I have a list of records that have duplicate information. My goal is to remove the duplicates, BUT stay with 1 Record from each of the duplicates. Kinda like feature in excel 2007 "remove duplicates"

Is this possible?

[LaurenKuhlman]

Nestor,

I was looking through your posts and found this. I just wrote a lengthy response to your other question so, I thought I'd help here. This is not a built in functionality but its possible with a relationship and a looping script. PLEASE BE CAREFUL!

Essentially, draw a relationship based on something you know is a duplicate. In my sample, I created a calculation based on Last Name, House Number and Zip Code.

Again, PLEASE BE CAREFUL. This can be dangerous. Test thoroughly in a backup. I suspect the dangerousness (word?) of this is why no one has responded.

[The Big Bear]

Nestor

I have done this. Make a field called duplicate and another field called gname and make this field a gobal field and sort your file by a field that would have the duplicate information and write a looping script something like this

goto last record

set field gname;name

goto first record

loop

if gmane = name

set field duplicate:x

else

goto next record exit after last

setfield gname;name

end loop

enter find mode

set field duplicate; X

perform find

delete all records

show all records

The script is not tested so it will need some tweeking but it will give you an idea of how it done.

This way you have not marked the first record for each duplicate and have marked all the record that are duplicate and then you preform a find for all the marked records and then delete them.

Please try this on a copy before during orginal.

Lionel

Edited December 12, 2009 by Guest