Jump to content
Sign in to follow this  
susan

deleting duplicate files

Recommended Posts

I have a file with many many records, all of which have at least one duplicate. I want to be able to find all the unique records and then delete the duplicates. Is there a quick and dirty way to do this?

Susan

Share this post


Link to post
Share on other sites

Hi Susan,

You probably already know that you can use the ! Find operator to find duplicates in a field.

Then you can sort your found set by the values in that same field -- thus lining up all the duplicates.

Then you can use a script that loops through the found set, omitting the first instance of each value, but leaving the duplicates. After the loop gets to the bottom of the list, you can delete the remaining found set.

For a quick reference on using omit in a looping scripts, check out this thread:

http://www.fmforums.com/cgi-bin/ultimatebb.cgi?ubb=get_topic&f=15&t=000821

Hopefully this is enough to get you started. Good luck!

Share this post


Link to post
Share on other sites

As I read it, Andrew's solution works if there is only one duplicate for any record. ere is a sample of a script I use to delete duplicates where there is/maybe more than one duplicate.

Enter Browse Mode

Perform Find [ Request 1: master field ! ]

[ Restore find requests ]

Sort [ Sort Order: master field (Ascending) ]

[ Restore sort order, No dialog ]

Go to Record/Request/Page

[ First ]

Set Field [ g_duplicates, master field ]

Omit Record

Loop

If [ master field = g_duplicates ]

Delete Record/Request

[ No dialog ]

Else

Set Field [ g_duplicates, master field ]

Omit Record

End If

Exit Loop If [ Status( CurrentFoundCount) = 0 ]

End Loop

Set Field [ g_duplicates, "" ]

Show All Records

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.