Omitting Duplicates

[pigslow]

I have a script that does a find then does a couple of constrains to omit some of the found records to get the records I really want. The last constrain Omits ! on the key id for the records.

I run this once a week and it has been fine for about 6 weeks. For some reason it it finding records that are not really duplicates and is omitting them.

The only thing that was done was that we removed a large portion of the records that were not being used. Would this have possibly created a problem with the ID that can not be seen on the surface?

Is there another way to omit duplicates?

[PeterHW]

Hi, is this your primary key field you are searching duplicates for ? If so, why do you allow for duplicates in the first place. Of course, I have no idea how the data is structured and what it is used for, but allowing duplicates on a PK field is bound to cause trouble. How is the ID field set ?

Regards,

Peter

[pigslow]

This "ID" that I am doing the find on is a primary in another table.

The data is imported into this filemaker table weekly from a CSV file. That CSV file always has duplicates in it. The file is generated from a MySQL database. The MySQL database contains logs of how many times these records are used. Each log represents a weeks worth of access of those records. The CSV file contains all of the weekly logs from inception of the database to the current date, so the file grows weekly.