Jump to content

Importing without duplicating existing records


This topic is 386 days old. Please don't post here. Open a new topic instead.

Recommended Posts

Hello

  I have a database which imports data from a csv file. This csv file is generated by a machine usually every day. But it does not delete previous data, it just generates the whole data it has on the same file (overwriting it). Obviously I don't want duplicated records in my database.

  The first solution was to actually delete the whole records and re-import them, but this wasn't the preferred solution.

  So I thought of a second solution which I would like to share for any advice.

  I created a second table, Table_B with same fields as in first table Table_A. Then added a relation between the two tables where Table_A::field_a = Table_B::field_a. Then created a calculation field in Table_B which is equal to Table_A::field_a (Table_B::field_related = Table_A::field_a). The import has two steps, first I import from csv into Table_B then I perform a find where Table_B::field_related is empty and then I import what is found in Table_B into Table_A.

  I remember I have read time ago, in this forum I believe, a similar solution with related tables but different approach. Can not find that post.

  Do you think this is a good approach or can something be done differently?

Link to post
Share on other sites

Yes comment, that can be used too, but it would still involve a find on Table_A since the update matching records works on current found set. Beside, who knows, maybe in future it will not be needed to update the imported records because of possible editing of the records.

Link to post
Share on other sites
2 hours ago, Toni said:

it would still involve a find

Isn't your import scripted? Surely the elaborate scheme you have described in your question would have to be scripted? As would be your current solution of deleting all existing records before importing new ones So the question you should be asking yourself is what is the simplest route to the desired goal. 

Link to post
Share on other sites

This topic is 386 days old. Please don't post here. Open a new topic instead.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.