import mapping issue

[step wright]

I constantly have issues with my files whereby the field mappings constantly mismatch.

Just to describe the situation:

I have a 'master' file running off of a server. This regularly has additional fields added to it's tables.

I then have a series of local files that have data imported into the master. They regularly have extra fields added to them.

The problem is I have hundreds of local files run by many users, and upwards of 10 complicated import scripts in the master.

What I find is happening is that almost everytime I add (or maybe delete) fields from one or both of these files, the mapping goes askew and i have to remap everything.

I spend about 30% of my support dealing with issues related to imports. Am i missing something?

Any help desperately appreciated.

[Søren Dyhr]

They regularly have extra fields added to them

An as close as posible to a fully normalized structure? What is the fields per record ratio, are you sure it weren't better to break stuff out into related records instead of adding fields.

--sd

[Ender]

What I find is happening is that almost everytime I add (or maybe delete) fields from one or both of these files, the mapping goes askew and i have to remap everything.

Yes, since the import mapping is by creation order, it will have to be reset when earlier-created fields are deleted in either file.

I'd suggest, if possible, using a completely networked solution so no synchronization is necessary. Otherwise, manage your changes so they are rolled out in bigger change intervals that everyone receives at once, and don't delete fields until you can remap and test the imports on the rollout version.

But Soren's advice is also useful here. If the business isn't changing it's needs, then a relational structure will accommodate things as the data changes. Maybe that's all you really need.

[step wright]

I know what you mean about breaking the data out into different tables. However, we already do that to a large degree. These fields unfortunately do need to be in the same tables.

The local copies also cannot be networked for operational reasons.

So unfortunately, it seems i am doing nothing wrong. I already do incremental updates from a test copy and try and keep field structure changes to the minimum.

Oh well, not real short cuts. THanks for the quick responses.

Cheers

[Søren Dyhr]

I know what you mean about breaking the data out into different tables. However, we already do that to a large degree. These fields unfortunately do need to be in the same tables

Well, I do not agree here - one thing is the layouting matter of the issue which can benefit from the seaparation model, but the structural side of it, can via an extra table act as grid for related records ...take a look at:

http://www.fmforums.com/forum/attachment.php?attid/7285/

...and tell me before scrutinizing it, how many fields are required for such a page full ...well then poke into it!!!

--sd

[step wright]

Cheers Søren,

I'll take a look

Steve