Extra fields for unknown reason

[Charity]

I cannot find it but I read to create 500 fields in every table.  Then name and use them but leave the ones you do not use.

 

I saw someone's file also where this was done except there were only 200 fields like Field25, Field26, but they said they did not do it and they did not know why.  I search for 'extra fields' and get nothing similar.

 

Why do people sometimes do this?  I realize I posted in Relationships and this isn't relationships but I did not know where else to put it.  Thank you.

[Josh Ormond]

I've seen this done on a VERY RARE occasion in solutions where they are trying to avoid future structure changes to plan for future expansion.  However, in general practice, fields are expensive. The more fields you have in a table, the larger the impact on network performance and the length of time finds and such take.  Two-hundred fields is a lot of structure that needs to communicate back and forth between the host and client machine.

 

That many fields may definitely be necessary for some things, but there are often other ways to handle something to avoid that kind of structure.

[Charity]

Thank you Josh.  We found a document called Structural Requirements dated 2010 which says:

 

"If adding a new field, re-purpose existing field by placing the existing field name in the comment for history and renaming it.  This preserves the field creation order and field ids will match during updates to each of the company databases.  Never add fields directly and never delete fields."  It is unsigned and the developer is no longer alive.

 

During this last update, we do not see where it interacts with the data at all and seems to run between the OS itself and FileMaker directly without touching the files themselves. I went back through browser history but there are thousands of links.  I recall it was a small video since I just saw a table view with regular fields then Field16, Field17.  They had nothing in them.

 

I guess this is an old practice that isn't needed any more and we won't worry about it since you aren't.  Thanks for the assist.

[comment]

"If adding a new field, re-purpose existing field by placing the existing field name in the comment for history and renaming it.  This preserves the field creation order and field ids will match during updates to each of the company databases.  Never add fields directly and never delete fields." 

 

 

I am guessing the reason behind this was the import mapping bug - read about it here:

http://fmforums.com/forum/topic/67001-import-mapping-bug/

 

 

Whether that's a good reason is another question - hard to tell without knowing what "updates to each of the company databases" means. In any case, if you're in v.11 you may still be affected by the bug, so be careful.

 

 

***

BTW, thanks for your welcome the other day.

[bcooney]

I'm thinking you saw this:

 

http://www.filemakerhacks.com/?p=7380

[Charity]

Wow.  What a read - both of them.  Yep, this is the stuff, Barbara, thank you.  And the import bug is outrageous also, I appreciate it Comment.  It all seems connected because import map uses field ids also, right?

 

 

The reason for doing this is to maintain internal id parity between fields in the your copy of the Data file and the client’s copy of the Data file. 

 

It sounds important.  I know our sister company has many imports which might explain those extra fields and the document we found.  We will run day and night and maybe it facilitates administration of data changes but it feels a bit much.

 

What alternative could I use to prevent this internal id parity from happening?  I will watch for the import bug and add fields in specific order always.  But most importantly, I would never let anyone get into define databases. 

[comment]

 It all seems connected because import map uses field ids also, right?

 

Yes, and it was the mention of field ids in your document that led me to believe they were trying to preserve their import mappings.

 

 

What alternative could I use to prevent this internal id parity from happening?

 

Ahm, I think you want to maintain id parity - not prevent it. Anyway, other than upgrading to v.12, you need to recheck your import mappings any time you add or delete (esp. delete) a field. Either that or follow the existing strategy of not adding/deleting fields. Although it's kind of 'two wrongs make a right', it does work.

[Charity]

I am bothered by this and I appreciate your input  The link referenced is by a top developer also and it was recent publication using version 12.  I searched and can find nothing on id parity linked to fm that will help explain where it might come into play.  If only during import then it will not affect those in version 12 and the extra fields process can be dropped and I would let him know since it will save him heaps of time as well as me heaps if I do not follow that suggestion.  

 

But if there is any other place where id parity might matter then I would like to know so I do not stumble upon it.  I suppose I should ask him but I prefer this site and you people and Comment I know you know this stuff.  If it only affects unix or something then I wouldn't worry either.  I read to keep number of fields down and this "extra fields" stuff is in stark contrast to all that other advice.  

 

Or if there are links I can research maybe under a different name than 'id parity'?  I appreciate everyone here who helps me.

[comment]

The link referenced is by a top developer also and it was recent publication using version 12.

 

The article by Kevin Frank describes "speculative and experimental techniques that are in the proof-of-concept stage." I don't think you need to worry about anything mentioned in it, except possibly this part:

 

With [traditional] separation some developers pre-define a chunk of “reserved” fields in each table of the Data file before initial deployment. ... The reason for doing this is to maintain internal id parity between fields in the your copy of the Data file and the client’s copy of the Data file. If that sounds like gibberish, bear in mind that FileMaker assigns an internal ID to every object in the database, including fields, which is why you can typically rename a field with impunity. And the reason you might care about this is that, at some point, unbeknownst to you, the client may choose to add some additional fields of their own to the Data file ...

 

If the above describes your situation - i.e. [a] your solution contains multiple files and both developer and client may independently add/delete fields in any one file - then indeed you have a reason for concern, even if you have upgraded to v.12.

Other than that I cannot think of a practical scenario where ID disparity would become an issue. That doesn't mean it cannot exist, only that I cannot think of one. In the normal course of things, Filemaker manages its internal IDs on its own without needing assistance from the developer. The exception to this rule is (or was) the import mapping bug, where Filemaker dropped the ball and did crazy things.

[Charity]

I read everywhere not to let them into field definitions.  Well, I can not see a good reason then to add a bunch of extra fields since we will be in 13 by the time I get this silly thing finished and at this point I cannot think of any reason to let anybody in field definitions particularly me.

 

that was a joke. 

 

Thank you very much for clarifying.  I feel much better and my file will be much thinner.

[comment]

I feel I need to clarify one point still further. For the purposes of the current discussion, it doesn't matter if you let them into field definitions. You could let every Tom, Dick and Harry add and delete fields to their heart's content and they still couldn't create a field id disparity. It's when you switch files between two copies of the solution that the thing happens. Because then the new UI file calls for field #1234 from the Data file and instead of getting the expected Customers::SpouseName it retrieves data from Refunds::ReasonForReturn.

[Charity]

Okay it is your pest again but honestly one more question.  So we will use separation so if I make changes in my UI and it connects to my data and I change fields in my Data in that part such as script which uses calc based upon data in those new fields and THEN make the changes in the served version of Data but in wrong order and THEN put my UI there to use ... will it mishap as you explain because my UI looks to the field ID number so it first asks where is my other partner file DATA then it marries all the pieces together using those internal ids?

 

If this is true then we will want to 1. be careful about the field order if we update fields which is simple discipline or 2. migrate the data using imports which now work but is slow and server must come down or 3. add all the extra fields so file runs slow every time it runs?  A solid method of tracking the field order should be okay for us.

 

Thank you.  If I am off on this then please set me square.  Great information.

[comment]

make the changes in the served version of Data but in wrong order and THEN put my UI there to use ... will it mishap

That's a good question. Short answer: Why don't you try it and see for yourself? Long answer: Yes.

 

Note that having those extra fields won't help you if you re-purpose them in the wrong order.