how successful is separation model?

[Stuart Taylor]

Fenton you have rocked my world.

I'm grad to say i was missing something.

Thank you! :

[Stuart Taylor]

Hmmmm

So its probably worth having 3 Files.

1. Data, Server Side

2. Solution, Client Side (for GUI updates)

3. Print Layouts, Client Side (for print layout updates and to stop having to update print layouts when the GUI is modified)

[igotit]

Hi Fenton,

Your separation download got me thinking. I know I’m making this sound simpler than it is. But isn’t this a lot like the old days with FMP 5&6 when separate files were the norm, using file references to join other files etc.

I know back then each file generally had both interface and data but they typically related back to one main file. Am I off track here or is that pretty much the case with the separation model?

If the answer to that is yes, then my question is, if i'm familiar with how it was done in FMP 5&6 will that help me better understand and be able to more easily build the separation model?

Milo

Edited May 16, 2007 by Guest

[Fenton]

No, it's not at all like FileMaker 6 structure.* In 6 each file contained its own data, shown on its own layouts. Any data from another file had to be shown in portals (or as a single instance). To view another file's data in a list, and work with it as local, you had to go to the other file.

If you look at the file I uploaded you'll see that the data "shows" on regular layouts as regular fields in BOTH files. It is editable, searchable, etc.. as if it was local to that file. But there is NO table at all in that Interface file (there could be, and likely would be, for globals/constants at least, but for this simplest example there is not).

When you go to the Relationship Graph in the Interface file, you will create and work with table occurrences of the DATA file just as if you were in the DATA file itself. In my example, a DATA file TO is the only TO on the graph in the Interface file.

Same with Scripts. About the only time you really notice that you're in the Interface file is when you go into Define Database, then look at Tables or Fields. Sometimes you do this by mistake; "Duh, they're in the other file!". It's that transparent.

Table occurrences are based on File References. These can be from any FileMaker file available in your environment. You can create a table occurrence then a layout based on it. The data is then available as local. It is NOT related data.

It's a difficult concept when you're used to thinking in FileMaker 6 paradigm. But it's so much more convenient. It makes complex inter-table scripting so much easier.

*Actually, perhaps I've gone off on a rant (it happens :-). You're right, it is similar to 6, the concept that you can open (hidden) and interact with another file by creating a relationship to it (which created a File Reference, but it was not visible). I'm sure that was quite a (welcome) surprise to the FileMaker 2 folks. But it's way beyond that.

[igotit]

Hey Fenton,

I for one like your ranting because I learn a lot from listening to you. Sometimes how you get to something is as important as the final destination.

One more thought. When FMP went to 7 some in the FileMaker community were not happy because FileMaker changed the way things were done. I wasn’t bothered by it that much. But I didn’t have a large server driven solution to convert either.

One big feature of the separation model that is important at our company is the ability to make interface changes that can be swapped out as new versions are developed. The separation model is standard operating procedure for many databases but seemingly never really intended in FileMaker.

That said maybe you could get your crystal ball out with this question. [color:blue]Do you think that FileMaker will make future versions so data separation is no longer supported?

Milo

[Fenton]

From what I can see, it is supported by the basic relational structure of FileMaker 7+ itself. It's just taking advantage of it. I'm not much up on the technical stuff, but cannot imagine they'd ever need to go back.

[The Shadow]

Also, the same concepts used by the separation model are how a multi-file FM6 solution is automatically converted to a FM7 by the upgrade, with data tables in other files with TO's for them in the converted file.

That seems to me like a pretty fundamental need for these abilities on their part, so I wouldn't be concerned about it going away.

On the other hand, if enough people use it, I would expect them to improve support for it.

[igotit]

Hello again,

I'm sure you guys are getting tired of my dumb questions, but i'm trying to get this believe me.

Yesterday I was re-re-re-re-reading this thread and clicking on each of the links to read that information as well. I'm always amazed at the clarity that comes from do so, especially on a thread like this.

I came across this paragraph and I'm sorry I didn't take down the name of who posted it, but it said,

[color:red]By splitting the database you can have different front ends address the same data. One front end is (eg) for data entry and the other for analysis; thus obviating the need for security privileges within the database.

I'm not sure I understand what this means and thought you guys could expand on it for me. Are they talking about different layouts within the same interface file. Like one for Admin, one for accounting and one for sales etc? It seems important and want to make sure I understand it.

Thanks again guys,

Milo

[igotit]

Hi Again,

One other thing that would really help is a list/chart of what should go where in a SM. One place I read DO NOT put calculations in the Data file. Another says HEY THATS FINE to have calculations there! What do you guys who use and design them say? What should go in the INTERFACE file and what should go into the DATA file?

Thanks again for your help,

Milo

Edited May 24, 2007 by Guest

[Fenton]

I think there are really 2 types of separation model, the normal 2-file structure, Interface and Data, and a more extreme 3-file structure: just basic Data, calculations file, and Interface file. I've never used a 3-file method, and know little about it. I think you'd be better off learning the 2-file structure first. If you have access to the Data file, for small things like adding a field, modifying calculations, etc., then a 3-file is not much needed. But, as I said, I don't know much :-]

Other than that I think you'll find it pretty obvious what to put in each file. There are no data tables in the Interface file, other than perhaps a table for constants, and whatever you need for the interface (some people have pretty elaborate schemes for this). So obviously data fields, and whatever relationships they need for lookups, auto-enter calculations, calculations, all goes in the Data file; things required for field definitions.

Things like custom value lists should also go in the Data file, so changes are not wiped out by a new Interface upgrade. Personally I avoid custom VL's unless needed. The value lists also need to be defined in the Interface file, but they can, for custom VL's, "Use value list from another file", i.e., the Data file; otherwise they can "Use values from a field" (with whatever relationship needed; all in the Interface file).

Almost all scripts go in the Interface file. The only time I've found that you must put one in the Data file is if it needs to run with Full Access privileges and act on data. Because that option does not cross files. This is a very rare need. I've needed it for Copy All Records, when users did not have Export privileges. But with FileMaker 8.5 you can, in most cases, use the List() function instead.

There was discussion, on another email list, about what to do about Accounts and Passwords with the SM. I suggested that accounts could be a table in the Data file, with account creation/deletion all handled by scripts. A new Interface file would need to run a script to synchronize its accounts when updating.

Passwords are a bit of a problem. If they are stored as data, then it's a bit of a security risk. Though I don't really see that it's much more of a risk than any other data. The Full Access account is not scriptable, and should not be stored in the data. Also, you can use a Custom Function to encrypt/decrypt PW's. Then it is easy to populate the Interface file with accounts & pw's.

Otherwise you'd pretty much have to delete their accounts, and force everyone to create a new password the 1st time they logged in. That's not really such a big deal, as they'd only have to do it once after an upgrade. Account scripting is very fast, so this is not a laborious process.

I have some example files posted, in the Sample Files section, for scripting accounts, but not for the SM model; I'll have to mess with this some more :-]

[Genx]

DO NOT put calculations in the Data file

Out of curiosity, where did you hear that?

[igotit]

Thanks Fenton, that was the information I needed. I just started the SM on one of our larger single file solutions and so far so good. I'm just trying not to do something really stupid to derail the project. The only problem so far is some of the calcs do not work correctly. One of them takes a product ID with 6 different prices for 6 different weights and computes the correct price base on the total weight of the invoice. It does not work for whatever reason I've not been able to figure out as yet. Works perfectly in single file but will not even show a value from the data file. Everything else seems to work seemlessly.

Hey Genx, I did a Google search for "FileMaker Pro Separation Model" and it was in one of those that I read about not having calcs in the data file. I'm sure I could run it down for you if you'd like.

As always, thanks so much for your help.

Milo

Edited May 25, 2007 by Guest

[Genx]

Hi Milo,

If you could track that article down, that would be great. I am only curious for the authors reasoning behind that statement.

Cheers.

[Vaughan]

Perhaps the "no calcs in data file" comes from the typical 3-tiered design used by big-iron development systems: data layer, business layer, presentation layer. Calcs would typically go into the business layer, because the calcs are generaly required to satisfy business rules (a need to display the person's name in a particular way, etc).

One use for the separation layer I'm using right now I don;t think has been mentioned yet: repurposing existing databases.

The company I'm working for has an existing database of film information -- definitive and accurate. Other parts of the organisation are duplicating various subsets of this information in a multitude of other smaller systems. I'm using "separation" techniques to allow access to the main database information from their smaller systems, without actually letting people into the main system directly.

So far it's working out really well.

[Genx]

I'm using "separation" techniques to allow access to the main database information from their smaller systems

Neat Idea.

So to sum up the three way seperation Model.

Data -- Pure Data Storage Fields

Business Logic -- Summaries, Calcs, etc.

Interface -- Just interface, maybe session globals.

Here's my question with the Business Logic / Data thing though right:

DATA

id

firstName

lastName

BUSINESS LOGIC

id

calc

... Either you would need an individual record in the Business Logic tables for each record in the Data Table for which you wanted to calculate something...

Or, you would have to store the active record ID in a global field somewhere... which would probably make reporting difficult. Anyway, could someone please explain to me how that portion actually works because i have the feeling I may be missing something obvious in my current state of mind.

Cheers.

[igotit]

Hi Guys,

You guys keep talking i'm learning a bunch of good information!

Genx,

The link I clicked on the “FileMaker Pro Separation Model” Google search was “Only Click Here If You Can Read!” Click on the following link.

http://filemakertoday.com/com/showthread.php?t=8403&page=2

The conversation on the Separation Model starts with Post#14 by HarryGlos, Ender and Comment and ends with Post number 21 by Comment!

The following quote is from Post#16 4th paragraph.

Lets use a contacts file as an example. One of the first things you want to do is display a first and last name in a unique fashion. Data display issues are the main issue you will have to deal with using a separation model. The goal is don’t dirty your data file, with calculations, scripts etc. You want to keep all functionality in your interface file.

Milo

Edited May 25, 2007 by Guest

[wingwalker]

I've been following this thread with great interest and started converting a current file into the SM discussed here. I used the cloning method that Ender provided a link for and have a question.

I made 2 clones of the file one for INTERFACE and one for DATA and did all of the things mentioned on the link and everything is working fine. My question is what about all of the layouts, scripts, custom functions, etc in the DATA file left over from the cloning. Should I just go through and delete them? I know there are a few scripts I need to keep in the DATA file, but should I delete the rest of the stuff?

Thanks,

James

Edited May 29, 2007 by Guest

[Fenton]

Normally you'll have very few scripts in the data file, so you should probably delete almost all of them. Same for layouts, though you'll likely want at least one layout per table, for troubleshooting if nothing else. I personally dislike troubleshooting reams of data on user interface layouts; too much junk in the way, no ability to add/remove fields on-the-fly.

As far as Custom Functions, keep any that are used in field definition calculations (most common use), but delete the others (if they're in the Interface file; don't lose them).

[Leather Knight]

The separation model looks like it will work for my distrubited app, however, I have over 20 forms and tables in one .usr file with reports and so on. It is a desktop app and so networking is not an issue. However, with so many fields how am I to implement this separation model in my app? I do not get how to actually make it.

[HarryGlos]

Hey Kevin,

I don't get over to this forum very often, but this is a great thread by folks who know what they're talking about. Were I you I'd start with the post by Ender http://www.fmforums.com/forum/showpost.php?post/12... Now Ender doesn’t have many good ideas so you need to really jump on the ones he has! : Follow it and you'll be going the right direction. If you have FMP 8.5 Advanced that will help you along. But "Follow the Bouncing Ball" so to speak and you'll do fine.

You'll have some question along the way, many of which have already been covered on this thread. Just shoot em on over and the good folk here will have you well in hand.

The question was asked in an earlier post if SM is easier in 8.5 than it was in 7 and the answer is defiantly!

Live Long & Prosper

Harry

Edited June 7, 2007 by Guest

[mr_vodka]

Hey Harry,

Its been a while. So you coming over from Cafe to stir up trouble here as well now? :

[Genx]

Harry are you sure you posted the right link?

[HarryGlos]

Hey their Mr. Vodka and Genx,

Mr. Vodka said: It’s been a while. So you coming over from Cafe to stir up trouble here as well now?

I guess I’ll need to spend more time here if you guys keep coming up with threads like this. The FileMaker community has little clue of the importance of what we call the “Separation Model”! I call it the “Common Logic” model myself! There is simply no reason to build otherwise. I know, I know, I here it all the time, “FileMaker is not a SQL system” and was never built for SM. But if God didn’t intent for us to use the Separation Model he would not of given us FMP 9! Have you seen that beauty? Whoa!

Genx, you are so right… Somehow I entered the wrong link and here is the right one… http://www.fmforums.com/forum/showpost.php?post/123239/

It’s Ender I tell you, the boy just makes you want to do wild and crazy stuff. : Hey, in case anyone has missed it “This really is a great thread and I can’t tell you how happy I am I stumbled upon it.

Oh, as far as me being over here on the good ol FM Forum, I should tell you that I’m on my 4th series of shots and have had my picture taken for the visa. After that its just the blood test and I can be a full member here! The strip search not withstanding. Yee-Ha, Yee-Ha! : :

Live Long & Prosper

Harry

[Genx]

But if God didn’t intent for us to use the Separation Model he would not of given us FMP 9! Have you seen that beauty? Whoa!

If you have, I am extremely, extremely, extremely, jealous... But can't wait lol.

Anyway, Stick around eh? :

[Leather Knight]

Ok, I read his thread. I can understand that better now. Having literally over a hundred fields and calcs, this will be a task for sure. I appreciate this guidance, I really need to be able to update my db when I am done. I will let you know what happens...

[HarryGlos]

Hey Kevin,

You have no problem in the world if I'm reading you correctly... You have 8 Advanced so your life will be much easier. The hundreds of fields and calcs are really of little consequence because you don't have to do anything with them. Keep em all in the Data file, strip them out (not the relationships) of the Interface file, point the relationships to the data file (remembering to maintain the exact relationship names) and as they say "Botta-Bing-Botta-Boom"!

Id like to say it's real difficult and you better send it over to me for conversion and I'll only charge you $5,000.00 and save you the grief. Man, just having to read more than a few lines from my good friend Ender (He doesn't even know who I am) : is punishment enough. Hey... It ain't a big deal. Use to be but not now. As Fenton and others have said. It's so seamless you go to the interface file forgetting the fields and calcs aren't there anymore.

Look, if it doesn't work you can always blame Ender... We all do! :

[Genx]

Lol, whats up with you and Ender eh?

It's so seamless you go to the interface file forgetting the fields and calcs aren't there anymore.

Entirely true. At first it was a bit annoying, but now the only place i logically go is to my data file to define stuff.

[Leather Knight]

Thanks Harry, but although the data file is receiving the data from the interface file fine, if I was to make new fields in the interface file and send it to the customer as an updated improvement to the database, how would the data file know how to recognize the new fields added and the data generated from them?

Say I have the 2 files, data and interface. Now I forgot to add a phone number to the whole scheme. I distribute the database. Now I have a number of people with this bad database. I add the phone number field in the interface and send it to the customers. Boom, they have an updated database, however the data file they have does not have a place for the new addition of the phone number.

Edited June 8, 2007 by Guest

[mr_vodka]

In the case you just specified, you would have to send a new data file as well since you just ADDED a new field for additional Data. Then to display it, you would have to send a new interface file as well.

[HarryGlos]

Yes Kevin, It's what he said… After a few belts for clarity John comes through with both verve and panache. Ok so maybe just the verve part for the moment, but I know John’s been working on the panache part for years… :

Genx Said: Lol, whats up with you and Ender eh?

Mike and I go back a long way over on the Café. He is one of those people who can give it and take it with a great sense of humor and incredibly knowledgeable about FileMaker. Please say nothing to him about such accolades. You can’t live with him now and this would just throw the boy right over the fence. Plus he thinks “Accolades” is a topping for an ice-cream Sunday! :

Live Long & Prosper,

Harry

Edited June 8, 2007 by Guest

[Søren Dyhr]

In the case you just specified, you would have to send a new data file as well since you just ADDED a new field for additional Data. Then to display it, you would have to send a new interface file as well

Not nesserary true, this can be taken to a higher abstraction level, like Raz in this thread and Johnathan Stark in these two links:

http://www.jonathanstark.com/downloads/Ginko.fp7.zip

http://www.jonathanstark.com/downloads/uddm_examples.zip

Now this is admitted, way to deep to my way of thinking, but the way I add would add each new field to a solution, could be utilizing this:

http://www.filemakermagazine.com/videos/data-tagging-classification-vs-organization.html

...this means that a field is a foreignID, label and data.

--sd

[Leather Knight]

You all are great. That is one reason I have stayed with FM, great sense of humor and superb assistance from some really great minds. I guess I have decided to rethink my strategy. I will develop my app and if there are any bugs I can update the interface and all is good. If there are any additions, it will be in my next release, while making a list of new features. This will help keep ,my mind from frizzing out. Again, thanks so much for getting me on the right track. :

[HarryGlos]

Hey Guys,

I know this may be offensive to some of the purists in the group. But for those who are less, shall we say, squeamish and more, shall we say, enlightened, I would recommend ...Unused Fields… "Don’t look Molly he said the “U” word"! Yes I did… A few unused fields sprinkled here and there, yep I got that from the food forum, can solve a lot of future headaches!

Remember we’re talking about Separation Model here… Just think about it boys and girls… :

Look, if you cut us do we not bleed? If you kick us do we not say ouch? If you show us a hot chick in a bikini do we not say hubba-hubba? If you say Microsoft Access do we not suffer in the region of the gas-tonal intestinal tract? :

Deploy your solution with unused fields (Think Data File) and save yourself a lot of grief later! Just think about it for a while and we’ll talk more later!

Hey Kevin, Neat Web Site!

Live Long & Prosper,

Harry

Edited June 9, 2007 by Guest

[Leather Knight]

Amazing Harry... Thanks for the comment : I was just about to post my idea of having say, 5 fields for future access as needed... : And then you said it first....you beat me to the punch. But it will solve future headaches, you are right. I was thinking 5 numeric, 5 data, and so on as a backup. Way to go.

[Fenton]

And remember, that you can have a Globals table in the Interface file, for temporary data. Though this is much ameliorated by the script Variables in FileMaker 8. Still, sometimes you just gotta use a global, for relationships, etc..