When are seperate files advantagous?

[JerrySalem]

I think, like most of us, I finally have my brain (partway) around the new structure. I am now experimenting with how to convert my existing solution to FMP7. But I am alittle scared to make a naive mistake(s) that I will regret later.

When is it appropriate to create a solution that is made up of multiple files, each containing multiple tables?

Take a common structure we all are familiar with. A database containing a table of people, invoices, lineitems, and items. In this hypothetical example there are also a couple of lookup tables for info that isn't transactional. Is there a situation you would want this in multiple files?

I hope someone can shed some light on this for me.

TIA

Jerry

Version: v6.x

Platform: Mac OS X Jaguar

[The Shadow]

I would guess it would almost always make sense to have simple lookup tables in another file, that way you're not backing them up with your solution, etc. This might be a non-issue if they're small tables.

Many have mentioned having a separate "interface file" that contains the majority of the layouts and scripts, to make updating a solution easier. There was an excellent paper floating around about this, I regret I don't have the link on-hand.

Also, there may be a speed advantage with Server 7 for multiple files, as the potential would exist for multiple files to be read/written in parallel. I have seen other posts mentioning better performance with FM7 when they combined tables into one file. I suspect its too early to call this issue one way or the other yet.

Personally, I was thinking of structuring the data "logically", and choosing a break-down into files based on that. For example, if I had a people table with names, and a separate table with addresses of the people (so a person could have multiple addresses), I'd combine them into one file. I don't think I'd do so with the people and lineitems file.

In your example, I would consider combining lineitems and invoices as those two togather represent the concept of an "invoice", the lineitem is only split out to a second table so that the database is in "relational" form. Of course, if the performance wasn't acceptable, I'd take it back out.

[Guest]

Our non-FileMaker business information system (ERP) runs on an Oracle DB. Everything is (essentially) in one file that contains 1200+ related tables.

I don't know enough about 7.0 yet to be able to intelligently answer this question but had FileMaker done this one file many table thing since version 1.0 and was just now giving people the option with v7 to be able to connect file to file I have the feeling that nearly everybody would be saying "Why in the world would anyone want to do that!"

[Kurt Knippel]

As I see it the advantages of seperate files is Modularity. With seperate files you can updated, close, open, change, remove those files (and thier tables) without impacting the other parts of the solution. You can also do load balancing (spreading the load across multiple servers) with multiple files.

I am basically looking at the idea of some logical groupings of related data going into the same file, with a seperate central file to hold all of the interface and business logic.

[dkemme]

In our medical office we scan documents into our FM solution. Currently almost 30,000 images for a total size of 7.5Gb. Since pre-7 FMP was limited to 2 Gb the images were pasted as references with a copy of each image placed on each client to cut down on network traffic. I am currently rewriting the solution for FM7 and wonder what is best for all these images. Should I place them in a table within the main solution or a separate database file or just continue to paste as reference placing a copy on each machine?

TIA

DJK

[Singlequanta]

Well... it's always **very** handy to create external files from your solution when you have a project that will be installed on multiple machines where there are slight configuration changes necessary on a per station basis, but there are a bunch of general configuration options which are the same. As an example, I wrote a system for managing an internet cafe, which is installed on every computer. The general configuration on each station contains things such as which applications the users can run, company info etc. However there are a few machine specific configurations. After installing the solution on one workstation and setting it up just the way I wanted, it was a matter of copying the external "config" file to every machine on the network, which needless to say was a lot faster than re-configuring every station.

Hope this helps.

Steve

[dkemme]

It is a bit more to update the files on twenty machines every time the images change though, which occurs everytime the file initially serving the images gets too large. With FM7 breaking the file size limit, I am tempted to put the images back in. But I am concerned about backup times, network traffic and server usage if it is in the same file/machine as the solution. Does anyone have any hints to the speed of Server 7 yet?

[Hurican]

Something that I might suggest would be to put these images in a dedicated table within the same file. FileMaker 7 has about 8 Terabytes of storage per file. So far as I know, no one has come forward and said that they tested those limits, but I believe that you have a way to go before you hit performance dogging.

With images, backup times signifigantly increase. Network traffic and server usages are affected, but not as badly.

Hope that gives you some additional info...

[bruceR]

Hurican said:

Something that I might suggest would be to put these images in a dedicated table within the same file. FileMaker 7 has about 8 Terabytes of storage per file. So far as I know, no one has come forward and said that they tested those limits, but I believe that you have a way to go before you hit performance dogging.

I talked to another developer with film industry connections, and they had given some thought to storing full length films, but 8TB isn't enough for them. But it will certainly be more than enough for most applications.

[dkemme]

If backup times increase signficantly, why not put the images in a separate file and back them up with a different strategy than the main file with the other tables?

[falkaholic]

That reminds me, with automatic backup systems, only files that have been touched (modified) since the last backup, get backed up. So if you have a 5 TB file, and 1 byte of data to it, you backup software will back up the whole 5 TB file.

So if you throwing around that much data on a regular basiss. Try and make new file every X GB or so.

dkemme said:

If backup times increase signficantly, why not put the images in a separate file and back them up with a different strategy than the main file with the other tables?

[jasonwood]

One reason to keep separate files would be if there is one table you frequently update off-site and you have to perform imports to get the data in the updated version. If it's all one file, you have to import EVERYTHING, which will take time.

[George TOUBALIS]

HI there...! (sorry for my bad English)

I have G4 450 with 1Gb Ram and A Raid 0 Disk (5 Scsi Disks 10.000prm)

I have a very large X icons collection (About 12.000)

I developed (!) In Fm 7 a file with 2 fields

1) icon name----text

2) Icon --------container

In the beginning (when I import (not a referense) the Folder with my Icons) the import process is fair good -about 15 icons/sec -

after 2000 icons the performance was 5 icons/sec

after 6000 icons the performance was 1 icon/sec

after 7000 icons the performance was 1 icon/2sec

after 7500 icons the performance was 1 icon/10sec ...wow...!

Then I try several things like to put 1000icons/folder... nothing happened

The icons size is 25k-90k (tiff with an alfa channel)

Then I try With a FM6 file.... done in 3 hours... This file (600Mb) I try then to open with Fm7 ....Conversion-1hour...Convversion 10hours... Conversion 2 days... Come on give me a brake... I log out

Well ...I think ... I can't collect 50.000 images (6Gb) In one file... before 2007

I wonder if FM pro 7 is ready to cover not the needs of a professional but the needs of a student to archive his party photos....:

8 TB per file ha..ha...ha!

George

[Kurt Knippel]

Hmm, your results are very interesting.

I have a photo album solution that is structured similiarly to your icon collection, although I have many more fields. I have also found that the more photos that are stored the slower the import gets.

I have not noticed any performance differences when actually using the DB, only on the import.

I have found that if I run it on a standalone system (the server that hosts the files) in a normal copy of Filemaker Pro, it runs significiently faster. Not surprising, but when I have alot of photos this really speeds up the import of more photos. Once imported, I open up Filemaker Server again and put the DB to use.

Now, I have NOT tested this system in Filemaker 7 yet, so I cannot comment on its performance on these tasks relative to Filemaker 6.