Large Solution Syncing

[NRT.Rob]

Hi All,

So I recently got an older powerbook so I can do my filemaker 8 developement at home, late at night.

I have a comprehensive relational database suite consisting of about 35 files in two folders (a main and a "support databases" folder).

My issue is, I go home, I do some changes on a copy of the databases. then I come in at some point and I want to deploy my updated solution, which I may have been working on for a month or a day, depending on the additions I have made. In the meantime, 5 users in the office have been working on the real databases. So, I need to sync the two.

Ideally, I would clone all of my new DBs, import all the data from the existing DBs into the new clones, and do any manual clean up that OF COURSE I clearly noted for myself to go over in the final copy of the new, ready to be deployed databases.

The problem is.. this is about 35 files. Doing 35 clones and 35 imports is pretty ridiculous, at least manually.

Is it possible to applescript the cloning and import process? If so.. I imagine I am not the first person to do this.. are there any examples of this already out there?

Or, does anyone have a better idea about how to do this continual "redeployment"?

Thanks in advance.

[David Jondreau]

1) 35 files or 35 tables? FM 7+ allows multiple tables in one file. You only need to clone once. You still need to import 35 times.

2) You could look into a data file and interface file, but that will only be helpful if the development you're doing is mostly on the interface (layout) side of things.

3) You can script the cloning/import process within Filemaker, you don't need Applescript.

4) It would be simpler to either only work on the db when no changes will occur or plan your development so you only need one import every couple months. Since you're working from home at night, I'd recommend the former. You can set up either a ftp download or a remote desktop that allows you to be "locally" working on the actual deployed file.

Make frequent backups and store them in separate folders.

[Ender]

If you choose to script the imports, you may still need to check each import mapping. The problem you have to watch out for is that the import mapping is remembered by creation order, not by name. This is problematic if a field is deleted in the either source or destination, causing the mapping to shift up.

I'd recommend keeping a log of changes as you go. You could record the general or specific changes that are made to each file, and have those date-stamped. This would be useful to know which files need to be replaced and what fields need to be checked or populated. This is also useful if corruption turns up or a major bug is introduced and things need to be cleaned up.

[NRT.Rob]

David, thanks for the response. If I had home internet and a helper in the office I could trust I would probably use the method you describe, which is basically what I do now when I code while I'm at the office. I am programming both interface and data regularly as I expand our database suite's capabilities. and the file merge seems to have only marginal benefits.. the cloning, specifically, and some less duplication in scripts. That's about it, right?

Ender, VERY good point about the mapping. I don't delete fields very often but that is something I would not have thought of, thank you.

I would like to keep the log, but I struggle with it for sure. Anything that interrupts my mojo when I'm coding causes problems, because it sure does suck me in... I just plain forget to write down what I'm doing when I'm "jamming" on it. Any tips, or plugins to ease documentation out there? Ideally I'd love to do a programming session, then hit a button and have something tell me everything that changed. I can document like mad, but it's way easier for me to do it all at once with the list of changed items...

[El_Pablo]

I use google docs to keep a log of my modification. The nice thing about this, is that it is available anywhere.

Edited May 24, 2007 by Guest

[NRT.Rob]

again, I need internet for that though : Also, the big problem I have is having the presence of mind to stop in mid-programming-session and write down what I'm doing.. I'm usually thinking so hard about what I'm doing, I forget to do anything else. I can easily keep a text file on my notebook desktop.. my developement machine is mobile.. but It's actually stopping and writing things down that's hard for me to do midstream.

So to document, I really need an end of session button that will help me see the difference between the current state of my DBs and the original state, so I can then write down all my doc stuff. Or, that's at least the "ideal" solution I'm dreaming of. I suppose it would be more ideal for it to flawlessly write the documentation for me :, but the end of session comparison button would do as well...

[Fenton]

1. There are a few commercial solutions that let you compare files, Inspector, FMDiff (I've not used). There is also a free template, Process DDR, by Jonathon Stark

http://jonathanstark.com/downloads.php

which will let you get basic info, fields, script steps from a set of files. It uses the DDR report (xml), so requires FileMaker Pro Advanced. It can be extended to do more, but that requires some xml/xsl knowledge.

Or

2. Don't delete fields

3. The real problem is not so much documentation, though that is fine and dandy :-] It is that you need an automated import of all 35 tables; or at least the ones where data changes. This is all scriptable, though tedious.

During the imports also need to update the next serial number from any file that needs it. There is a special function and script step to do this GetNextSerialValue()* and Set Next Serial Value.

*In Design functions section. Some people just Show All, Go To Record/Request [Last], get the last and 1. Not officially correct, but often works fine.

4. Value lists can be a problem, if people are editing Custom ones. If there's a lot of this going on, then you should consider a separate data table for these value lists, and "Use values from a field"

It would need an interface for editing the values. Not as cute as a Custom value list, but importable. There are FileMaker functions to get the values from exisiting lists; ValueListNames(), ValueListItems()

Personally I think most people overuse Custom value lists. Using values from a regular field can often do the same job with much less hassle (for the developer anyway :-).

5. AppleScript could be used, at the very end, to rename the old and new files (they need to be closed). There is also a (scary) method to do it with Save a Copy As.

6. Consider splitting the solution into a Data file and an Interface file, using the Separation method. It should be possible to map the Interface file onto an existing Data file, as it is. Then, at some point, either swapping the Data file, or (carefully) stripping it of unnecessary stuff, or just leaving it as is (but that's no fun).

Edited May 24, 2007 by Guest
separation

[NRT.Rob]

Well, some downloads later and I think I have some helpful stuff, thanks Fenton! I do have FMP8 advanced, now, to work on this stuff.

thanks so much for the serial field tip (another bit I would have fogotten) and the value list issue, which is in fact an issue NOW... luckily, It should not be a problem adding the field based method to the database which is actually an improvement I've wanted to make for awhile anyway in a few areas... good stuff.

I can't really easily separate the data and interface here. I'm continually editing both aspects as the databases continue to grow, and i have very limited developement time I would rather dedicate to adding functionality than to going back and basically starting from scratch... This has been a several year project for me so far. It would probably set me back a few months to try and do a separation at this point.

Thanks so much!

[NRT.Rob]

I just wanted to say thanks for all the idea here. I'm fairly proud of myself, I now have a "syncmaster" file with a nice big button, that will automatically clone all of my developement databases, import data from the working copies, and set all serial field values. It's pretty gratifying to watch it all fly when I hit that button!!

Now I'm just working on expunging any custom value lists that have any chance of changing.

Thanks again everyone!!