Upgrade vs. re-write

[FMNan]

I'm curious as to how many of you have chosen to upgrade existing databases, and how many chose to re-write them in FM 7/8? Any advice on deciding which direction to go? (I'm looking at an FM 4 database going to 8, multi-platform).

Thanks to all

[SteveB]

I upgraded a 35 file, 2500 script Win platform solution, as I thought rewritting was way too long. I know there has been a lot of discussion about it, but I can't see a total rewrite as being worth it.

Steve

[gdurniak]

we started to re-write a 32 file solution from FM4 to 8, and quickly saw that it would just take too long

there was still a lot of "tweaking", e.g. adding "commit record" and "select window" to many scripts, but much easier overall

if your solution works, and is fairly well designed, conversion makes sense

greg

[FMNan]

Thanks Greg & Steve for your replies.

As far as "if your solution works, and is fairly well designed, conversion makes sense" -- I inherited this FM4 database which was written long ago by an employee who read a book. There are broken relationships, missing files, and the Users/Groups is all over the place. Many scripts are "dead" - meaning they were ideas once that either stopped being used (or never were).

Unfortunately, the powers that be all believe that "upgrading" will be the quickest & most economical way to go. I am explaining that it is more than a simple upgrade, and that when the conversion is done, it will be the same database only on a newer version (unless I start re-writing sections as I convert).

One question I would like to ask is how long did it take, from start to finish, to convert your old databases & have the new ones running & "live"? And how long did the "final tweaking" stage last?

Thanks again to all.

[Razumovsky]

Said with big smile:

"Sure Boss, I can adjust that feature exactly how you want no problem. It will take me X hours. With a well thought out design, it would take me X/50 hours. My children thank you in advance for their college education."

And do make sure you are charging by the hour.

-Raz

[SteveB]

The problem with answering your question is that you'll want to take advantage of some of the new features, such as the Tab widget, toolips, custom functions, writing to PDFs, script variables, etc. FM8 is so rich and things are so much easier to do that some change is unavoidable. Someone who has done lots of conversions, and understood your app *** might *** be able to predict how long. Most of us have only done one app (like me).

Steve

[Fenton]

Yes, this is a difficult question to answer, for several reasons. One of which is that it depends on the database. I've converted databases that I've written earlier, as well as a few templates which were written by professionals, and not had too much trouble. Though, in every case, the final result was not as clean as recent ones I've written from scratch.

On the other hand, I've seen some real disastrous files, which could only be "converted" by basically throwing away more than half of the structure, and re-writing at least half of the remainder. By which time it's not so much a "conversion" as a "remodel."

The very worst scenario is a large complex old solution that is badly written. You know the ones I mean. Lots and lots of layouts, many of which look almost the same, are probably duplicates, all of which look like crap. Lots and lots of scripts, most visible. Scripts that go back and forth, but don't really seem to get much done. Lots of copy/paste. Lots of single-step scripts, like "New Record", "Find".

The big problem is that it is often harder to figure out what is NOT needed that it is to create what IS needed. It takes a lot of time. Because this stuff is not needed, no one is exactly sure what it does or who would use it. Often no one every did. Beginners like to create things, but are afraid to delete them; which is like always buying stuff at yard sales, but never cleaning your house.

One approach you can take is to convert the files, but basically gut them. Keep a copy of the converted files pretty much as is, but work on another copy, using a heavy hand. Keep the basic fields and layouts (only the basics), and the good relationships. Analyze what scripts are doing. Then combine, rewrite, modularize, whatever to get what you really want.

There are 2 tools you must have. FileMaker Pro Advanced, for copy/pasting tables, fields, scripts, etc., script debugger, etc.. And something like Inspector, to quickly find out just how useless some of that junk is. If they are not willing to pay for these essential tools, then it is pretty much, if not a hopeless case, certainly an unpleasant one.

You will need a fair amount of time, and quite a bit of feedback from others; unless you know the whole business yourself, in which case carry on.

You can tell the boss you're converting the files. But what you do inside them is pretty much up to you. It would be far far better to NOT try and use this project for live data, until you're almost completely finished at least. But you should have people testing. If it turns out you took out something they needed, no big deal, copy/paste it back in.

On the bright side, you will be upgrading to the latest version of FileMaker. As Steve says, you can use tools like the new Tabs to consolidate layouts. The new relational model eliminates most needs for extra calculations to "tunnel" data. There are awesome new features like Go To Related Records, for the whole found set.

Rather than looking at conversion as a big problem, which just needs to be gotten over with as quickly as possible, with as little change as possible, they should be looking at it as a chance to get things done better, more efficiently, as well as to expand in ways you couldn't easily before. If their business expects to grow into the future that is what they must do. You don't save money by using crappy tools and wasting people's efforts. Believe me, I see this far too often.

[gdurniak]

(cont'd) 32 file solution from FM4 to 8 took about 40 hours, fixing scripts, file references, and re-setting the accounts & groups

we used MetadataMagic to clean up the file references in FM4, and to print a "design" report

PS

FileMaker really goofed by not including at least a basic file reference fixer in the product

[FMNan]

RAZ - LOL!

I love starting my day off with laughter, makes for a good day!!

Thank you

[FMNan]

Fenton - thank you very much. Hope you don't mind, but I've printed your reply out & will use some of your verbage in my discussions today. FM Pro Advanced I will have (is on it's way), Inspector you mentioned... is that similar to Analyzer that I used with FM 5/6? Running on a database, it will list broken or missing items & check basic structure -- reporting what are potential problems? Is Inspector a separate FM product, or 3rd party?

Thanks again

[FMNan]

Greg - thanks for your history on your conversion. (I know all DB's are not created equal, so timing will always be different -- really wanted some ballpark times to look at).

One question - MetadataMagic -- I'm ok with the file reference fixes, but you caught my attention with "design report." What exactly will this do/produce? Perhaps a diagram/schema of the database?

Does anyone know of any other products/utilites for producing relational diagrams of an FM database? (it would be nice if FM would printout a diagram, or at least a listing of the relationships -- or does this exist in 8 and I am missing it?)

Thanks again

[mav]

I can't agree more for you to get a copy of Inspector to analise your file.

The demo download comes with a licence that expires after a short time period. This demo time should give you enough time to see if the software is worth it's price.

It does take a while to get to know Inspector, but it's time well spent, as it will tell you things you need to know about your files.

I would suggest building a small file, and do some simple test runs, seeing what Inspector can find and tell you, rather than wasting hours on a big file while you are still learing.

Matt over at FileMaker Magazine did a great interview with the author, and his free quicktime file is actually a great place to start, so you will want to watch, as you do your first runs.

Pesonally, I am in the middle of taking a pre FM7 30 file database and reducing it to 12 files, and the whole solution has 90,000 elements.

Inspector not only shows where the real problems are, but also allows you to see if an element, has dependancies or references, It also has a search function which can show you where a script step has been disabled, or an auto entered calcuation has been disabled when you have imported a Table.

If you haven't already, have a read of the Migration PDFs at FIleMaker's site also.

Happy converting . . .

[Fenton]

I believe that Inspector and Analyzer were written by the same person. The two are very similar; Inspector is newer. I prefer Inspector, because the last time I tried Analyzer, it could not sort the field names in the viewing portal. On large files this was quite awkward. If they've fixed it feel free to set me straight.

Metadata Magic is also very useful. It is somewhat expensive, but will save many hours. You must fix your File References in any case, either before or after conversion. Otherwise your file will always be slower than it should be. How much time it will take to fix them manually depends on several factors. Old files developed by non-professional developers are going to have a mess of incorrect file references, to machines that died years ago.

It is not terribly difficult, though tedious, to retarget the table occurrences on the Relationship Graph to the correct one. But every single external call in scripts and external or related value lists also must be retargetted. It is one reason why restarting from scratch is popular -] If you decide to drastically rebuild the files, combining files as tables in amalgamated module files, then you will be able to discard much of this dreck, with the old files.

The optimal plan when converting an older solution is very difficult to decide. I'm sure most of us look forward to the day when we never see another pre-7 file. It won't be soon I'm afraid. It is also true that it is far more difficult for beginning developers, especially their first time doing it.

You really should read about it. Most of the problems that you'll face that are due to the conversion itself are well-documented. Illyse Kazar's article on record-locking is mandantory (in the big migration_foundations.pdf, 12.4 MB). As is much about the new security structure (Accounts & Privileges). Otherwise you'll have trouble, sooner rather than later.

There are also descriptions of the different plans for migration: Convert and just try to fix (fast, cheap, and often out of control -), Convert and renovate (can be good, if the original files were, saves some time by reusing much of the structure, but is often painfully confusing), and Build from scratch (ideal really, but not always an option).

I think there is a 4th method, a drastic form of renovate, which I think of as "Gut and rebuild", which I described earlier. Convert, but just keep original conversion for reference of what was, and for parts. Copy it, rename the original (with Advanced), then drastically alter the new one, as modules, with far fewer files. You may even decide that the existing relational model is inadequate; now's the time to redo it. You can copy/paste entire tables. Remove lots of junk, modernize remaining layouts, use Tabs where practical, completely rethink scripts and navigation. Keep users on the pre-7/8 files until you're satisfied that you have a working solution. You will be working locally. You will not be doing anything else. This is not a part-time job, except for brief periods of R&R.