My take on FM P/D 7: One huge dissappointment

[bankmann]

I've just tried converting my largest database (ca. 11.500 records/330MB), and I'm beginning to see what version 7 is worth. And disappointingly, that doesn't seem to be a lot...

I run FileMaker on the following systems:

1) Mac os X 10.3.5 (Panther) on G4/450MHz DP, 1GB RAM, ATI Rage128 Pro/16MB

2) Windows 2000 Professional with SP4 on Intel Pentium III Coppermine/1GHz, 512MB RAM, onboard GPU (Intel 82810E)

With version 7 I've noticed:

1) File-bloating; converted files (from *.fp5 to *.fp7) seems to grow to an average rate of 40%, in some

some cases even more. A 350MB file bloated up to 520MB on conversion. After compacting it,

it was still 45MB larger than the original (395MB). A 5,3MB file bloated to 8,3MB.

2) Connection to other files now has to be defined (?)

3) Pop-up/drop-down value lists are now extremely slow

You can now actually start typing in a field before the list even begins to show,

and the text typed is not reflected by chosen value (none)

This is not constant, but pervasive (ca 80 - 90%), and tied in with a general sluggishness.

I think this in one case is because a calculation field citing the maximum value of the serial

number field takes too long time to finish, when creating a new record. (Sometimes, even a

progress bar appears.) But pop-up lists are slow to appear even when just working in a record.

4) Once a value has been chosen, it's no longer possible to continue a slide down/up the value list (in the pop-u/drop-down). If the chosen value is incorrect, you need another click; to hold and chose new value doesn't work.

5) Entering a double letter (like 'ff') too fast (meaning 'at ordinary or slower speed') will not highlight a

value beginning with a double letter, just a value beginning with a single letter; while entering

different letters to make a choice works.

6) Entering double letters makes highlight jump past value beginning with two identical letters,

highlighting the next valuebeginning with the same letter.

7) On conversion to new format I found that an overwhelming number of records' serialnumber had

been changed

8) Text-justification-bug (most noticeable on Mac OS X); spreads letters out in a field, making text

difficult to read and edit (The font-smoothing blur-bug was fixed with 7.0v2)

9) The keyboard shortcut-changes made in version 6 have not been corrected (are no longer logical)

10) Window-flickering when entering/editing data (gives an impression of sluggishness)

11) There is an overal increased sluggishness, that, combined with the slowness of dropdowns and

the languid screen/window refresh rate, makes it difficult to work fast and

effectively; if the user dosn't wait for the system to keep up, data will be entered in the wrong

fields. As FileMaker saves data dynamically, offering no undo-function of entered data, this is a

recipe for impending disaster.

12) The visual feedback on saving large files (Save as.../Compact) (330MB) is too poor. It takes several

minutes before a progress bar appears. The system seems to think FileMaker is hanging during

a save operation, and CPU load hits the roof. I've enlarged the file cache to its maximum (256MB),

but without any effect.

13) Import is slower, and older files must be opened and converted first.

14) The Mac OS version still doesn't support scroll-wheel mice.

Frankly, I've conluded that version 7 i impossible to work comfortably with. I have to backpedal to version 6. Does anyone have any good advice on how to do this, since version 6 obviously doesn't support the verion 7 format? (I've added records to the bases after conversion.)

Bankmann

[marianco]

1) I converted my 22,000 record 450 MB database from FM Pro 6 to FM Pro 7. It doesn't have the bloat you are finding. You may have too many indexes or are storing too many calculated fields.

2) Your systems are inadequate for handling FM Pro 7. FM Pro 7 needs a faster machine since it adds many more features which can slow things down. On my Dual 1 Gig G4 PowerMac with 1 Gig RAM, FM Pro 7 works well.

3) The ability to open multiple windows within a database is a highly useful addition to FM Pro's capabilities.

4) Connection to other files do have to be defined - since FM Pro 7 is now fully relational. Relationships are defined differently than in FM Pro 6.

5) Drop-Down lists are less functional in FM Pro 7. It should be improved with updates. This is my primary gripe with FM Pro 7.

6) FM Pro 7 has such a different relational database model to FM Pro 6, that if you are having difficulty in doing the automatic conversion, it is better to redesign your database from scratch.

7) Yes, FM Pro 7 is slower than FM Pro 6. However, this should improve over time as the code becomes more optimized. This is similar to moving from Mac OS 9 to Mac OS X or earlier versions of Windows to Windows XP. That didn

[Fenton]

Much of the sluggishness you're seeing are due to your machine being too slow. I first tried 7 on a 700 MHz eMac. It was OK, but not quite fast enough to be comfortable for development. I got a 1.25 MHz eMac, and now it's quite usable. There are still a few bugs, such as Import on networks (all records only), and the Import speed. I don't know what's going on exactly with the Preview (jagged bitmapped), or printing PDFs (also jagged).

The drop-down lists being slow is a known problem, apparently even worse on XP. Not to mention non-sorting on the 2nd field. Hopefully all of the above will be fixed in the version update (whether or when I have no idea).

The lack of a scroll wheel is just plain annoying. It's been so long I've forgetten it ever worked. I use the Page Down key a lot, where I used to just scroll a little. They can't really blame it on OS X, as every other app has managed to get it working.

But, even with all of that, I love version 7. Once you've referenced a "table occurrence" more than one relationship away you'll never look back. There are so many little things that just work like they should: assign practically any script step to a button, pass parameters to scripts from same, go To related records and specify the layout in one step, specify Finds, Sorts, Imports, Printing, all right there in the step. Mulitples of those steps in one script.

Try something like a Find in an related field, or accessing a remote database and you'll see that 7 is many times faster than 6 at this stuff. It is longer painful to tweak a client's file from home; you can do practically anything (not that you should, but sometimes you must).

Oh, left out multiple tables per file. You can do one of those, "pass parameters to child table, new record, pass something to another table, set something, return to the original table" all in one script; no external script needed, no jumping around trying to figure out what went wrong (not that it would go wrong for you, but it happens with me from time to time -)

So save your pennies and get a faster machine. Hopefully by then 7.0v3? will be out. You won't be sorry.

[Steve N]

Just a thought from the peanut gallery (those of us who are not professional programs).

I agree withe everything fenton amd marianco said.

One big help would be if you could import a file/table including layouts, scripts, lists and data. Then you could bring your old solution in one file at a time and tweak each one. That would be like rewriting, but not entirely from scratch. Unless I missed someting you can only bring file(s) in all at one time.

Along those same lines I wish you could duplicate a table within fm 7, so if you have two tables that are similar but different, you only create one then duplicate and make modifications to the second table. I have a job ticket and an estimating that work very similar, but serve different purposes and contain different data.

I do wish you could import date from a fm 6 directly into fm 7. It seams like FM could write a conversion program of some kind.

Any thoughts on if FM will do what I have suggested, or do you think I am off base?

[Fenton]

I would certainly second the "duplicate table" step. Maybe we'll get that someday. I can't say that I would need it all that often, but, as you say, sometimes 2 tables closely resemble each other. But it would just be the base fields.

I think that a lot of our "bellyaching" about 7 (and I've done my share) comes from the fact that we've got so many version 6 solutions, that we dream of magically converting to 7, and somehow FileMaker's going to rebuild them the best way possible. The less I have to mess with converted files the happier I am. It takes almost as much time to clear all the junk out as it does to start over from scratch.

As far as "import from 6," once again, I think it's a waste of time for them to even think about that (and I doubt it's possible). You just convert the old files to 7, then import data into a clean version 7 file. You can then copy/paste layouts (which saves time; if they're good layouts).

Scripts, well, you can import, then fix. I think they've done a great job, by commenting out stuff that can't be matched, it's much easier to fix. But many scripts in 6 are simply not needed in 7, or done differently. Conversion sometimes introduces steps you don't want, Select Window steps especially. But you have to know what you're doing to remove them. And "external scripts," jumping from file to file; a chain of these will go into 1 script in a multiple table file.

So, yes, I miss a little of the modularity of 6, with its (too) many somewhat "reusable" files (I used to do some tricks with renaming using Developer). I think you can get some of that back with FM Robot. I imagine FileMaker may include some of this functionality (way) down the road; but they've got more pressing concerns at the moment; and I hope they stay focused on fixing the few real bugs. Once those are fixed 7 is going to leave 6 way back in the dust.

[joepok]

Hey Bankmann,

I had a similar experience in converting a huge 110 FP5 solution. After investing $6,200 in a FileMaker 7 upgrade, and spending a gruelling 2 weeks in a partial (11 files) migration process, I was heavily disappointed in the sluggish performance. In fact, after a one day test run with only one user, I went back to the original database... I just couldn't subject 35 users to this degradation in performance.

But, the security and relational features are incredible! I am hopeful that perhaps a new Citrix server (our current one is a dual 1 Ghz WIn 2000 Server) may boost the performance enough to go live next week.

All in all, it will probably be about a $10,000 investment in software and hardware to upgrade to FP7... a hefty chunk of change for a non-profit...

On another note, I bought the MetaData Magic package from New Millenium and used the File Reference Fixer on my FP5 databases before the conversion and this made a nice performance increase (compared to the non-fixed conversion). It seems that if the File References aren't fixed first, FP7 spends a bunch of time looking through various file reference options.

See Ya

Joe

[Young Padawan]

I have to agree with Bankmann that Filemaker Pro 7 is a disappointment. I'm very new to using filemaker (I only started last week) and although I actually like the product, I can already see perfomance degradation and implementation issues between FMP6 and FMP7 and it's truly unfortunate because FMP7 has some very good ideas in it.

For example what actually precludes FMP7 from being compatible with older operating systems? I have seen some responses given that it's because of the new features and others that say the designers simply did not want to go through the hassle of supporting older operating systems which seems much more believable.

Also it seems hard to believe that a 1ghz or above processor is required to handle even the simplest of designs using those added features. It seems more like a product that was rushed out the door before the designers had time to optimize it.

Add to that the bugs, some of them obvious ones, and where do all these issues leave their loyal customers? I can imagine many are left hanging and are deciding to continue with FMP6 because it's already hard enough in these economic times to convince a company to upgrade their software without having to also require them to upgrade their hardware and operating systems as well.

I think that at the end of the day and after all the patching FMP7 will be a great product and most issues will be resolved, but for now we can't take advantage of that great product and it's a shame.

[H.P.]

Hi Joe,

I think you should spend time taking a look @ the converted solution more closely. My database has over 100 tables (not over 100 files) and it is doing very well with over 20 users.

I had the same sluggish problem before and noticed that even one non-optimized calculation is enough to slow down the database significantly. I don't think 2 weeks are enough to really have a partial optimized version. Perhaps you might want to start from the scratch the partial 11 files in filemaker 7 (use 11 files as 11 entities in 1 file), import the data, and do a test run.

I don't know why you want to go live so early without really testing it out more carefully. If this is not business critial, I think it might be to spend more time scrunitizing it.

I don't think your seal 1ghz win 2000 server should be fast enough to handle @ least 100 users. How many memory do you have in the server? Did you try to increase the memory cache in the Filemaker 7 server? The number of files might also be a factor. I thrver is a problem. A duink you should do more testing before putting extra $ in hardware.

Plese spend more time in testing. Go throught script one by one again, check file references. If situation permits, reduce the number of files. Make files entities in the database instead of keeping multiple files.

In my opion, most likely it is the non-optimize database that is causing the sluggish problem in your solution.

Anyhow, best of lucks!

Ha Pham.

[bankmann]

I guess I should clearify a few point about my conversion experiment.

I did this test with a fairly simple research article database. All it does is to store news articles.

When a record is created, the following happens (or is supposed to, anyway)

1) Serialnumber, creation date are automatically entered.

2) The highest serialnumber is calculated in a hidden field. (This runs through the whole database, obviously, and takes time. Definitely a point for redesign. But it is a function I need for maintenance.)

3) Checkmark (used in functions for weeding out duplicates or empty records) is automatically entered.

2) Value list pop up give user choice of topic.

3) User enters text/copied text and pictures, enters source information (source, publishing date), and comments, if needed.

4) Two easy, but I guess potentially 'heavy' calculations takes place whenever user exits a field: (1) For ease of use, I use a calculation field to combine all the other text fields, so that I can perform a free text search. (It's a function I don't use often, but feel I need. I realise it's also a reason for the database's size.) (2) To keep a mental track of usage, I calculate the content size of the main article text field, and also of all data fields comined (including blobs). The latter calculation could perhaps be a source of some of the sluggishness?

The value lists (only two) are built via relationships to an external resource file I use with several other databases. Seemed like a good idea at the time.

I think the database is simple in design, so the conversion shouldn't really have to be the source of my problems. If my computers are too slow or weak to run FM7, then I really don't have much choice. I don't have the budget at work to upgrade just for one program, nor can I afford a brand new Macintosh at home (wouldn't want a new PC there...). So, even if FM7 promises a lot, it's been a waste of my money?

I haven't even dared try a conversion on my asset management solution...

Bankmann

[Fenton]

Though perhaps 7 is slower on some things, I don't think you can entirely blame FileMaker for your particular situation. You've made some decisions which cause your database to run a little slower and be larger. In any case, there are some things you could do to speed it up and slim it down.

2) BruceR has come up with a brilliant exploit of 7's global fields, to replace the old Max(SerialID), which is a killer on large files.

Or use two globals, and set up the second to be an auto-enter calc using (lookupNext, lower). Autoenter calcs are very powerful.

Set up a relation from the first global to the ID field, throw in an impossibly high number.

Set the second global to

(GetAsNumber(LookupNext ( Relation::IDField ; Lower) ) + 1

You get back your answer. Globals can be lookups now!

3) You don't tell us exactly how this Duplicate is calculated. Sounds like a self-relationship. It doesn't sound like a problem, but you didn't really say what it was.

4.1) Personally I don't like these "concatenate everything" fields for Finds. You could get more or less the same thing with a script, by entering your "search" term into a global field; which can now be entered in a Header, so it'll work in List views also. Just doing this will cut your file size tremendously.

4.2) Calculating the size of everything in record? Bound to take some time, at least while it calculates, such as imports. Are you including Unstored fields in the calculation? Is the total field on the user layout? I don't really see how you can combine the length of text fields with container fields and get useful info. How do you use this field? Would Modification Count be useful instead?

[bankmann]

I've just updated to version 7.0v3, and I'm seeing improvements. Mostly with drop downs/pop ups and type-ahead. I haven't done a lot yet, so this sort of just a first impression.

I think I can see more clearly that the remaining sluggishness has to do with my own design. Your input have given me good pointers.

I'll be trying out the tip on handling serials shortly.

Yes, I do make use of self-relationships. But I've found a portal listing in such a large database to be painfully slow. (I just haven't gotten around to remove this portal and the automatically inserted checkmark it uses.)

The script searching for duplicates is sort of long, and fairly unsophisticated. Basically, it sorts by title, and then compares one article with the next, going through all articles from the first to the last. Each time duplicates are found, the article serials and titles are logged. I've also complemented this script with a step checkmarking the record. After the check for duplicates, a search lists the duplicate records, and I can quickly go through them.

I'm not sure how to make use of search in combination with a global field and scripting, but it sounds lik a good alternative. Given the size of the database, and the rate of its growth, I'm all for trimming it down to an acceptable size.

As for size calculations; both fields are on the 'working' view, providing visual feedback to the user. I know it's not absolutely accurate; it just gives a fair idea of the main field's and the record's sizes. The main field size calculation was added because of previous FileMaker versions' restrictive capabilities. The record size calculation is just a luxury experiment. I'm just disappointed overall performance got to be so bad in version 7.

As I mentioned, I've upgraded to version 7.0v3, and the improvements are quite noticeable, at least so far. Enough to make be try redesigning a bit to see what I can get out of it, at least.

Thanx for the input!

Bankmann

[bankmann]

I think can confirm improvement in version 7.0v3.

I find FileMaker much more responsive, and it seems calculations are run much more intelligently (on exit record). I seem to have no more problems with drop down/pop up or type-ahead.

Creating a new record takes a bit, though.

I haven't tried the upgrade on Windows, yet.

Bankmann

[bankmann]

Uhm....

FileMaker just crashed on me when I aborted an edit value list dialog...

Now, that's funny, version 7 doesn't seem to complain about files being closed improperly, but does perform a scan for unused blocks...

Bankmann

[Fenton]

You should really go back to a version that hasn't crashed. It is more difficult to tell in 7, if your file is small you may not notice the "scan" dialog.

As far as using a global field in the "find in all fields" script. You just have a global field in the Header (list view). It looks like a search field on a web page, with a button next to it. The script just enters Find mode, sets your global into one of the fields, then a new Request, sets it into another field, etc. for all relevant fields. Then Perform Find.

It's essentially the same as doing a straight Find in a "kitchen sink" concatenated field. I don't know if it would be faster or slower; I'd imagine a little slower, but not much, 'cause each field is smaller.

Have you tried a search for "!" to find duplicates? I don't really know how well that performs with large text fields; but I imagine it's faster than a Loop with "=". You could start by finding in the Title field. That would be fast.

[bankmann]

I tried the trimmed version of the database on Windows with version 7.0v2 before I upgraded to version 7.0v3 there. It seems trimming the database design has had much more effect than the upgrade itself; I hardly notice any difference in performance between the two FM versions (Something I did notice on Mac OS X.). I do, however, have a much more responsive database. I'm still bothered with slow drop downs/pop ups and type-ahead, but feel working with the database is a lot better.

I haven't noticed any of the other problems mentioned earlier with version 7.0v3 yet.

Fenton, I'll be fooling around with your suggestions for the search functionality this weekend, I think. Thanx for input!

Bankmann

[transpower]

I was very disappointed with version 5; I'm very happy with version 7--there are many more calculatiion functions and many more script steps. Working with tables is much more convenient than with files. The relationship diagrams are neat!