Jump to content

  •  

Photo

FileMaker Performance - can it work?


  • Please log in to reply
6 replies to this topic

#1 Mark Welch  enthusiast

Mark Welch
  • Members
  • 33 posts
  • FM Application:9
  • :

Posted 29 January 2009 - 03:45 PM

I am afraid that I'm at the point of pulling my hair out over FileMaker's performance.

I finally surrendered and called technical support and used my "one free ticket" on essentially the most basic question, "Does FileMaker work?"

The answer, in the end, is that FileMaker will not work for my needs. I am abandoning 30 days of work attempting to get this tool to work for my needs. I will contact the client whose project first led me to this product, and advise them that the project is returning to "zero," with no technology solution selected. My personal project will also revert to exactly where it was 6 months ago when a programmer "flaked out" after partially developing a solution using PHP and MySQL.

FYI, the issue seems to be that FileMaker's "sweet spot" for performance lies somewhere between 250,000 and 500,000 records in the database (it might be the database size, I don't really know). When I pushed it past a million and then past 2 million, it slowed to a crawl and could simply no longer function. When I went through an excruciating series of steps to remove data, it crawled while removing records until it fell somewhere around 500,000, at which point performance was fast again.

Both of my projects require a database tool that can work with many millions of records. Therefore, I'm returning to MySQL and PHP, and will be starting over.

What's especially frustrating is that FileMaker (the company) kept "stringing me along," suggesting that if only I upgraded, my problems might be solved. So I did so, and I invested more time, until I was unable to proceed because of FileMaker's undisclosed limits.

I have uninstalled FileMaker from my system, and I am requesting a refund of the license fees paid. I don't know if they'll refund my money, but I need to stop wasting time on this solution and move on to a database solution that will work.
  • 0

#2 Rich R  enthusiast

Rich R
  • Members
  • 32 posts
  • FM Application:9 Advance
  • :

Posted 29 January 2009 - 03:59 PM

Hmm. I find that interesting. One of our solutions is running with extensive processing of 3-5 million records without any problems. It's an all mac shop.

But not every tool fits the job - such is the way of IT.
  • 0
[color:purple]CornerStone Solutions/IG Inc
rich@thinkcornerstone.com

#3 KirkR  enthusiast

KirkR
  • Members
  • 68 posts
  • FM Application:9 Advance
  • Time Online: 1h 51m 54s

Posted 29 January 2009 - 05:14 PM

One of the prime reasons for performance issues in FM is the generation of indices. More often than not, many more fields are indexed that are necessary for the application.

Although indices make for faster searches, constantly regenerating these indices eat processor and memory (emphasis on the latter).

Relational design has lots of pitfalls in performance issues. Took a VSAM flat file DB on a mainframe to a Oracle relational model (375 terabyte DB of 3 months window of data, changing constantly) on a 256 way Sun E10000. The processing time went from under an hour to over 27 hours to produce the reports - the migration project got canned.

Filemaker has a unique and powerful construct that affords significant performance improvements. Using global match fields and self-join table occurrences, you can have many of the select capabilities performed with near zero performance impacts.

The lesson that I guess I am attempting to communicate is that there are ways to get or restrict performance, mostly in design. Many factors impact performance, and FMs table occurrence model affords significant performance benefits, but requires some good - and possible unique - design approaches.
  • 0
"If you do what you have always done, you will get what you have always gotten" Tony Robbins

#4 Mark Welch  enthusiast

Mark Welch
  • Members
  • 33 posts
  • FM Application:9
  • :

Posted 29 January 2009 - 06:50 PM

> "...requires some good - and possible unique - design approaches" <

And there's the issue: to achieve success with FileMaker, I need to adopt and learn a "completely different" way of doing things. My whole reason for using FileMaker was for simplicity, ease of use, and quick development -- not a long learning curve requiring help at every single stage.

The cache issue is a perfect example. On the phone today, after I had complained several times about the fact that even running "flat-out," FileMaker rarely used more than a few percent of CPU capacity (peaking at 9% maximum), the FM technician eventually suggested that I check the cache size setting, which defaults to 8MB. Huh? An 8MB cache on a computer with 4GB? My cache size, for some bizarre reason, was 7MB. The technician suggested that I change it to 16MB to see if performance improved; I immediately asked, why 16MB? Why not 32MB or 64MB? The technician couldn't answer. When I set it to 16MB, CPU usage seemed to nearly double; when I set it to 64MB, CPU usage bumped up to 44 percent (it's a dual-core CPU; FileMaker can use only one, so its max CPU% would be 50%). I tried in vain to find any useful references to the cache settings in my FileMaker books or in the online help (I did find some comments in this forum, of course, but only after I knew what to search for). A learning curve, indeed.

And yes, getting CPU usage from 2% up to 44% would certainly have improved overall performance (especially in sorting and summarizing, and in deleting records). It would still not have solved the problems that triggered the worst performance (and I was only just started, with more fields still needing to be indexed, and more tables needing to be related).
  • 0

#5 KirkR  enthusiast

KirkR
  • Members
  • 68 posts
  • FM Application:9 Advance
  • Time Online: 1h 51m 54s

Posted 29 January 2009 - 07:10 PM

I believe that if you shift to a couple of self-joins, particularly for those things that are taking long times, you should see radical speed improvements.

I posted a global field URL_match approach to one of your comments. It is really quick to implement. If you are stuck, comment back, and I'll walk you through it. If you have something else that is more time intensive, describe it, and we'll see about walking through a quick and dirty test, so you can compare the speed. I think you'll be amazed.
  • 0
"If you do what you have always done, you will get what you have always gotten" Tony Robbins

#6 gdurniak  journeyman

gdurniak
  • Members
  • 234 posts
  • FM Application:8
  • Time Online: 1d 14h 41m 15s

Posted 07 July 2010 - 08:58 AM

I realize this is an old post, but could you please give an example of using a self join, to speed things up

thanks!
  • 0
Gregory Durniak
Bayside, NY
www.fileshoppe.com

#7 HOnza  enthusiast

HOnza
  • Members
  • 43 posts
  • FM Application:13 Advance
  • Platform:Mac OS X Mountain Lion
  • Skill Level:Expert
  • Certification:8, 10, 11, 12
  • Membership:TechNet, FileMaker Business Alliance
  • Time Online: 13h 43m 33s

Posted 25 February 2012 - 01:39 PM

Every database software has things that it does very fast and other things it does not do so well. I have seen a FileMaker database containing over 10 million records performing find in less than 1 second (over network).

When your solution becomes slow at some point it is most probably due to (not intentionally) using one of the things that are slow by nature (or by design), but there is often a simple way to avoid using that thing and implementing your feature in a more efficient way.

The only problem is finding the one point where this happens and which you should focus on. If you want to learn how to find this weak spot, check out my video at http://fmbench.com/bottleneck, or you can go directly to the homepage of 24U FM Bench, our new product built specifically to address this issue.

If you want to learn (or find help) how to optimize the bottleneck you find, or discuss how to get the most out of FileMaker Pro in specific tasks, you may find the FileMaker Optimizers LinkedIn Group useful.
  • 0

HOnza Koudelka
Software Division Manager, 24U s.r.o.
Filemaker Business Alliance Member
FileMaker 8, 10, 11, 12 Certified Developer
mailto:honza@24uSoftware.com
http://www.24uSoftware.com





FMForum Advertisers