Mr F

Hey there DrewAngell, Long time, no responses, eh? I have to admit, my experience with FMP/FMP support was exactly the same - We got the issue escalated umpteen times up to engineering, both directly and through one of the leading accredited developers here (Australia). We got absolutely no movement though (or indication as to what might actually be wrong). Without wanting to give away any exact specifics, I actually got as far as meeting people VERY high up in engineering/Asia-pac operations (think somewhere near the top). They basically apologised for the lack of support and the fact that the product didn't work and said they were going through a difficult stage with support of going from a desktop app to a (supposedly "enterprise") server product. The upshot was, it doesn't work properly - there are known/unfixed bugs, and the answer (at the time - this was a while ago) was basically: "Maybe wait until FMP 9 comes out and use the remote data source feature instead". Very poor service overall IMHO. We to are basically switching to an entirely different platform because of the awful integration performance. Cheers, Mr F.

Hi guys, As a "regular contributor" to this thread, I thought I'd chime back in. I've been away for two weeks on a FileMaker free holiday, so just catching up on things now - We still have an active support case with FileMaker, but have heard absolutely nothing back from support for several weeks now (since I last posted). I'd love to be able to give our DB setup to engineering to replicate, but unfortunately, it's rather huge and complicated. I'm hoping I can boil it down to a simple/replicatable issue on a smaller/distilled set of tables/data, but the exact profile that causes our problems to occur is obviously complex enough that it hasn't been found by FileMaker's normal/in-house testing or from enough other customers that they're aware of the problem. This now appears to be the main thread on the internet (according to google) about WPE/FM performance issues, so hopefully together we can help FileMaker sort it out. More news as it happens... Cheers, Mr F.

Just adding on to my own findings, here's further info we've worked out/sent through to FileMaker engineering: ----- Furthering to this, we've been doing some further investigation - We've noticed that the web publishing engine seems to be able to get itself into a state where it will get stuck in a loop of some sort and utilize all CPU's at around 35% forever, until the service is forcibly terminated and restarted. We've had this a couple of times, including today, where we cut off all external access to the WPE. At this point, we proved there was NO traffic hitting the WPE front end (via netstat) making requests, yet the WPE (wpc.exe) sat utilizing all 4 CPU's non stop, seemingly forever (see attached image, right side of graph). When this is occuring, the WPE runs VERY slowly - ie: 2-3 minutes to answer ANY request. It also will not terminate gracefully via the services manager, and has to be forcibly killed and restarted to fix the problem. This does not happen all the time and is separate from our general bad performance issues, but I thought the info may help the engineers track down where the problem lies. -----

Just a further update on my investigations - the below is essentially paraphrased from part of my FileMaker support process: ----- It appears that the WPE doesn't thread properly, or has some sort of locking/race-condition issue in some cases. I set up a test suit that attempted to gather performance metrics in pulling up a record via primary/indexed key and the impact of threading/multitasking (ie: what number of simultaneous queries gave the best performance). To my surprise, I have found that the WPE does NOT answer multiple requests at once, or rather, some sort of locking issue causes them all to be answered asynchronously, ie: 1 query takes 3.5 seconds, 2 queries take 7 seconds, 3 queries takes 10.5 seconds (etc, etc) - See attached PDF for full results. In this case, NONE of the queries respond until all have completed, making it look like there is some sort of race-condition/locking type issue, which explains the terrible performance (in our particular setup). So far, the problem is 100% replicatable all the time on all machines tested (tried both on single CPU FMSA/WPE combined server and separated 2 x 4 x CPU separated FMSA/WPE servers). I would hazard a guess that it is due to the number of complex relationships in our particular setup, but have so far not narrowed it down to precisely what recipe of relationships/etc allows this specific problem to occur. Either way, it does highlight some sort of very real problem with the way the WPE is locking/querying, where it becomes impossible for us to run more than 1 query at a time on a given table via XML. ------ I'd be interested in seeing if anyone else is able to replicate/has seen similar issues with their own FileMaker/WPE/XML setups. Will let you guys know how I proceed, Mr F. FileMaker_non-threading_performance.pdf

Hi there, If you have no major requirement for data to be realtime, then you'd be better off doing a batch export of data from FileMaker to flat XML (then into a MySQL DB or somesuch), or to propegate it via ODBC. Alternately, FX.php does contain a caching module (PHPCACHE which is fairly undocumented), which you could very possibly use to get around most problems. Unfortunately, for our application, a lot of the data MUST be realtime, which is what causes all our problems. You might be OK, but I certainly wouldn't attempt to run any sort of website that relied on FMP queries via XML to serve pages to give any sort of reasonable performance. Cheers, Mr F.

Hey DrewAngell, This sort of CPU spike is the sort of performance issues we've seen (see previous posts by myself). In answer to your question - from our experience, the XML interface does NOT scale well, particularly with parallelizing requests. We found we had to implement write-back caching and write all data syncronously just to keep performance even slightly vaguely acceptable. Combining this with very heavy caching of reads has been the only way we've been able to maintain stable (if fairly awful) performance. Keep in mind, this is running on two very high end quad-CPU xeons with 4GB RAM each and you can barely maintain a few simultaneous requests. We have a support process happening with FM, and I'll post my results/experiences as I move forwards. Cheers! Mr F.

I have a tech support case open and have spoken to the head engineer at FileMaker Asia-Pacific. So far, they don't know what causes this, but it's been escalated to US Engineering. I am currently working through that process with them. I'll keep you guys updated as to how I go, but so far, the progress has been quite slow. The indication/feeling I got from speaking with them (the Asia-Pac engineering were very helpful/thorough) is that it could take a long time to solve (ie: many months). Any info that we're able to work out independently would be greatly appreciated. I'll keep you guys updated as to how I go.

This is interesting stuff (nice work by the way)! I'll be very interested in the results you get here. That said, are the performance issues you experience easy to replicate just by increasing load slightly (ie: submitting multiple queries at once)? The reason I ask is that my setup is fairly snappy in a test environment (and for a portion of the time in production, for that matter), it just all falls apart horribly when load increases even slightly. Cheers again for the efforts.

Our machines (DB server & WPE) both have 8GB of RAM, of which only 4GB is addressable by windows (hah!). They are both quad CPU 3.2Ghz Xeons with GigE between them. They (IMHO) should be well and truly massively overspec'ed for what we are attempting to do. Interestingly, once the WPE slows down, the size of the resultsets becomes irrelevent. You can execute a query that results in a single integer being returned, and the performance is still absolutely atrocious (ie: 2-3 minutes). This is a very real and severe issue. We have had no response from FileMaker (US) yet either.

I am experiencing this exact problem. We found that as soon as the load increases on the FMSA server, the response times to queries go up MASSIVELY. ie: a 10% increase in load causes a ~580% drop in performance. We have extensive logging and graphing of CPU utilization/load/etc vs query times, and the end result is that if ANYTHING significant is happening on the FM server, the WPE basically hangs. This has been replicated and reported to FileMaker (Australia in my case) and has been escalated to engineering in the US. No response as yet....