FMP Server and Threading Issues.

[fishma]

I have a client which has 5 local users within a LAN and 7 remote users via VPN, all hitting the FMP 5.5 server. The local users are experiencing the "worm" on some very simple work procedures (sorting on indexed fields...etc) and are complaining about the slow performance.

I know that FMPServer processes are single threaded. So what I suspect is that the remote users and their workload are dragging the down local users performance. Meaning, the local users server requests must wait in line behind the remote users server requests. This in effect slows the entire system to a grinding halt.

Any light somebody may shed on this subject is greatly appreciated.

Thank you for your input.

[Anatoli]

No, FM Server is excellent multithreaded application on multithreaded OS like NT or W2k.

[fishma]

My clent, although their network is mixed (Win2K, MAC) has their server set up on a Mac. Is the Mac my problem here? Could the remote users be dragging the system down? The work slowdowns are becoming unbearable and I would gladly move the server to a Windows machine if that would solve the problems. I am also selling the idea of a Citrix solution for the VPN users but that can and will be an expensive solution.

Again, let me know what you think.

[Guest]

fishma said:

I have a client which has 5 local users within a LAN and 7 remote users via VPN, all hitting the FMP 5.5 server. The local users are experiencing the "worm" on some very simple work procedures (sorting on indexed fields...etc) and are complaining about the slow performance.

As I understand it, sorting and indexing occurs on the server and only the found set is sent to the client. So a large task can slow down the server.You can download the FMPS white paper "Best Practices" or the PDF Administrator's Guide at http://www.filemaker.com/downloads/index.html#manuals.

[kennedy]

They have their server on Mac... OS9 or OSX?

[fishma]

9.2

[Kurt Knippel]

This does not strictly sound like a Mac problem. Filemaker Server is multithreaded on all OSs that it is capable of running on.

You could have any number of other problems going on. However I would first look at what is being performed and determine if there is a real problem or if the problem is impatience. Remember that network functions are like 100x slower than local access (i.e. a 10 second sort on a local DB could translate to a 1000 second sort when hosted).

Many people do not realize this and it does take some adjustment.

[fishma]

I understand the slow down for remote users. My question focuses on remote users slowing down local users. My local users are receiving the "network worm" when they are performing simple tasks. If the server is truely multi-threaded, then I shouldn't be having these problems. FMP used to be single a single threaded server. That is why a dual processor with FMP server will not increase DB performance. After reading the white paper on the server application, I am not convinced it has become multi-threaded. I am at a loss to explain these problems.

If I am wrong about what I have said, please clarify the issue for me. I am not too proud to accept that I don't know everything. From my experience, when it comes to how FMP Server operates and handles server requests, the answers start to become hearsay and conjecture.

Also, could you shed some light on the proper settings for the Disk Cache, Virtual Memory, and RAM Disk on the Mac 9.2 OS (Control Panel>>Memory). The white papers for Best Practices for Server addresses only Virtual Memory.

[kennedy]

And let me further stress those questions. In a security-related thread, it was asserted that FMS is a "thin-server"... that it does not execute any of the DB functionality, but rather just serves up the records/files to the FMP clients that do all the work. All FMS does is maintain the record locks from each of the FMP clients so that you don't have two FMP's changing the same record at once.

Based on that, I would not expect an FMS-hosted sort to take any longer than an local sort... other than the time for FMS to spew the whole file to the FMP client. My experience would support that, as I don't see much slow down due to FMS vs. FMP.

Is there a technical writeup anywhere that truly explains the protocol between FMS and FMP, which does what, and how performance is impacted?

Also, I don't quite understand how FMS could be truly multi-threaded on OS9... didn't know that was possible.

[Anatoli]

Switch the FM server to W2K -- client not server -- machine, set the background tasks to max and watch FM fly.

FM is very fast on PC's. 3-10 times faster is my best guess, and FM server is running as service on excellent multithreaded preemptive OS.

During 5 years on Windows our FM servers never crashed.

[kennedy]

Windows an "excellent multithreaded preemptive OS"!

Wow, and I didn't vomit while typing that. I could go

with "well tested" and maybe even "robust", but not "excellent" (ick).

That's like saying "Outlook is an excellent email client".

FWIW, I can say "ever since OS X Server 10.1 was released, our FM Server has never crashed, and performance has been more than adequate (no waiting)".

And OS X truly is an "excellent multithreaded preemptive OS"!

Independent of the "my OS is better than your OS" debate, I'd love to hear answers to the queries above. What is it that FMS actually does? What is it the clients actually do? And what information/dialog moves between the two?

Thanks for any enlightenment.

[kenneth2k1]

Haha

I will not enter the fray this time.

[Guest]

kennedy said: In a security-related thread, it was asserted that FMS is a "thin-server"... that it does not execute any of the DB functionality, but rather just serves up the records/files to the FMP clients that do all the work. All FMS does is maintain the record locks from each of the FMP clients so that you don't have two FMP's changing the same record at once.

Based on that, I would not expect an FMS-hosted sort to take any longer than an local sort... other than the time for FMS to spew the whole file to the FMP client. My experience would support that, as I don't see much slow down due to FMS vs. FMP.

Also, I don't quite understand how FMS could be truly multi-threaded on OS9... didn't know that was possible.

Administrators Guide P109 states that (FMPS 5.5) sorting.... and other calculation intensive tasks ... will impact other users.

P15 states the OSX multitasking is supported. It does not mention OS9.2

We are now 8 months running OSX 10.2, mirrored drives, FMPS 5.5 without a reboot.

[fishma]

This is exactly the dialog I wanted to illicit. I want a debate here. I feel that there is no right or wrong answer here. I'm not a newbie here. I am a professional who has been developing for 10 years. I know after 10 years, the answer PFM (Pure F'n Magic) doesn't cut it anymore. How does the server work and which platform is best? I have people investing a lot of money on my advice and I don't want to guess anymore. Gandolph, you stated that simple tasks will impact others and cited which page to find that information. But what is more important is why it affects other users. What is going on at the server that is slowing others down. What I don't want is an argument about which OS is better. You all will debate your favorite to the bitter end. What I want to know is why one OS is better than the other. Is OS Mac 9.2 multi-threaded; is OS Mac X multi-threaded; and even though we know Windows is multi-threaded, does FMP Server utilize the multi-threaded environment.

Kennedy, you are right when you ask for enlightenment here. It is imperative that we, who qualify ourselves as experts, warrant an expert rating. Let's focus on learning from one another.

[Guest]

Well, my hazy recollection that FMP Server uses host based indexing is apparently true. Moving to OSX may significantly improve performance over OS9, but don't do development on line and do backup the databases before shutting down the server.

http://www.filemaker.com/products/fms_faqs.html

How is FileMaker Server able to achieve the increase in performance and scalability?

FileMaker Server increases the performance of multi-user FileMaker Pro databases through asynchronous file I/O, multi threading, host-based indexing and other engine optimizations.

http://www.euregio.net/mactivity/0296nl/X0010_9602-ENNL__Claris_sh.html

Faster Data Access for Macintosh and Windows Users

Workgroups can experience dramatically faster performance compared to FileMaker Pro peer-to-peer networking when using FileMaker Pro Server 3.0 to host their FileMaker Pro 3.0 databases. Network intensive tasks, previously handled locally on a FileMaker Pro "peer-to-peer" network, such as indexing, are off-loaded to the FileMaker Pro Server 3.0, dramatically reducing the amount of data traffic that passes over the network.

http://216.239.53.104/search?q=cache:0xPk8ITbbokJ:www.cs.unc.edu/~hays/INLS191/lessons/databases.ppt+Filemaker+server+indexing&hl=en&ie=UTF-8

Filemaker Pro Server as backend database

Controls data and access

Handles indexing, sorts and finds

http://fmbench.markcon.com/fms55.html

Filemaker Server 5.5 & Client 5.5 are just as fast over a network as Server 3/Client 4 used to be, if not faster--as long as you're not running OS9 on your server hardware.

Server 5.5 for Mac OS X is 40% to 80% faster on the same hardware as Server 5.5 for OS9.

All my OSX-based FM Server tests were run on a Dual Processor G4. I don't know if the dual processors can be "blamed" for the speed increase over the OS 9 tests. I've heard that dual processors have limited impact on performance in Filemaker Server, and my suspicion is that, like Win2K, OSX's advanced I/O is responsible for the increase in performance.

[kenneth2k1]

Couldn't help myself:

Fishma, you said that you are using MacOS9, but didn't give any more technical specs.

I mean, every request to the program is going to be a thread on the processor table. So your system profile is essential to know if you have a good setup.

Ken

[Kurt Knippel]

Multi-threaded and Multi-tasking are two TOTALLY different things. Neither have anything to do with Dual/Multi-processors.

Filemaker Pro and Filemaker Unlimited are both single threaded, this means that it can only do a single "task" at one time. Most OSs are multi-threaded and have some level of multi-tasking built in, so that more than one task can occur in the OS at one time. Filemaker Server is multi-threaded and can run some number of tasks concurrently.

That all said, Mac OS 9.x does not have a very robust engine for multi-tasking or multi-threading and so any application run on this OS will not operate as well as one run on a more powerful OS, such as OS X or Win2K.

Once you understand all this, be aware that LAN use is 100x slower than local use (i.e. running off harddrive on your own machine) and that remote/WAN use is 10x slower than LAN use. This has nothing to do with what is happening on the server and either way, the Filemaker client will display this as the squiggly line.

Finally at some point the server (and you did not mention what hardware this is) can only handle so many tasks/threads at one time and can only dish out so much data at once, eventually it will seems to the clients that they are slowing down as the server strives to serve them all thier requests.

In general get the fastest harddrive possible, run in RAM disk if possible and get the fastest pipe from the server to your network (100 megabit at least 1 gigabit if possible). Use switches and not hubs, which allows a dedicated 100 megabit pipe to each port. At some point upgrqading to a more powerful processor is also helpful. More than 40MB of RAM allocated to FMServer can have a serious detrimental effect, and even having lots of free RAM on the system can be problematic.

Here is what we run on 5 Filemaker Servers: Apple G4/450 Cube each with 512MB - 1GB RAM, running RAMbunctious RAM disk with Filemaker Server and all the database residing in the RAM disk itself. We also use Peek-a-Boo to control processor priorities to our various tasks, giving Filemaker Server highest priority, and in many case lowering the priority to all other tasks. We have had these running for years without problems.

[DykstrL]

I guess I'll jump into this fray and add my two cents...

As to server performance for users: In my experience, there are several things that user do that can drastically affect other users on a FM Server. One is importing and exporting record sets. The data has to go from the server to the client and then to the output file. The other big hit to other users is complicated finds and sorts based on unindexed fields. The set first is searched for in the index on the server side and if it is not found it has to be created and processed on the client side, then added to the index which is stored on the server side.

Another performance hit that we have found by experience is indexed fields. We used to set just about all text, numeric, date and time fields as indexed, which slowed performance - especially opening files. We now only set commonly searched fields as indexed. It does slow some finds and sorts when a user does a find or sort on the unindexed fields, but overall the file performance is much improved for the other users.

As to the issue of which platform makes the best server: We have all flavors of FM Servers except Linux. OS 10.2 and Win2K workstation seem to be the fastest. My theory is that on these OS's FM Server is running as a service and not on top of the GUI interface. A really good NIC card and the fastest hard drives you can afford seem to be the key to fast performance. Too much RAM or too large hard drives can adversely affect performance if not set up just right.

[Anatoli]

RE: Filemaker Pro and Filemaker Unlimited are both single threaded

Why then my FM Pro and FM Unlim. are showing 15-20 threads on W2K in Task manager?

True is, that FM is presenting itself as single-threaded. You can easily block further progress with long sort or similar tasks.

To see proof that FM Pro *is* multithreaded do this:

Open FM so it stays in Open File dialog. Do nothing in FM, so first thread is waiting for user.

From OS double-click on FM database.

If the db open behind the thread no. 1 in thread number 2, your OS and FM must run in more threads and that is correct way of doing things. But there are big differences between various OS.

I believe the Mac X will be good system for FMS, but many users are reporting and complaining, that shutting down FMS is corrupting databases.

I wouldn't touch that configuration

And W2k or Linux costs much less

[Guest]

Use switches and not hubs, which allows a dedicated 100 megabit pipe to each port.

One caveat. Switches are not as resilient as hubs and a failed autoconfiguration can stomp network traffic. I learned the hard way and recommend _managed_ switches, where you can override the autoconfiguration. That said, a 100 pipe from the server to the switch can help, even if the clients are all 10. I am running about five hubs star_ed off a switch. Servers and routers also go to the switch. It may not be pretty, but I am not the only one doing this and I cannot afford any down time. Simply put, it works.

[Guest]

I wouldn't touch that configuration

You persist in this OS war nonsense. I run HP Unix, Windows NT and Mac OSX on my network. If you are going to point out a perceived weakness of OSX, why don't you also to discuss the past history of successfuly attacks against NT servers (including XP)? Did you patch _your_ SQL Server database servers in time?

[Anatoli]

Who is talking about war? I am interested ONLY and ONLY in performance/cost/reliability.

We can protect us and we are protected against NT attacks. In fact we didn't have single issue. So no history I am afraid.

But to use solution from ANY vendor, which is known to be so problematic like Mac X and FMS -- no thank you. Where you get the idea about OS war? Is it crime to say something about known problems? That problem was mentioned at least 100 times on different forums and lists.

If Windows will have problems with FMS, I'll switch to Linux. Maybe earlier, because of performance/cost comparison.

Is that also crime or call for war?

It is very interesting, that 90% of Mac users get upset, when there is some kind of problem with Mac and it is publicly mentioned. Why you are so touchy? Are you also suffering from "Mac superiority complex" as our server guy says often?

Bottom line -- any server OS, which corrupts databases on switch off or power down is piece of crap. That is my opinion for last 20 years and will be for rest of my life.

[Guest]

>We can protect us and we are protected against NT attacks. In fact we didn't have single issue. So no history I am afraid.<

So you have never had a security breach. I have never experienced corruption on shutdown. What is your point.

>Bottom line -- any server OS, which corrupts databases on switch off or power down is piece of crap.<

Lets see. I may tell someone there _could_ be a problem when shutting down the server and recommend that they back up the database first. This is on an OS and database that should not be shutting down. That person _could_ very well elect to go to Linux or NT based on this information. You on the other hand call FMPS on OSX a piece of crap and extol the virtues of NT. You are the victim.

[Anatoli]

RE: You on the other hand call FMPS on OSX a piece of crap

Not me, never ever. Some another users, yes, they do

You are victim of your imagination.

[Anatoli]

From http://www.filemaker.com/ti/108428.html

Occasional Damage To Files When FileMaker Server 5.5 Is Shutdown With Open Files

--------------------------------------------------------------------------------

Who Should Read This Article:

Customers running FileMaker Server 5.5 on all versions of Mac OS X and Mac OS X Server.

Problem Summary:

There have been reports of files occasionally being damaged or corrupted when FileMaker Server 5.5 is shutdown before hosted files are closed. Damaged files will result in an entry in the Events.log file similar to the following (For more information on the Events.log file refer to your FileMaker Server 5.5 Administrator's Guide):

File "filename.fp5" close error: database may be damaged; try opening it in FileMaker Pro. (10001).

FileMaker is aware of these reports and is actively investigating the issue.

Update Available:

An update to address this issue is not available.

Affected Products:

* FileMaker Server 5.5

Affected Platforms:

* Mac OS X (all versions)

[AudioFreak]

I would like to say that I work on a Mac and am considering moving to a Pc. As far as Filemaker goes there is not much difference between the two IMHO. Our Linux server recently had a problem and we had to pull the files off the main server and host them on another machine. The owner wanted to put them on the new Imac we just purchased. I absolutely refused to let this happen. IMHO anyone hosting files with FMS on a OSX machine is just asking for problems. Our files are now on a Mac running OS 9 until we get our drives back for the main server.

[kennedy]

Anatoli said:Bottom line -- any server OS, which corrupts databases on switch off or power down is piece of crap.

Anatoli, I agree with that statement. Problem is, you are blaming the wrong thing! Its not OS X shutting down and corrupting databases... its FMS shutting down and corrupting databases! So, its not OS X that is the piece of crap... its FMS. But that's not unlike the piece-of-crap security that FMS and FMP offer (in contrast to the solid security that OS X provides).

Why on earth FMI would not quickly fix this FMS bug is beyond me... unless they cannot reproduce it. They've said "there have been reports of"... but given I have never seen it in all the times I've shutdown and restarted FMS on my OS X machine, I have to wonder if they've never been given a reproducible bug report on it so they can get it fixed. I wonder further whether the reports they received weren't actually something different. Hmmmmm...

[Anatoli]

You are probably right. But the combination of those two is piece of crap. I do not care much, if I should blame Apple for producing OS X or FMI/Apple for producing FMS.

If I have to use FM I will always use only what works 100%. Right now it is FMS and Linux and W2k or NT.

RE: But that's not unlike the piece-of-crap security that FMS and FMP offer (in contrast to the solid security that OS X provides).

Any security is better than FM build in on any platform. In some ways, FM security settings are easy if you have to set up replacement machine for FM server.