FMS8A Performance Issue

[carmenm]

We have FMS8A on a dedicated duo core server. We are experiencing performance issues where all queries (via CWP and FMS8 clients) are slowed up to 20 seconds, simultaneously.

On the actual client, just about any action will pause with the infamous coffee cup. A user could be clicking in a field to start data entry, or closing a file, or opening a file, or running a query, starting a new record, etc.

I feared this was due to load (we often have 15 clients up with 20 files open each in addition to a handful of CWP users). I also thought it might be due to old databases that have been converted several times over the years (originally developed in FM3) with many unstored calculation fields.

However, over this holiday break I have been able to close down all files on the server except three clean databases, which I have developed in FM8 with no unstored calculations.

Running a query on these clean databases occasionally and randomly causes the same 20 second delay. I was the only client on the server. At another time there were only two users, one CWP user and myself via the client, and we would experience the issue simultaneously.

In the past, I have contacted our server support personnel in another building who informed me that CPU usage on the server was only at 1-2%

Does anyone have suggestions for me? Does this sound like the same issue mentioned in the other performance thread?

If I have to contact FM tech support, what additional information should I have? Unfortunately I am not able to physically manage the server myself, but I should be able to get whatever information I need.

Thanks so much for whatever information/help you can provide.

Edited December 28, 2006 by Guest

[Steven H. Blackwell]

When asking questions such as this one, it is very helpful to provide a precise description of the hardware configuration in as much detail as you can.

Steven

[Martin Brändle]

As Steven said, we can not decide if you don't give more information. It may be a problem of the hardware (RAM, RAM amount, disk(s)). It may also be a problem of the database structure.

[carmenm]

Here are the server specs:

Windows Server 2003 SP1

Dual Intel Xeon 3GHz

2GB RAM

SAN storage

1GB ethernet

Filemaker Server 8.0v2

IIS 6

PHP

There are no other applications on the server. The only hosted web site is on the intranet specifically for FileMaker CWP, and it gets less than 50 hits per day. We aren’t using IWP, but it is also turned on.

As I said above, the server slows down suddenly, and all users connected to it are impacted simultaneously. Actions which are typically instantaneous take 15-30 seconds to resolve. When they do finally resolve, they all seem to “wake up” at the same time.

Over the break I closed all of our (~95) databases that were being hosted on the server. I left only 3 databases hosted, which I consider to be fairly clean. I can empty them and post them if needed. I was the only person using the database, and no one was using the web site. It paused for 20 seconds when I was scrolling down a list view of 300 records. I closed FileMaker client, reopened it, did the same thing again and it was instantaneous.

I have seen many other client actions get stuck in this way:

Deleting a record

Creating a record

Clicking in a field for data entry

Defining fields

Performing a sort

Performing a find on an indexed field

Switching to a different layout

And many others…

Unfortunately, I can’t describe any circumstance that will cause the hangup every time. I can’t even describe circumstances when the hangups seem to happen more often than others. I’m having a very difficult time calling it anything but random, but if anyone has tips for isolating the problem I’m happy to try anything.

I have asked about CPU usage on the server, it seems to hold steady at 1-3%, but I haven’t yet been able to check this DURING one of the famous coffee cups.

Thanks again

[Steven H. Blackwell]

Why are you using the 8.0v2 version of Server? Please upgrade.

Steven

[Steverino]

I've had the same issues...most often when first tabbing into a field or a pull-down. Usually things speed up after that, but I've had some odd and frequent delays with some data entry fields (pop-up calendar, value lists, going from layout to layout, etc.).

I don't know right now which version of Server Advanced we are using. Is there a way to return this information with a calc?

[carmenm]

I found my version number in the Remote Server Administrator.

We are going to apply the updates. This was something that had to go through our IT department which has a policy of not applying updates unless we are experiencing a specific problem the update will resolve.

I didn't see anything regarding this issue in the patch notes. But, I'll report back after the update.

[Steven H. Blackwell]

I don't know right now which version of Server Advanced we are using. Is there a way to return this information with a calc?

In the SAT Tool take a look at the Summary Overview page. it has the information. You should be running at least Server 8.0v3 and FMP client 8.0v3 or 8.5v1. Earlier combinations had many issues related to server side vrs. client side calculations.

Steven

[Steverino]

We are using the most recent server version -- and it's disappointing.

Do others of you have problem with lag time with remote (external server) access? In particular, start-up time, initial sorting, and auto-complete (with a set of 6,000 records) are all horrible.

[Steven H. Blackwell]

I have all kinds of WAN based (as well as LAN based) connections, and I am not seeing this type issue. There is an entire litany of items to check when troubleshooting this type thing, including clock speed of both server and workstation, amount of free drive space on workstation, memory assigned to cache on workstation, file architecture, NIC cards, circuit bandwidth and latency, etc.

Steven

[xochi]

Does your slowdown happen on a regular schedule? I have FMServer set to do hourly backups, and during the backups all clients slow down (slow enough that it is practically a full pause).

[Steven H. Blackwell]

I haven't had any reports of slowdowns while backups run.

Steven

[John May - Point In Space]

I don't know if it really helps you, but FYI we have quite a number of clients using FileMaker Network Sharing to log into their databases running on our servers here from remote locations, and have never heard any reports of this sort.

- John

[xochi]

Steven & John -- that's very odd. I had basically assumed that everyone was seeing the "slowdown/pause upon backup" issue that I am.

I wonder what's wrong/different with my system? Here are my specs:

Server

mac osx 10.4.8 on core duo 1.6ghz

OS & database on external firewire drive

FileMaker:

FMSA 8v4

FMS cache set to 480MB / flush over 00:10 minutes

1 gigabyte database

Backup Tasks

individual backup scripts for every hour of the day, backing up to a unique folder on the same drive.

What happens:

When the backup kicks in (at 00:00 on the hour) any clients that are connected slow way down for about 30 seconds. Nothing bad happens, and all tasks in progress recover and complete, but there is a noticeable slowdown for clients both on WAN and LAN (not sure about IWP clients).

[John May - Point In Space]

My comment was regarding FileMaker Network Sharing in general - not specifically when backups are running.

- John

[Steven H. Blackwell]

OS & database on external firewire drive

Why is this? Why aren't they on drives on the CPU?

Also, do you have at least 2 GB RAM isntalled?

Steven

[xochi]

Why is this? Why aren't they on drives on the CPU?

Also, do you have at least 2 GB RAM isntalled?

Steven

Yes, 2GB of RAM. Drives are on external firewire 400. The drives are not terribly fast ones (design goals of this server were low cost, low power and long runtime upon power failure, so we may be suffering a bit there).

Also, I just realized this: my "hourly" backup scripts are actually a series of rotating 24-hour backups. E.g. the "1pm" script runs once per day at 1pm, saving the backups to a folder named "1pm". This means that rather doing a series of 1 hour incremental backups, I'm actually doing a series of 24 hour incremental backups every hour.

Given the rather large database size (1GB+), and somewhat mediocre hard drive speeds and rotating 24-hour backups, it may be that there is simply a high ratio of cache-flushing to drive speed?

If others aren't seeing the slowdowns, then that gives hope that a drive speed upgrade may help.

[Ender]

Yeah, for good performance neither the OS or the database files should reside on a Firewire 400 drive. Use internal drives with a fast bus, or maybe a fast external with fiber channel.

I'd recommend the OS on an internal and the data on a second RAID (striped) drive with a fast bus.

[xochi]

One of my interests is in environmentally sound computing, which generally means low power. I've played around with 2.5" laptop drives as server drives. Great power savings, but not so good performance.

I just saw an announcement of 2.5" form factor Flash drives which boast some impressive performance -- probably beating the fastest 7200 RPM 5.25" raid configurations. $350 for 32GB.

http://www.extremetech.com/article2/0,1697,2103810,00.asp

These might make a killer drive for a low-power server.

[mr_vodka]

Cool, I wonder if I can stick this baby into my Playstation. hehe. :)

[xochi]

Another vote in favor of using Mac Minis (with better internal drives) as servers: http://www.networkjack.info/blog/2007/03/29/intel-macminis-the-os-x-blade-server/

[Steven H. Blackwell]

A mini Mac is not, [color:red]repeat, not an acceptable CPU or drive for a Server.

Steven

[Vaughan]

You are also turning over a 480MB cache over a 10 minute period.

I thought that big cache sizes were discussed years ago and somewhat dismissed as a bad idea.

[xochi]

A mini Mac is not, [color:red]repeat, not an acceptable CPU or drive for a Server.

Steven

You are right. It's not just acceptible. It's actually an EXCELLENT server (in the case where one defines quality based on performance, or perhaps even price/performance ratio.) What sort of metric do you use to determine quality?

[xochi]

You are also turning over a 480MB cache over a 10 minute period.

According to the docs, "By default, FMS attempts to scan the entire cache every minute [and flush it to disc]". So my setting it to a 10 minute interval should (in theory) result in less disk activity. As I understand it, this setting is basically controlling how long "stale" data is allowed to sit in RAM before it's written out to disk. Higher values will generally give better performance, but then you run the risk of losing data if the server crashes. Since my server is stable and I have hourly backups anyway, this is not a worry for me.

I thought that big cache sizes were discussed years ago and somewhat dismissed as a bad idea.

FMS5.5 and FM Client 6 had some cache performance problems, but as far as I can tell, in FM Server 7 and 8, you want as much cache as possible.

I'm not sure I understand the nuances of cache settings on FM Client in 7 and 8, but for Server I'm 100% convinced it should be as high as possible. I've done performance tests where setting cache too low results in horrible performance.

[Steven H. Blackwell]

Set the cache to the maximum allowed and set the flush interval to one minute unless you have very very large files that require frequent backups. This is the recommended Best Practice.

Steven