FMS *loses* an entire day's work?

[Doktor Joy]

I have a very curious tale to tell.

At 19:11 on 15th July, FMS 9 launched a server-side script that appeared to execute successfully. However, no further scripts were launched after that time (and they should be every 5 minutes).

The solution was used throughout the day on 16th July. The users of the solution reported that it functioned fine. (The server-side scripts perform non-vital tasks so their failure went unnoticed).

At 17:00 on the 16th July, it was noticed that fmserverd was using 99% of the CPU constantly - a symptom that we have noticed before, of a server-side script being stuck in a loop.

All clients disconnected from the solution, and we attempted to shut down the server via Admin Console. It did not respond to this (and never does, in my experience, when server-side scripting has gone wrong).

So, the server box was then rebooted using a normal system-level restart. It restarted fine.

But -- the data in the solution was all rolled back to where it was at 19:11 on the 15th. All changes made to the data between 19:11 on the 15th, and 1700 on the 16th, were gone.

It seems therefore, that FMS did not do any of the following after the server-side scripting crashed:

1/ It didn't write anything out to disk - no cache flushing

2/ It didn't run any other scripts

3/ In fact, it didn't run any other schedules at all. The hourly backups were not written out, the last one occurred at 1900 on the 15th. The daily backup on the 15th scheduled for 2300 also did not happen.

But it did:

1/ Continue to serve the solution out, and reflect changes made to data, so that if one user made a change another user could search for that changed data later on.

In other words, the users couldn't tell anything was wrong.

But after the restart, an entire day of changes is gone. This is a CRM, Inventory Control and Accounts package, so you can imagine how much fun this is - losing an entire day's worth of data.

Anybody else had this problem? Anybody advise how we can counter it? We have hourly backups in place, but FMS didn't execute them. We can't use server-side scripts to test for this and warn us, because if it happens, server-side scripts stop working. But, to the users of the system, it appears that all is normal.

The setup:

Server: Mac Mini Duo-Core 1.83GHZ, 1GB Ram, OS X 10.5.4 with JSE6 *not* installed. Server version 9.0.3.325. No other software running on the server box.

The clients - Filemaker Pro 9 and 9 Advanced, no Web Publishing clients.

[Steven H. Blackwell]

Symptomatic of a drive failure and/or a memory failure.

Your 'server" is way out of spec, so I am not really too surprised.

Steven

[FudgeMaker]

Contrary to the reply which suggested the reason for the problem is that your server is out of spec, I have had exactly the same problem and I'm running an HP dual processor machine with 4Gb of RAM and an 8 disk array with 276Gb of space.

Luckily, this server is in testing and has not been deployed for live data, but the symptoms were the same. Server side scripts stopped running, the log file stopped being written to, backups stopped being run, but client copies of FileMaker could still open databases remotely.

I noticed it when the Admin Console would not let me access the setup saying that another user was editing the Logging Options screen.

Upshot: I can't deploy this server until I find out more.

Anyone with any ideas?

[LaRetta]

There are indeed strange things going on. We are up to spec as well. And we have all files closing and not logging the reason at all. Our Admin console locks up as well. But what is similar to your situation is that the date/timestamp on all the files does not always update. Users can work in the file for two hours (and their entries are recorded and modification timestamps update etc) but when the files unexpectedly all close, the FM files themselves do not update their timestamps. Very strange.

I have been watching and reading all server posts and the only thing in common appears to be server-side scripting is being used. That is NOT the reason all our files are closing (that is something else that FM engineers and Citrix people are working on) but regardless the reason for the file closures, those date/timestamps should be updating! We even switched to brand new server and still experienced problems. If one of you wasn't on Mac, I would suspect a Windows update because their last update dealt with ports 4000-6000 or some such series but it included 5003.

I would highly suggest that everyone with these types of problems contact FileMaker and open a case.