FMSA 9 Crashing

[doobiedog]

we are having the same problem with IWP crashing every 4 days or so and we have the latest update v2. Now we are trying a scheduled task to close the databases and all associated services, then copy our backups manually then restart the services. Below is the batch file.

@echo off

REM shuts down IIS Service

net stop w3svc

REM disconnects all clients from filemaker databases and gives them 120 seconds to comply

"C:Program FilesFileMakerFileMaker ServerDatabase Serverfmsadmin.exe" disconnect --yes --grace-time 120 -u administrator -p lou1948

REM closes all open databases

"C:Program FilesFileMakerFileMaker ServerDatabase Serverfmsadmin.exe" close --yes -u administrator -p lou1948

REM stops the database server

"C:Program FilesFileMakerFileMaker ServerDatabase Serverfmsadmin.exe" stop --yes -u administrator -p lou1948

REM shuts down filemaker service

net stop "FileMaker Server"

REM makes backup copies of all databases and puts the files in folders for the seven days of the week

xcopy "C:Program FilesFileMakerFileMaker ServerDataDatabasesTothOS*" "D:BackupsFilemaker Daily Backups%date:~0,3%*" /Y /Q /S

REM turns on IIS Service

net start w3svc

REM turns on filemaker service

net start "FileMaker Server"

REM may not be needed because starting filemaker service should start the database server

REM "C:Program FilesFileMakerFileMaker ServerDatabase Serverfmsadmin.exe" START SERVER --yes -u administrator -p lou1948

REM may not be needed because all databases should be open by default

REM "C:Program FilesFileMakerFileMaker ServerDatabase Serverfmsadmin.exe" open --yes -u administrator -p lou1948

REM reboots the server if any error is encountered

iisreset /restart /timeout:180 /rebootonerror

[Steven H. Blackwell]

Everyone having these issues please report them to FMI.

Steven

[Laurent2]

This issue is directly related to the amount of requests on FMS.

[Zardoz]

It might be worthwhile to do a network trace to see what types of packets are being received by FMS around the time of the crash. Examination of network trace results ultimately led to a better understanding of what was causing older versions of FMS to unexpectedly stop responding, as documented in the "It happened again: Lost host on network" thread in this forum.

Back then, it was random network delays in the delivery of UDP response packets from FMS to the client during the discovery phase of a client connection that were generating Internet Control Message Protocol (ICMP) port unreachable packets back to FMS; receipt of these ICMP packets would cause FMS to stop accepting new client connections. The only solution was to stop and restart the FMS service.

This problem was occurring for those who had upgraded to W2K or W2K3 servers from Windows NT servers; Windows NT did not pass these ICMP messages back to FMS, whereas W2K and W2K3 did; the versions of FMS in question (I believe it was affecting FMS3 to FMS5) were not programmed to handle receipt of ICMP messages.

The network traces were especially useful because you could see that, as soon as an ICMP packet was received, FMS would no longer respond to any more UDP packets received from clients who were attempting to connect.

The failure would occur each time a FMS response packet (UDP) would take longer than one second to traverse the network to a client anywhere on the network who was attempting to connect to the FMS. Normally, this wasn't a problem, as the traces showed that the FMS response packets are typically received by a client in well under 1 second. But randomly occurring heavier-than-usual network traffic could cause delays leading to the failure, especially when clients in regional locations (where network distances are greater) were attempting to connect.

As far as I could determine at the time, as of the version 7 product line, FMP and FMS no longer rely on a mutual exchange of UDP packets to establish a client connection, so the current issue reported in this thread likely doesn't have anything to do with UDP, but there may be some similar mechanisms at play with other packet types. I'm not currently in a work environment where FileMaker is used, so I can't do any testing with the more recent versions described in this thread.

Admittedly, the product versions are very different, but there are enough similarities in the symptoms described here with those in the earlier thread to suggest that a network trace might be revealing.

Edited February 8, 2008 by Guest

[Ted S]

I installed FMS 9.0v3 for the first time last night. I haven't had any crashing problems after install but the installation was troublesome.

After installing I kept on getting a Database Won't Start... error or some message to that effect. Turns out that the reason that the DB wouldn't start is because the license was already in use on the network. Earlier in the week I had installed FMS on a virtual machine so that I could be familiar with the install process and could do some testing. I forgot to turn the test box off.

But how did it know that the license was already in use? It must have done some kind of broadcast on the LAN looking for a license with the same value.

Now this makes me wonder if perhaps FMS does a periodic broadcast on the LAN (weekly maybe) looking for another server with the same license. If it is doing this broadcast then maybe there are issues of the type that Zardoz speaks of involved with Filemaker once again. I'm not saying that the reason for the trouble is because of two licenses in use at the same time, maybe it is because the broadcast or the return message simply gets messed up and the result is that FMS goes down.

Zardoz has made an excellent suggestion. Maybe someone having this trouble can check the packets.

PS. I hope it doesn't end up being me.

[Tompa79]

I also get faulty ntdll module... Does v3 solve this problem? Ted, anyone? Replies much appreciated.

/Thanks!

(I run fms9v2 today. Solution runs fine at one place with approx 10 users, but crashes at another with 15-20 users.)

[xtrim]

HI,

I've installed 9.0v3 tonight (on Win 2003 Ent) and it just made more problems for me.

outside everything SEEMS to work fine, BUT fmsadmin still have 9.02.151 as its version and WPE as WS seems to be down (although they are not).

AND fmsadmin keep sending me errors while the event viewr does not show any error nor warnings.

SO...If I were you...I'll wait.

Hope I could revert the installation..

Anyone else encountered an issue wityh 9.03?

[Tompa79]

Ouch, did you upgrade because you had ntdll issues or for some other reason?

If I do upgrade, I think I will do an uninstall and reinstall with a complete v3 install (without upgrade) if I can find one.

[xtrim]

Yeap, once in several days I had the ntdll crash, and one of the fixes in update 3 is this issue.

But apperantly it causes more problems then it fixes

[Tompa79]

Thanks for a quick answer! I will try a complete v3 re-install this week, no upgrade - hope that eliminates the issues you have encountered.

[Steven H. Blackwell]

I've installed 9.0v3 tonight (on Win 2003 Ent)

Enterprise is not an approved OS. Use Windows Server 2003 Standard Edition SP 2.

Steven

[Jim Burrell]

Thanks for a quick answer! I will try a complete v3 re-install this week, no upgrade - hope that eliminates the issues you have encountered.

Tompa79... how'd it go. Did this fix your issue??

We have the same problems here as quoted above in many other posts. Running WinServer2003 std.

Haven't opened a ticket with FM yet but guess I will.

Thanks,

JB...

[xtrim]

I ahve a strange workaround...2 week now working perfectly..

1 - install the v2 update

2 - install the v3 update

3 - install the v2 update

after the last v2 update the version stays v3 and the v3 problems goes away and the ntdll is now stable...strange I know, but its working

[Tompa79]

Actually, I did not install v3 until last night. The v2 server ran fine for approx, 2-3 weeks but then ntdll error occured again - but now I removed v2 from the computer and installed a fresh v3 - hopefully this will work.

If I stumble upon anything with ntdll again with this server I will leave a post here...

BTW, you can use FMS9v3 Trial availiable from filemakers website if you don't have a complete v3 install file - just change the pre-entered trial serial to your bought license key. Very nice solution - Thanks FMI!

[Tompa79]

Still running here. Clean v3-install. Installed four weeks ago...

[Tompa79]

It seems like murphy is laughing at me...

Not ntdll.dll this time though, but the error is similar in some ways I think.

New Post on OmniORB crash

[Spiro]

This problem has also happened to me and caused me a lot of grief. Despite calls to FMP support, no help. Then I saw this stream and decided to try the installation of V3 then V2. This did not work as it said that the version was incorrect (i.e. didn't like to install down to V2). So I decided to uninstall and try again. It was then that I think I stumbled on the problem (at least for me). Before I tell you this, a bit of background and my thoughts:

1. The problem for me: The server initially was working well for 12 months flawlessly. Then I made some changes to a few of the database field validatations which meant that some of the data would have ordinarily failed validation. I had scripts running daily on the server and each time they would run, due to data validation failure, the server would go into melt down and become non-responsive with nothing in the log. I initially upgraded to V3 due to this problem and then later realised that it was the script crashing due to data validation failure. Anyway once this was sorted out the server started working well again, for about 1-2 weeks that is. Then it randomly started to just shut down with nothing in the logs. I disabled backups, scripts, everything I could think of but it continued to crash. That is when I found this stream.

2. The WPE Component not responding error is not the problem but more a symptom. This occurs when you try and contact the IIS server when the server is down

3. When I tried to install down to V2 as suggested in this forum and it failed, I uninstalled and also went into the FMP server directories. It was then in the LOG directory (C:Program FilesFileMakerFileMaker ServerLogs) that I saw a file pe_application_log.txt. This is a log of all script errors. I could not open it because it was 4.6 gig (likely as a result of my previous script error) !! So I figured, if I could not open this file in Word, Note pad or text editor then if FMP server tries to write to it it will also have problems. I deleted the log and hey presto, no more crashes (for now).

So FMP you need to:

1. Have better handling of scripts errors when run server side or through the IIS (i.e. stop them going into a loop and enable timeouts to be set)

2. Limit the size of all logs and not just the Event log.

I hope this helps some of you as it has been driving me nuts!!!

I will keep you posted if any more crashes occur

[RafRey]

I am currently having this same issue, but with FMS 5.5 on W2K3 SP2. The FM services will run fine for a few weeks, then unexpectedly stop with an application error 1004.

I know this configuration is not supported, and I thought that was the issue. But if this is still happening with FM9, why upgrade?

Has anyone found a solution to this yet?

Thanks!

[Spiro]

NOPE Still crashing!! I am going nuts with this problem. I have tried everything I can think of!!! Every 4-5 days, the Database Server gets a red light and stops responding in the admin console. Can't restart unless the system is rebooted!!! PLEASE anyone have suggestions??

[John May - Point In Space]

Do you have SSL connections over FileMaker Network Sharing enabled? If so (and if possible) try turning them off and see if that helps.

- John

[Dan@sacs]

IWP + FMS10 +IIS+ win2003 SP2 = random unexpected exit sessions without warning.

Just wondering if anyone has experienced this while using the above and for some reason or another the iwp user is exited completely out of their session (as if someone has executed an exit application or the session has timed out - however there is no 'bad request - session timeout' msg and they had definitely not been within the session for the full extent of the timeout set value). This has started to appear ONLY this week after i realized that the web publishing log file was getting filled up with a lot of errors (saw your similar posts) on scripts that weren't valid during IWP e.g. adjust window, resize window, freeze window and other ones like go to record or go to field when there is no records so err 101 was occurring, so after installing the proper error capturing and disabling all of these non web compliant scripts i've found that for some reason its now randomly exiting on some say 1 out of every 10 (as a rough guide) users from their session after clicking a button that cycles through a lot of external DBs executing scripts/commits etc. Its as if the database isn't being found or for some reason its exiting, could one of the files be corrupted without it necessary telling me or without any issues during fmapp? or what specific scenarios could cause an IWP user to exit randomly like that? I thought it might be network based but it appears not to be. The weird thing is all of last year running the same scripts on FMS9 it worked like a treat AND it seemed to work fine before i started disabling these no compliant script steps, doesn't make sense though now as the log file is filling up at a much less rate (which i thought might have been cause of the original crashes we were having). To give you more of a background IWP/IIS has always crashed every 2 weeks (or so) and so i setup a schedule to restart it every week on the weekend which had been working well for a year or so, the start of this week it crashed a couple of times on Monday morning and i had to restart the server for IWP to work successfully then, so in attempting to resolve the server crashes it appears i've caused even bigger problems for our clients :