Error 701 Database Server process has terminated abnormally

[matthewrollinson]

Hi!

I recently moved my Filemaker databases to a new Windows server. And since then, I’m having sporadic “Database Server process has terminated abnormally” Error 701. This is accompanied in the Windows event logs with application error 1000, and these are the details:

Faulting application name: fmserver.exe, version: 15.0.3.308, time stamp: 0x58124553
Faulting module name: ntdll.dll, version: 6.3.9600.18438, time stamp: 0x57ae642e
Exception code: 0xc0000374
Fault offset: 0x00000000000f1b70
Faulting process id: 0x624
Faulting application start time: 0x01d2a8b983d812e3
Faulting application path: C:\Program Files\FileMaker\FileMaker Server\Database Server\fmserver.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 2fea4014-160e-11e7-80c2-109836ac1fba
Faulting package full name:
Faulting package-relative application ID:

This did not happen on the older Windows Server, which was 2008. The new server is 2012 R2 Standard

It first occurred on the first day I starting using the databases on the new server. So updated FM Server 15 to latest (15.3). I then performed all available Windows updates, updated Java. Then a few days elapsed before Filemaker failed again, with exact same errors.

So I performed a recover on the databases within Filemaker, just to check. All fine. I also downgraded FM Server to version 14 to eliminate FM Server 15 as an issue. After downgrading to 14 we had 6 days then before it failed. But last night it went again with same errors.

It seems to occur when a scheduled Filemaker script is being run on the server. But oddly, when the server is quiet, ie. After 18:00 when people have logged off and gone home, or at lunch break.  The Filemaker scripts are run every 30 mins during office hours, so run many many times over the days between failures with success, and did so for years before I moved to new server.

When the 701 error occurs, I’m forced to reboot the server. It doesn’t auto restart the process after failure.

I would be incredibly grateful for any advice on this. Anyone aware of issues similar?

Edited April 7, 2017 by matthewrollinson

[Wim Decorte]

What does the schedule do?  Does it use plugins?

The indicated error is a Heap Corruption.  Since it happens with both 15 and 14 on your machine I would be inclined to think it is something in the deployment itself, something on the server that is different.

Any non-FMS backups going?  Any anti-virus scanning?  Any Windows snapshots?

Are there any DMP files in the FMS logs folder?

If you go back to FMS15, enable the top call stats log and also make sure that the stats.log is enabled (it's off by default).  That could help show some performance data leading up to the crash and the top call log report may show exactly what FMS is doing when it runs into the issue.

[matthewrollinson]

Thanks Wim.

The scripts don't rely on any plugin. My database is a CRM, that connects to a remote Magento store backend, the Magento db is MySQL. FMS connects to the remote db via a system DSN using MySQL ODBC driver on the same server. The scripts are a rudimentary method of getting Magento records > Filemaker local database for office/sales staff use. The Script searches the MySQL shadow table for new records every 30 mins, copies a primary ID, performs a lookup or autocalc for the rest of data and then flags that record as “synced”.  Probably not the most elegant solution, but it works. And these scripts worked without issue on the old server since 2011.

No antivirus. No non FM backups running. No checkpoints. And no, nothing in the logs file other than Event log.

Over the weekend I will revert to FMS15 and enable top calls stats log.

Edited April 7, 2017 by matthewrollinson

[Wim Decorte]

Note that when you use ESS, every FM version has specific requirements for the ODBC drivers to use and the supported version of the MySQL backend.

http://help.filemaker.com/app/answers/detail/a_id/6420/~/which-odbc-data-sources%2Fdrivers-are-supported-with-external-sql-data-sources%3F

If you don't use the supported versions of the driver or backend then you can have all sorts of crashes.

[matthewrollinson]

The ODBC driver version was in the back of my mind tbh.

And actually I'm using the lastest version, MySQL ODBC 5.3 Unicode Driver version 5.3. Where as that article suggests MySQL ODBC 5.2 Unicode Driver version 5.2.7. As the error occurs at the point where the ODBC connection is being utilised, that may well be the factor!

Although..on my old Win 2008 server I was still using 5.3. Certainly worth a shot though.

[martinkutter]

Hi Matt

I know, it's a few months later, but we have to exact same issue on a clients' server.

Did you replace the ODBC driver on the Win2k12 server (downgrade) and installed 5.2.7 as suggested on https://support.filemaker.com/s/answerview?language=en_US&anum=6420 ?

No more crashes since?

Thank you for a quick note about your experiences.

[AlbertoGalla]

HI All, I've the same issue that a could not fix. I've an external backup service which run on all computer of the company. Could that be the problem?

what happens is: 

restarting the server this comes: "Errore 701 SERVERFM" Process FileMaker Data API Engine terminated anomally.

and than

 

HTTP Status 503 - Servizio non disponibile: la macchina worker non è in funzione.

---

FileMaker Server ha incontrato un errore: 
The requested service is not currently available.
 

Se la pagina di avvio del server non è disponibile, per assistenza visitare il sito Sito Web FileMaker.

Service not available: the machine worker is not working

but actually everything is running properly.

thank you

Alberto