FMS14v1 FileMaker Script Engine process has terminated abnormally

[xochi]

Since upgrading a server from FMS11 to FMS14, we've seen periodic (about weekly) crashes of the FileMaker Script Engine (FMSE also called fmsaed or fmse).

 

Anyone else seeing this?   The two most recent crashes are shown below.

They don't look like a crash in the same location, but they both look like a crash entirely within the FileMaker framework.    ( We had an earlier crash which was due to Dacons Mail.It plugin crashing, but we believe we solved that issue and it doesn't seem to be relevant for these crashes).

 

Any ideas?

OSX 10.10.5 with all updates.

FMSA 14v1

(We haven't updated to FMS14v2 yet as the release notes don't suggest any reason to).

 

Thread 19 Crashed:
0   com.filemaker.support.framework    0x0000000109757ebc std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> >::basic_string(std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> > const&) + 16
1   com.filemaker.dbengine.framework    0x000000010834d235 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 297
2   com.filemaker.dbengine.framework    0x000000010834d0f9 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::insert(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 111
3   com.filemaker.dbengine.framework    0x000000010834c89e Draco::DBStrMap<Draco::DBPluginOpcode>::Add(Draco::unistring const&, Draco::DBPluginOpcode const&) + 144
4   com.filemaker.dbengine.framework    0x000000010834908c Draco::DBCalcScan::RegisterExternalFunction(Draco::QuadChar const&, short, Draco::unistring const&, Draco::unistring const&, short, short, unsigned int, int (* const&)(short, Draco::DBExpressionEnv const&, Draco::DBDataVect const&, Draco::DBData&), bool) + 262
5   com.filemaker.fmengine.framework    0x0000000108c1eea6 Draco::InstallDesignFunction() + 250
6   com.filemaker.fmengine.framework    0x0000000108c2acde Draco::FMSession::FMSession(Draco::DBAuthenticationInfo const&) + 644
7   com.filemaker.fmengine.framework    0x0000000108c60a75 Draco::SMSession::SMSession(bool, Draco::DBAuthenticationInfo const&) + 21
8   fmsased                           0x0000000107b94d10 FMScriptThread::RunScript(FMSTypes::ScriptMakerOperation const*, Draco::DBAuthenticationInfo&) + 88
9   fmsased                           0x0000000107b94a2e FMScriptThread::Main() + 378
10  com.filemaker.support.framework    0x000000010978234f Draco::Thread::RunThread() + 59
11  com.filemaker.support.framework    0x00000001097813d0 Draco::Thread::ThreadUnixProc(void*) + 54
12  libsystem_pthread.dylib           0x00007fff9327005a _pthread_body + 131
13  libsystem_pthread.dylib           0x00007fff9326ffd7 _pthread_start + 176
14  libsystem_pthread.dylib           0x00007fff9326d3ed thread_start + 13

 

 

Thread 20 Crashed:
0   libsystem_notify.dylib            0x00007fff9c5b60d0 _nc_table_find_64 + 42
1   libsystem_notify.dylib            0x00007fff9c5b46ae notify_check + 92
2   libsystem_c.dylib                 0x00007fff8e6b4797 notify_check_tz + 24
3   libsystem_c.dylib                 0x00007fff8e6b447c tzsetwall_basic + 45
4   libsystem_c.dylib                 0x00007fff8e6b46e0 _st_tzset_basic + 363
5   libsystem_c.dylib                 0x00007fff8e6b5e6f localtime_r + 41
6   com.filemaker.support.framework    0x0000000104b3f53e Draco::DateTime::NowMicroSec() + 40
7   com.filemaker.support.framework    0x0000000104b40dda Draco::Debug::MessagePrefix() + 60
8   com.filemaker.support.framework    0x0000000104bbf3cd Draco::Thread::RunThread() + 185
9   com.filemaker.support.framework    0x0000000104bbe3d0 Draco::Thread::ThreadUnixProc(void*) + 54
10  libsystem_pthread.dylib           0x00007fff9327005a _pthread_body + 131
11  libsystem_pthread.dylib           0x00007fff9326ffd7 _pthread_start + 176
12  libsystem_pthread.dylib           0x00007fff9326d3ed thread_start + 13

 

 

 

 

[laurentades]

Even though I am slowly testing FMS14, i have lately wasted a lot if time on similar matter only to discover that my problem was with plugins: either not installed/registered correctly, filesystem level permissions issues or simply unsupported with the FMS version i was using or even more frustrating, reacting  unpredictably to processing exceptions depending on value of arguments passed to their functions...

Your log does refer a lot to plugin and namely Draco. This is where i would start first... Get support from the author for any of the areas cited above.

Hope it helps.

 

[xochi]

Unfortunately, Draco is the name of FileMaker's internal database engine - not a plugin.

[xochi]

Another crash last night:

 

Thread 20 Crashed:
0   libsystem_kernel.dylib            0x00007fff97122286 __pthread_kill + 10
1   libsystem_c.dylib                 0x00007fff8e6bd9b3 abort + 129
2   libsystem_malloc.dylib            0x00007fff9049e1cb free + 428
3   com.filemaker.support.framework    0x0000000102d62462 std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> >::assign(std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> > const&) + 114
4   com.filemaker.dbengine.framework    0x00000001019984eb Draco::DBStrPair<Draco::DBPluginOpcode>* std::__copy_backward<false, std::random_access_iterator_tag>::__copy_b<Draco::DBStrPair<Draco::DBPluginOpcode>*, Draco::DBStrPair<Draco::DBPluginOpcode>*>(Draco::DBStrPair<Draco::DBPluginOpcode>*, Draco::DBStrPair<Draco::DBPluginOpcode>*, Draco::DBStrPair<Draco::DBPluginOpcode>*) + 81
5   com.filemaker.dbengine.framework    0x0000000101998193 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 135
6   com.filemaker.dbengine.framework    0x00000001019980f9 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::insert(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 111
7   com.filemaker.dbengine.framework    0x000000010199789e Draco::DBStrMap<Draco::DBPluginOpcode>::Add(Draco::unistring const&, Draco::DBPluginOpcode const&) + 144
8   com.filemaker.dbengine.framework    0x0000000101994057 Draco::DBCalcScan::RegisterExternalFunction(Draco::QuadChar const&, short, Draco::unistring const&, Draco::unistring const&, short, short, unsigned int, int (* const&)(short, Draco::DBExpressionEnv const&, Draco::DBDataVect const&, Draco::DBData&), bool) + 209
9   com.filemaker.fmengine.framework    0x0000000102267ea6 Draco::InstallDesignFunction() + 250
10  com.filemaker.fmengine.framework    0x0000000102273cde Draco::FMSession::FMSession(Draco::DBAuthenticationInfo const&) + 644
11  com.filemaker.fmengine.framework    0x00000001022a9a75 Draco::SMSession::SMSession(bool, Draco::DBAuthenticationInfo const&) + 21
12  fmsased                           0x00000001011e2d10 FMScriptThread::RunScript(FMSTypes::ScriptMakerOperation const*, Draco::DBAuthenticationInfo&) + 88
13  fmsased                           0x00000001011e2a2e FMScriptThread::Main() + 378
14  com.filemaker.support.framework    0x0000000102dd134f Draco::Thread::RunThread() + 59
15  com.filemaker.support.framework    0x0000000102dd03d0 Draco::Thread::ThreadUnixProc(void*) + 54
16  libsystem_pthread.dylib           0x00007fff9327005a _pthread_body + 131
17  libsystem_pthread.dylib           0x00007fff9326ffd7 _pthread_start + 176
18  libsystem_pthread.dylib           0x00007fff9326d3ed thread_start + 13

[xochi]

Decided that I'm tired of this bug, so I wrote a little shell script - it simply tries to re-launch the fmse engine every 10 minutes.

 

#!/bin/bash

echo '##################################################'
echo '# FMSAdmin FMSE Restarter'
echo '# will try to re-launch fmse every 10 minutes'
echo '##################################################'

while true
do
    echo `date`
    echo '# Sleeping for 10 minutes'
    sleep 600
    echo '# issuing command - a 10006 error is normal, indicating the process was already running'
    cmd="fmsadmin start fmse"
    echo "$cmd"
    $cmd
    echo ""
    echo ""
done

 

[bruceR]

That's why FMS 14.0v3 was introduced. Looks like you should install that.

[xochi]

That's why FMS 14.0v3 was introduced. Looks like you should install that.

Hey, when did that come out?  Usually when FM announces a new update I get about 5 emails.   I didn't get any notification emails this time for 14v3.   I'll check it out, thanks!

[Josh Ormond]

Emails have been coming out a little slow recently xochi. Not sure why. Whatever they are using to send the emails out is incredibly slow. Or they are batch sending stuff. Who knows?!

[GerthKe]

I had the same problem with Yosemite so I downgraded to Mavericks and everything has been working perfectly ever since. 

[xochi]

Update: After upgrading to FMS14v3, the problem is still happening.  Here's the latest stack trace from the crash last night:

 

Thread 19 Crashed:
0   com.filemaker.support.framework    0x0000000104364ebc std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> >::basic_string(std::basic_string<unsigned short, std::char_traits<unsigned short>, std::allocator<unsigned short> > const&) + 16
1   com.filemaker.dbengine.framework    0x0000000102f5bce7 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 59
2   com.filemaker.dbengine.framework    0x0000000102f5bc99 std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > >::insert(__gnu_cxx::__normal_iterator<Draco::DBStrPair<Draco::DBPluginOpcode>*, std::vector<Draco::DBStrPair<Draco::DBPluginOpcode>, std::allocator<Draco::DBStrPair<Draco::DBPluginOpcode> > > >, Draco::DBStrPair<Draco::DBPluginOpcode> const&) + 111
3   com.filemaker.dbengine.framework    0x0000000102f5b43e Draco::DBStrMap<Draco::DBPluginOpcode>::Add(Draco::unistring const&, Draco::DBPluginOpcode const&) + 144
4   com.filemaker.dbengine.framework    0x0000000102f57bf7 Draco::DBCalcScan::RegisterExternalFunction(Draco::QuadChar const&, short, Draco::unistring const&, Draco::unistring const&, short, short, unsigned int, int (* const&)(short, Draco::DBExpressionEnv const&, Draco::DBDataVect const&, Draco::DBData&), bool) + 209
5   com.filemaker.fmengine.framework    0x000000010382bea6 Draco::InstallDesignFunction() + 250
6   com.filemaker.fmengine.framework    0x0000000103837cde Draco::FMSession::FMSession(Draco::DBAuthenticationInfo const&) + 644
7   com.filemaker.fmengine.framework    0x000000010386da75 Draco::SMSession::SMSession(bool, Draco::DBAuthenticationInfo const&) + 21
8   fmsased                           0x00000001027a4d10 FMScriptThread::RunScript(FMSTypes::ScriptMakerOperation const*, Draco::DBAuthenticationInfo&) + 88
9   fmsased                           0x00000001027a4a2e FMScriptThread::Main() + 378
10  com.filemaker.support.framework    0x000000010438f34f Draco::Thread::RunThread() + 59
11  com.filemaker.support.framework    0x000000010438e3d0 Draco::Thread::ThreadUnixProc(void*) + 54
12  libsystem_pthread.dylib           0x00007fff8c6b205a _pthread_body + 131
13  libsystem_pthread.dylib           0x00007fff8c6b1fd7 _pthread_start + 176
14  libsystem_pthread.dylib           0x00007fff8c6af3ed thread_start + 13

[Wim Decorte]

From the trace, it looks like you are using plugins?  If so, which ones?

[xochi]

I'm using plugins, yes: MailIt from Dacons.   The crash stack doesn't seem to be crashing within the Dacons code, but I do see where calls to Draco::DBpluginOpcode and RegisterExternalFunction are referenced, which sounds like it could be plugin related, doesn't it.  Hmm. 

[Bailey Kessing]

I am having the same issue and use the ScriptMaster plugin. Windows server FM 14v3

Edited October 13, 2015 by Bailey Kessing

[xochi]

It's not crashing very often, to be sure: My setup has an email script which runs every 5 minutes, and it's crashing perhaps every 2 weeks, meaning that on average it runs (60/5 * 24 * 14) about 4000 times before it crashes.     Not easy to debug something that crashes that rarely.    I have looked at the fmsaed process and it doesn't appear to be leaking memory over time or anything obvious like that.

I am having the same issue and use the ScriptMaster plugin. 

I have never used that particular plugin - this is a good clue as it suggests it's not necessarily any specific plugin that's the problem, which seems to put the bug back in FileMaker's court.

Edited October 13, 2015 by xochi
typos

[Bailey Kessing]

And I should have mentioned that we have had to re-installed FM14 twice in the past month bring back up the web components. Restarting was not enough. Something gets corrupted. 

Edited October 15, 2015 by Bailey Kessing