Is MirrorSync appropriate to help solve our problem?

[Doug Lerner]

Our FMS 15 is running on a dedicated server at a hosting provider (running on Windows 2008 R2). We have two databases - a small 30 MB "UI" front end for accessing via FM Pro clients and a larger 7 GB database containing all the data.

 

The only access to our databases are either (1) via FM Pro clients or (2) via WPE calls made from another non-FM server to retrieve data and add/update records.

 

We are having a problem keeping the WPE up during times when staff are also doing other tasks on their computers via their FM Pro clients - in particular, when they are uploading spreadsheets of data which trigger scripts to process the data. Symptoms are the WPE just turns itself off in the admin console. Or even if it's on the WPE  goes unresponsive and needs to be toggled off an on again. 

 

We are working with FM tech support now to see why, because they admit that running scripts like this should not cause the WPE to stop, and they are pouring through logs for us now. Our server meets all their requirements and has plenty of available RAM, etc. But it's like FMS can't walk and chew gum at the same time.

 

When the staff are not updating things with spreadsheets, the WPE, if it times out at all, it might be 1 timeout out of 1,000 WPE calls. Overnight last night the WPE ran fine for about 12 hours straight. But when the staff came in and uploaded a few spreadsheets the WPE came down again. It came down 4 times today.

 

So... I'm wondering if MirrorSync can help with this problem.

 

Looking at the price list, it seems MirrorSync has a free and $400 solution that I think, from what I read, we could use in the following manner (that's what I want to confirm and why I'm posting today):

 

• We continue to run just one FileMaker server. But the staff don't connect to the remote server with their FM clients to the server for the update work they are doing. 
• Instead, each staff member who needs to do updates of the time-consuming spreadsheets opens up a local copy of the database and does their work on their own Mac or Windows. No server is required at this point, and the scripts run offline on their computer.
• Then when done, MirrorSync will sync their changes into the database on the server.
• Staff can still connect to the FileMaker server if needed. But would just avoid doing the problematic updates of spreadsheets directly on the server. They would do them locally and then sync.

 

Does this kind of work flow sound correct in terms of what MirrorSync does?

 

If so, how do the local FM Pro clients also stay synced with changes being made in the database via the WPE during this time? The WPE calls might add or update records in different layouts. Is there some process where a staff member would first sync with the server to get the updated data, then do their own data updates and reports offline, and then sync again with the server to send the data back online? Or is the work flow different from that?

 

Thanks.

 

Doug

[Jesse Barnum]

MirrorSync can do what you want - you could have offline copies for your users to run their reports, and every time they run the sync script, it will push all changes to server and pull all changes from server. If the WPE makes changes, then presumably that will cause the modification timestamp to be updated, which will be picked up by MirrorSync on the next sync.

From a pricing standpoint, the first device and first sync configuration are free. Assuming you only need one sync configuration, that means you'll just be paying for devices beyond the first. Prices start at $95/device for 1 and go as low as $40/device for 100, you can find out more by going to http://mirrorsync.com and clicking the 'pricing / buy' button.

With that said, it's not how I would solve the problem. Can you switch to using xDBC instead of WPE? xDBC can do everything WPE does except for running FileMaker scripts. It's faster and more reliable.

I would also update to FMS 16 and Windows Server 2012r2 or later, installed from scratch on a brand new box, to see if that helps with stability.

[Doug Lerner]

Thanks for your reply. I see you are with 360Works. Before posting to these forums I also sent two queries via your contact form but didn't get a response, which is why I posted here. Maybe it's better to post here.

Can I ask a few followup questions? I have to admit I am not a FileMaker professional by any means. I studied it a bit, and wrote my own simple solution for my own small company to keep track of servers, customer, contacts and relate them all to each other. We have a FM developer who created the current problematic solution for a client of mine. So I'm not familiar with all the ins and outs of FM and have been diving in to help my client, and have just recently gotten more familiar with things like the Windows Task Manager (I'm a Mac person), the server manager, the fmsadmin command in the terminal and more into the admin console and log.

My questions are:

1. You says, if the WPE makes changes, then presumably that will cause the modification timestamp to be updated, which will be picked up by MirrorSync on the next sync.

OK. First it's good to know that there are "modification timestamps." I didn't know that and was wondering how MirrorSync decided something needed to be synced and how it synced so quickly. When you say you presume the WPE causes the modification timestamp to be updated is that something done automatically by the WPE or is that something I need to confirm with our FM developer that he is actually doing? Also, how does MirrorSync quickly find all the modification timestamps on both sides so it knows what to sync? The database is about 7 GB in size and, for example, in one table (member records) there are over 250,000 records. I'm guessing you have a method other than just looping through all the records one-by-one, right? 

2. If we did this, to get started could we download the latest daily backup and run it locally and then first sync to the current database on the server? Would the work flow be (1) sync with the FMS; (2) do our local work; (3) sync to the server again?

3. What happens with regard to deleted records on either side?

4. With regards to the pricing, it seems to go in two steps from $0 to $400 for a total of 6 devices, I believe. That's fine. I imagine I would use the 1 device to first test (unless there was a trial period for the 6 devices). The 6 devices fits perfectly into our small office situation where only a handful of people are using FM Pro clients right now. One question about this though - is each device registered and unique, or is it "devices able to simultaneously connect and sync"? So, for example, if I test on my machine before going ahead an purchasing, and then would like to pause and have a staff member test on her machine, would that then and there require another license, or if I've paused and am not using it could she use my license for her set of tests?

5. You say you would not solve the problem that way. Is that because you consider the WPE unstable? Some people tell us (now) that they think the WPE is unstable. Some people say it is stable. FM itself is unclear on this point.  FM technical support is actively looking into why certain FM Pro client-side activities, which trigger server-side scripts to run (like uploading lists of webinar attendees) cause the WPE to fail. Our FM developer says they are unrelated to the WPE and theoretically should not impact on the WPE at all because it's different threads. FM admits the scripts shouldn't bring the WPE down if if they are time-consuming, but they do. In answer to your question about xDBC though, I don't know. I don't know what xDBC is and what is required to substitute out our WPE calls for xDBC calls. I just Googled it, but don't see a simple answer to your question. Our social network server which is making the WPE calls to FMS is not PHP based, and is a non-SQL object oriented database system that is very non-standard, and it runs on a Linux server. It can make different kinds of calls to external servers, like XMLRPC, REST, and GET calls, which we are using for the WPE. We've spent a year developing these WPE calls, and the way we parse the returned XML, and how it all interacts with our system and there are a couple of dozen of these calls made. I don't know anything about xDBC so I can't say if it is practical to substitute out the WPE calls with xDBC calls. My guess is we're currently stuck with using the WPE for now without a major rewrite. If xDBC is just a different kind of format and we can easily substitute the calls we currently make with xDBC formatted calls then maybe. But if it requires some standard LAMP architecture, PHP libraries, etc.,  or requires Windows, we can't, because our calling server is not that architecture. 

4. This new Windows 2008 R2 server was actually recently installed from scratch. That's what our hosting service (a FM partner solution) provided us with. I don't know why they went that way instead of with Windows 2012 R2. I have to admit I was also surprised when I saw the date. We could ask them to upgrade to FMS 16 at some point, but not at the moment, because we tested and we have to upgrade our own server first because we are, unfortunately, still making TLS 1.0 outgoing calls and FMS 16 requires TLS 1.2 for https connections. We don't know if it's possible to "downgrade" the IIS used by FMS 16 to allow TLS 1.0 connections instead. I tried modifying the IIS configuration files where it appeared that was set, but it didn't help. So we stuck with FMS 15 for now. Of course we should upgrade our own server's OpenSSL to let it make TLS 1.2 connections, and we're working on that. If it was possible to allow FMS 16 to make TLS 1.0 connections, which FMS 15 does, we would try that upgrade. Interestingly enough, upgrading to FMS 16 or upgrading the Windows server version is not something suggested by FM technical support yet, which is also working on this problem.

Right now we urgently just need some way to have a couple of staff members do these problematic updates without bringing the WPE down. Even if it's not the permanent solution, we just need some way of their work uploading spreadsheets and triggering the problematic scripts to be done completely off the FMS so the WPE can keep running. Even if it's just a temporary solution while we sort everything else. When those scripts aren't running on the FMS the WPE for the most part runs ok, with less than 1 in 1,000 calls timing out. 

I really do greatly appreciate your feedback! I hope you can clarify these points.

Thanks,

doug

[Jesse Barnum]

Hi Doug - that is a lot of questions to type a reply to :-)

Might be better to call us at 770-234-9293.

[Doug Lerner]

5 hours ago, Jesse Barnum said:

Hi Doug - that is a lot of questions to type a reply to :-)

Might be better to call us at 770-234-9293.

I just saw your message (I am in Japan) and tried calling but apparently missed your office hours by about 30 seconds.

If there is no reply I'll try reaching you again by phone, but you don't open until midnight here.

doug