Jump to content
Doug Lerner

Is MirrorSync appropriate to help solve our problem?

Recommended Posts

Our FMS 15 is running on a dedicated server at a hosting provider (running on Windows 2008 R2). We have two databases - a small 30 MB "UI" front end for accessing via FM Pro clients and a larger 7 GB database containing all the data.
 
The only access to our databases are either (1) via FM Pro clients or (2) via WPE calls made from another non-FM server to retrieve data and add/update records.
 
We are having a problem keeping the WPE up during times when staff are also doing other tasks on their computers via their FM Pro clients - in particular, when they are uploading spreadsheets of data which trigger scripts to process the data. Symptoms are the WPE just turns itself off in the admin console. Or even if it's on the WPE  goes unresponsive and needs to be toggled off an on again. 
 
We are working with FM tech support now to see why, because they admit that running scripts like this should not cause the WPE to stop, and they are pouring through logs for us now. Our server meets all their requirements and has plenty of available RAM, etc. But it's like FMS can't walk and chew gum at the same time.
 
When the staff are not updating things with spreadsheets, the WPE, if it times out at all, it might be 1 timeout out of 1,000 WPE calls. Overnight last night the WPE ran fine for about 12 hours straight. But when the staff came in and uploaded a few spreadsheets the WPE came down again. It came down 4 times today.
 
So... I'm wondering if MirrorSync can help with this problem.
 
Looking at the price list, it seems MirrorSync has a free and $400 solution that I think, from what I read, we could use in the following manner (that's what I want to confirm and why I'm posting today):
 
  • We continue to run just one FileMaker server. But the staff don't connect to the remote server with their FM clients to the server for the update work they are doing. 
  • Instead, each staff member who needs to do updates of the time-consuming spreadsheets opens up a local copy of the database and does their work on their own Mac or Windows. No server is required at this point, and the scripts run offline on their computer.
  • Then when done, MirrorSync will sync their changes into the database on the server.
  • Staff can still connect to the FileMaker server if needed. But would just avoid doing the problematic updates of spreadsheets directly on the server. They would do them locally and then sync.
 
Does this kind of work flow sound correct in terms of what MirrorSync does?
 
If so, how do the local FM Pro clients also stay synced with changes being made in the database via the WPE during this time? The WPE calls might add or update records in different layouts. Is there some process where a staff member would first sync with the server to get the updated data, then do their own data updates and reports offline, and then sync again with the server to send the data back online? Or is the work flow different from that?
 
Thanks.
 
Doug

Share this post


Link to post
Share on other sites

MirrorSync can do what you want - you could have offline copies for your users to run their reports, and every time they run the sync script, it will push all changes to server and pull all changes from server. If the WPE makes changes, then presumably that will cause the modification timestamp to be updated, which will be picked up by MirrorSync on the next sync.

From a pricing standpoint, the first device and first sync configuration are free. Assuming you only need one sync configuration, that means you'll just be paying for devices beyond the first. Prices start at $95/device for 1 and go as low as $40/device for 100, you can find out more by going to http://mirrorsync.com and clicking the 'pricing / buy' button.

With that said, it's not how I would solve the problem. Can you switch to using xDBC instead of WPE? xDBC can do everything WPE does except for running FileMaker scripts. It's faster and more reliable.

I would also update to FMS 16 and Windows Server 2012r2 or later, installed from scratch on a brand new box, to see if that helps with stability.

Share this post


Link to post
Share on other sites

Thanks for your reply. I see you are with 360Works. Before posting to these forums I also sent two queries via your contact form but didn't get a response, which is why I posted here. Maybe it's better to post here.

Can I ask a few followup questions? I have to admit I am not a FileMaker professional by any means. I studied it a bit, and wrote my own simple solution for my own small company to keep track of servers, customer, contacts and relate them all to each other. We have a FM developer who created the current problematic solution for a client of mine. So I'm not familiar with all the ins and outs of FM and have been diving in to help my client, and have just recently gotten more familiar with things like the Windows Task Manager (I'm a Mac person), the server manager, the fmsadmin command in the terminal and more into the admin console and log.

My questions are:

1. You says, if the WPE makes changes, then presumably that will cause the modification timestamp to be updated, which will be picked up by MirrorSync on the next sync.

OK. First it's good to know that there are "modification timestamps." I didn't know that and was wondering how MirrorSync decided something needed to be synced and how it synced so quickly. When you say you presume the WPE causes the modification timestamp to be updated is that something done automatically by the WPE or is that something I need to confirm with our FM developer that he is actually doing? Also, how does MirrorSync quickly find all the modification timestamps on both sides so it knows what to sync? The database is about 7 GB in size and, for example, in one table (member records) there are over 250,000 records. I'm guessing you have a method other than just looping through all the records one-by-one, right? 

2. If we did this, to get started could we download the latest daily backup and run it locally and then first sync to the current database on the server? Would the work flow be (1) sync with the FMS; (2) do our local work; (3) sync to the server again?

3. What happens with regard to deleted records on either side?

4. With regards to the pricing, it seems to go in two steps from $0 to $400 for a total of 6 devices, I believe. That's fine. I imagine I would use the 1 device to first test (unless there was a trial period for the 6 devices). The 6 devices fits perfectly into our small office situation where only a handful of people are using FM Pro clients right now. One question about this though - is each device registered and unique, or is it "devices able to simultaneously connect and sync"? So, for example, if I test on my machine before going ahead an purchasing, and then would like to pause and have a staff member test on her machine, would that then and there require another license, or if I've paused and am not using it could she use my license for her set of tests?

5. You say you would not solve the problem that way. Is that because you consider the WPE unstable? Some people tell us (now) that they think the WPE is unstable. Some people say it is stable. FM itself is unclear on this point.  FM technical support is actively looking into why certain FM Pro client-side activities, which trigger server-side scripts to run (like uploading lists of webinar attendees) cause the WPE to fail. Our FM developer says they are unrelated to the WPE and theoretically should not impact on the WPE at all because it's different threads. FM admits the scripts shouldn't bring the WPE down if if they are time-consuming, but they do. In answer to your question about xDBC though, I don't know. I don't know what xDBC is and what is required to substitute out our WPE calls for xDBC calls. I just Googled it, but don't see a simple answer to your question. Our social network server which is making the WPE calls to FMS is not PHP based, and is a non-SQL object oriented database system that is very non-standard, and it runs on a Linux server. It can make different kinds of calls to external servers, like XMLRPC, REST, and GET calls, which we are using for the WPE. We've spent a year developing these WPE calls, and the way we parse the returned XML, and how it all interacts with our system and there are a couple of dozen of these calls made. I don't know anything about xDBC so I can't say if it is practical to substitute out the WPE calls with xDBC calls. My guess is we're currently stuck with using the WPE for now without a major rewrite. If xDBC is just a different kind of format and we can easily substitute the calls we currently make with xDBC formatted calls then maybe. But if it requires some standard LAMP architecture, PHP libraries, etc.,  or requires Windows, we can't, because our calling server is not that architecture. 

4. This new Windows 2008 R2 server was actually recently installed from scratch. That's what our hosting service (a FM partner solution) provided us with. I don't know why they went that way instead of with Windows 2012 R2. I have to admit I was also surprised when I saw the date. We could ask them to upgrade to FMS 16 at some point, but not at the moment, because we tested and we have to upgrade our own server first because we are, unfortunately, still making TLS 1.0 outgoing calls and FMS 16 requires TLS 1.2 for https connections. We don't know if it's possible to "downgrade" the IIS used by FMS 16 to allow TLS 1.0 connections instead. I tried modifying the IIS configuration files where it appeared that was set, but it didn't help. So we stuck with FMS 15 for now. Of course we should upgrade our own server's OpenSSL to let it make TLS 1.2 connections, and we're working on that. If it was possible to allow FMS 16 to make TLS 1.0 connections, which FMS 15 does, we would try that upgrade. Interestingly enough, upgrading to FMS 16 or upgrading the Windows server version is not something suggested by FM technical support yet, which is also working on this problem.

Right now we urgently just need some way to have a couple of staff members do these problematic updates without bringing the WPE down. Even if it's not the permanent solution, we just need some way of their work uploading spreadsheets and triggering the problematic scripts to be done completely off the FMS so the WPE can keep running. Even if it's just a temporary solution while we sort everything else. When those scripts aren't running on the FMS the WPE for the most part runs ok, with less than 1 in 1,000 calls timing out. 

I really do greatly appreciate your feedback! I hope you can clarify these points.

Thanks,

doug

Share this post


Link to post
Share on other sites

Hi Doug - that is a lot of questions to type a reply to :-)

Might be better to call us at 770-234-9293.

Share this post


Link to post
Share on other sites
5 hours ago, Jesse Barnum said:

Hi Doug - that is a lot of questions to type a reply to :-)

Might be better to call us at 770-234-9293.

I just saw your message (I am in Japan) and tried calling but apparently missed your office hours by about 30 seconds. :(

If there is no reply I'll try reaching you again by phone, but you don't open until midnight here.

doug

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By amallison
      Since going from version 2 to 3 of MirrorSync last year we have constantly had MirrorSync crashing the Web Publishing Engine. 
      Sometimes it says "Error from server: FileMaker XML WPE Web Publishing Engine is not running at <the hosted IP>. Ensure that the WPE is working correctly, and then try this operation again."
      But usually it is an Error 10 error. "Error 10 while connecting to http://<the hosted IP>/sync?ping=1&config..."
      Early on we made the documented suggested changes for this error, to the connectionTimeout Property in the IIS Manager application. With no noticeable effect.
      Most of the time MirrorSync works as expected and does a great job and then clients can't sync and all hell breaks loose.
      Do you have any other suggestions for what might be causing this issue?
    • By David Hollander
      I successfully deployed SuperContainer (SC) via the Web Publishing Engine (WPE) of FileMaker Server 11 for years, on a Mac mini (2010) running OS 10.6.8 (Snow Leopard), with no problems. But I ran into the following problem with Mavericks and SC:   I upgraded to more modern versions of OS X and FMS by doing a completely clean installation: Formatted the Mac mini's drive; Did a clean installation of OS X 10.9.1 (Mavericks); and Installed FileMaker Server 12v6 without problems. Next, I downloaded the latest SuperContainer from 360Works (v2.896). When I double-clicked the "Mac Installer.app" (also known as "SCInstaller") to deploy SC with FMS, as I did before with FMS11, I was surprised to see this alert pop up:   "'Mac installer' is damaged and can't be opened. You should move it to the Trash." Mac Installer is damaged.tiff   I tried downloading a fresh copy, using Safari and Firefox; same error. I’d never seen this message before when installing any software, and I found no reports of it in this forum or in TechNet.   The icon displayed in the alert window indicated that it was originating from Mac OS Gatekeeper. So the solution I tried was to change the "Security & Privacy" System Preference of Mavericks, under the "General" tab, to "Allow apps downloaded from Anywhere", rather than the more restrictive "Mac App Store and identified developers" option. (Perhaps another option would have been just to control-click the app icon and then select “Open”, to exempt the app from Gatekeeper; I didn't test that.)   Apple’s support article on Gatekeeper mentions this “Damaged” app warning message is an indication that “the app has been altered by something other than the developer.”   I doubt that any “true damage” is present in the Mac Installer for SC. I think the message appeared because AppleScript scripts get modified after they’re signed. An Apple support article on Mavericks security states: “Some apps and tools, such as AppleScript applications and some legacy tools, modify themselves after signing. These types of apps cannot be opened unless the Gatekeeper setting in Security & Privacy preferences is set to Anywhere”   I had only tried the SC Installer on Mavericks, but I suppose the same error might have occurred on Mountain Lion or Lion, since Gatekeeper was introduced with Lion. It appears, for now, that we may need to temporarily disable Gatekeeper (by choosing the "Anywhere" option) in order for SC's Mac Installer to work.   David M. Hollander, MD Combined Fields Consulting
    • By wimpog
      According to the following KB article, FMS12 isn't compatible with 10.9:
      http://help.filemaker.com/app/answers/detail/a_id/11930
       
      However, it is possible to get it to work with a few adjustments. The database server works just fine, but WPE does not.
      Here is how to get WPE working (based on http://fmforums.com/forum/topic/89802-109-mavericks/#6 with a few corrections)
       
      Open up Terminal
      cd /Library/Server/Web/Config/apache2/ Add the following lines to the end of the httpd_server_app.conf file:
      Note: they may already be there, commented out, after installing the FileMaker Server
      <IfDefine WEBSERVICE_ON> Include '/Library/FileMaker Server/Admin/admin-helper/WEB-INF/conf/fmi-test.conf' Include '/Library/FileMaker Server/Admin/admin-helper/WEB-INF/conf/fm-server-status.conf' LoadModule proxy_ajp_module libexec/apache2/mod_proxy_ajp.so Include '/Library/FileMaker Server/Admin/admin-helper/WEB-INF/conf/mod_proxy.conf' </IfDefine> Some of the modules may be commented out with
      #FMI_Configuration_V1 which you will need to uncomment.
       
      Restart your Web server
      Restart the FileMaker Server
      Restart the Web Publishing Engine
      Go to http://<your_server_url>:16000 and perform FileMaker Server 12 Technology Tests in the Troubleshooting section.
       
      Tested 10.9, 10.9.1, FMS 12.0v4. Not tested with 12.0v5, but it should work as well.
  • Who Viewed the Topic

    9 members have viewed this topic:
    Peter Jambors  thedbpro  fmsavey  doughemi  Lee Smith  bcooney  siroos12  MonkeybreadSoftware  Ocean West 
×

Important Information

By using this site, you agree to our Terms of Use.