What Is This?

[Michelle Logan]

I keep watching for something to be posted here, but nothing yet.

Can someone tell me what FileMaker 14 Stand by Server Deployment is about, as I have not yet installed FMS 14...it sounds interesting. 

 

Michelle

[Wim Decorte]

It is meant to to help build a "high-availability" FMS deployment where you have a primary server and a standby server.  The standby server can take over when the primary server is not available anymore, in two scenarios:

- the server admin initiates a manual switch-over from primary to the standby server (the standby now becomes the primary)

- automatically when the primary crashes or otherwise becomes unavailable (the standby assumes the role of the primary)

I'm showing all of this at this year's Devcon.

[BrentHedden]

I'm showing all of this at this year's Devcon.

​

Wim, I'm presuming that you've been able to setup this scenario and test it, correct?  I was excited to learn about this with last DevCon, but I haven't been able to test this out yet. As always, I look forward to seeing your presentation(s) this year as well.  I do have a few questions about this, if you've got the time to answer - 

 

- This is an Active/Inactive setup, right?  

- Supposing that the two systems will have different IP addresses, what would you instruct users to do if they are using a launcher file that points to a specific address, and the systems have to be switched over?

- How often does the two systems sync, and how much system & network resources (in general) does this take?  Anything to be concerned with and/or watch out for?

- It looks like the interface for this is only through the command line?  Are there any other setup options besides just turning the feature on and off?

- How can I tell if the standby/backup server is actually being updated, presuming that the setup was made correctly? There doesn't appear to be any kind of feedback/reporting.

[Tyra]

Another really cool feature, at least to me, is that the Stand-by server, can use the same license key!! As stated in this excerpt:

"1. Install FileMaker Server on the prospective standby server. During deployment, enter the user name, organization, and license key information that you used for the primary server.

2. Make sure that the prospective standby server configuration matches your primary server configuration. See “Standby server requirements” on page 82. Even though the prospective standby reports a license conflict, you can use Admin Console to configure the settings. "

[Stephen Hill]

- This is an Active/Inactive setup, right?  

- Supposing that the two systems will have different IP addresses, what would you instruct users to do if they are using a launcher file that points to a specific address, and the systems have to be switched over?

- How often does the two systems sync, and how much system & network resources (in general) does this take?  Anything to be concerned with and/or watch out for?

[...]

I have not setup my test servers yet, but until someone with greater detail jumps in:

* Only one active server, yes.  The second server should only receive interaction from the primary until activated in a failover.

* There are quite a few solutions for handling the failover with IP access. A couple of the ones we've used are below, they're easy to setup but you can take extra steps if you want it to work a bit different.

   - a DNS service that supports failover changes -- when the failure is detected, you'll have to tell the standby to go live and poke the DNS so that it provides the standby IP address instead.  There can be some delays, but a good DNS changeover will get updated addresses out pretty quickly.

   - In a launcher file, you can specify multiple destinations, if the first is inaccessible, Filemaker will drop to accessing the second.  There is a delay as it tries the primary before failing over to the secondary entry, but it's a nice quick solution.  [Manage External Data Sources -> "File Path List (in search order)"]

* I do not have information on the resource usage, but this basically uses the progressive backup system.  The primary should submit progressive updates to the standby, which will apply them to keep the databases extremely close in time.  My guess is that frequency will be related to progressive backup timing.​

 

It sounds like it's just CLI management for the moment -- I'm guessing this will improve as the feature matures, but the CLI control is a fairly useful start for using failover scripting.

[Wim Decorte]

​

Wim, I'm presuming that you've been able to setup this scenario and test it, correct?  I was excited to learn about this with last DevCon, but I haven't been able to test this out yet. As always, I look forward to seeing your presentation(s) this year as well.  I do have a few questions about this, if you've got the time to answer - 

 

- This is an Active/Inactive setup, right?  

 

Yes, it's Active/Passive or "Warm" (not Cold or Hot)​

[Wim Decorte]

 

* I do not have information on the resource usage, but this basically uses the progressive backup system.  The primary should submit progressive updates to the standby, which will apply them to keep the databases extremely close in time.  My guess is that frequency will be related to progressive backup timing.​

 

It sounds like it's just CLI management for the moment -- I'm guessing this will improve as the feature matures, but the CLI control is a fairly useful start for using failover scripting.

​Right on both counts.  The progressive backup timing dictates when the changes get pushed.

Part of my devcon preso will be showing a FM-based UI to keep track of the status and do some of the CLI stuff from a UI.

[Wim Decorte]

 

- Supposing that the two systems will have different IP addresses, what would you instruct users to do if they are using a launcher file that points to a specific address, and the systems have to be switched over?

​When FMP/Go/WebD makes a connection to a file hosted on the Primary, FMS sends info on both servers (primary AND standby) to the client.  So the client's session already knows who to connect to.

When the primary dies you'll see a "communication was lost" dialog, with the option to reconnect.  The reconnect will connect to the standby if the primary is not available.  Or to the primary if it is available.  You can see that dialog in my SolianTV youtube episode on network resilience.

Best regards,

Wim

[Wim Decorte]

​

- How can I tell if the standby/backup server is actually being updated, presuming that the setup was made correctly? There doesn't appear to be any kind of feedback/reporting.

​Yes,  I was surprised at that myself.  Progressive backups are also not reported in the logs and this uses the same mechanism.  There is a CLI command to ask for the status so I ended up writing my own UI to do this periodically.

 

 

[LaRetta]

Hello Wim!

Wow, I've noticed this as well.  Are you willing to share it? :-)

Edited May 21, 2015 by LaRetta

[xochi]

I believe the standby server has to be on the "same" network as the main server - which suggests they'd need to be in the same building, which is not ideal from a disaster-preparation standpoint.  Does anyone know what the actual limitations are?  E.g. could you have them in different physical locations but networked on the same subnet using VPN etc?

[Wim Decorte]

They do have to be on the network so a VPN would work as long as you're not introducing extra latency and such.

The main focus of the feature is high-availability.  Getting clients connected quickly again with minimal data loss.  True DR can be done too but usually does not require the kind of strict uptime SLAs that you can now achieve with this feature.

[Ocean West]

Wim is it correct to state the EAR credentials will need to be entered on the standby server after it has become primary in order to open the databases - and does it need to happen each time or only the first time this machine becomes primary.

 

[Wim Decorte]

Hi OW,  in my testing (pre-release and a good number of weeks back) it was only the first time it became primary.  Provided of course that you also chose to let FMS store the EAR key so that it could open the files up for you.

Two scenarios really:

- fail-over: the primary goes away and the standby becomes a regular standalone server.  There is Primary/Standby anymore.

- switch-over: the primary and standby swap roles.  There still a Primary/Standby setup.  

[BrentHedden]

Wow, I've noticed this as well.  Are you willing to share it? :-)

Wim, if you're willing to share this monitoring utility, it would certainly be a handy thing to have.  Or perhaps explain how it works.  Maybe this can be a bonus thing for attending your DevCon presentation?  

 

Thanks everyone for chiming in on this!

[Wim Decorte]

Maybe this can be a bonus thing for attending your DevCon presentation?  

 

​That's the idea! 

Time permitting I could run it by the folks here before devcon and you can all help me improve it...

[BrentHedden]

​That's the idea! 

Time permitting I could run it by the folks here before devcon and you can all help me improve it...

​Now that I remember it, at my company we setup something very similar to this.  It checks to see if all of the FM services we are using are running at that moment in time.  If not, then send an email out.  It's dumping all of the results in a log file.  You can also check the log inside of FileMaker, which is simply reading and displaying the log file.  

 

I don't have any issue with sharing this technique if it'll help you and the presentation out.....

[Wim Decorte]

Thanks, Brent.  Can't hurt to have a look at it.  I'm working on my demo file over the next two weeks (got to be ready for Devcon :))

 

 

[BrentHedden]

Can't hurt to have a look at it.

 

​Written for Mac, but the same can be done through Powershell or plain command line for PC.  Cron executes this script once every 10 minutes.  PCs can use the Task Scheduler.  All of the variables are declared and set at the beginning.  Sends an email if the service is not running. Appends the results to a log file.  You can adjust what services are to be checked by adjusting the SERVICES variable.  

 

#!/usr/bin/env bash
# ------------------------------------------------------------------
# FileMaker Server Notifier
# ------------------------------------------------------------------
# 
# Config
NAME="FileMaker Server Notifier"
VERSION="0.0.1"
SERVICES=("ADMINSERVER" "SERVER" "FMSE" "FMSIB" "XDBC" "WPE")
LOG_PATH="/Library/Logs/fmserver_notifier.log"
FMSADMIN_BIN="/usr/bin/fmsadmin"
MAIL_TO="[email protected]"

CHAR_ERROR="\0342\0234\0226"
CHAR_SUCCESS="\0342\0234\0223"
INDENT="  "

# Functions
function log () {
   echo -e "$1" >> "$LOG_PATH"
}

function checkService () {

    echo "Checking $1..."
    RESULTS=$($FMSADMIN_BIN start "$1")
    if [[ -z "$RESULTS" ]]
    then
    RESULT_TEXT="$INDENT$CHAR_ERROR $1 had to be started"
    echo -e "$1 was restarted!" | mail -s "$NAME Error: $1" "$MAIL_TO"
    else
        RESULT_TEXT="$INDENT$CHAR_SUCCESS $1 is running."
    fi
    echo -e "$RESULT_TEXT"
    log "$RESULT_TEXT"
}

# Check and start services
log "[[ Starting => $(date) =============================="

for service in ${SERVICES[*]}
do
    checkService "$service"
done

log "]] Finished <= $(date) =============================="
log ""