RAID

[tv_kid]

A client wants to set up a RAID disk on their server which is running FM Server 7. Is this a bad idea given the caveats about backing up live files?

Thanks

[Ender]

A RAID is a great idea, but Server will perform best if the boot drive is not the data drive. So use the RAID as the data drive, keeping the internal drive for the boot drive.

As for backups, use Server's scheduled backup capability to save regular backups to either drive (I'd suggest backing up to the internal drive, so that if the RAID fails, you can still get up and running while the RAID is being rebuilt.)

[xochi]

1. A great RAID tutorial is here: http://www.acnc.com/raid.html Remember that some RAID levels are for performance (and can actually reduce reliability), and some RAID levels are for reliability.

2. Typical hardware RAID is completely transparent to all applications / server processes, so the copy of all data on both drives (if you are using RAID 1) is perfectly up-to-date at all times.

3. RAID is not a panacea -- although hard drives fail, other things can go wrong too. The HD catalog can become corrupted, a user can accidentally delete the files, a database can become corrupted for other reasons, etc. If something like this happens on a RAID 1 system, then you are hosed, because the changes will be made simultaneously to both drives!

4. Moral of the story: RAID is good, but you also must have a sensible backup strategy as well, so that you have the ability to go back in time and use a prior version of the database if something goes drastically wrong.

[tv_kid]

Thank you Ender and Xochi for your speedy and useful advice.

I have since found out more, the client is going to use 'Super Duper!' software to back up their machine Link

I've seen it in use, and although it was billed as a 'mirror', all it appears to be doing is copying the entire hard disk at a specified time in the day. As far as I'm aware this isn't true mirroring, as the backup disk could be up to 23hrs 59 mins out of date. Has anyone come across this software?

[Ender]

Well, SuperDuper! is great as a disk image maker/backup scheduler, and it can save those images/backups to whatever drive you wish. But it certainly wouldn't help with performance or hard drive redundance like a RAID. And you can't use it to backup live (hosted) databases; you'd still need Server's scheduled backups to do that.

You should think of SuperDuper! as more of a secondary backup/archive tool, where it creates a regular archive of the backup folder. Although, you might just as well create a shell script or cron job that does this.

[tv_kid]

That's what I thought, and as the client already uses Retrospect (although it's a pain in the rear end) for automated backups, I think they've been mis-sold this 'Super Duper!' software.

You mentioned the shell script option, which brings up another issue I was going to post about. I've attempted to initiate the shell script as descibed in Wim Decorte's "Backups" white paper Link . (The OSX version).

When I set up the script to run via an FM Server schedule, I just ended up with a 4KB zip file that wouldn't unzip. Obviously it had no data, but I'm not sure what I'm doing wrong, and have no experience with the command line. I've made sure that the script is in the scripts folder of FM Server, and that this script and the destination folder specified in it have the correct file permissions.

Can you shed any light on why this might not be working?

Thanks

[xochi]

Be careful with shell scripts! A mis-placed space or quote mark can be quite disastrous. Not for the faint of heart.

That being said, here are some ways to debug your shell script:

1. cd to the folder it is in, and then execute it directly, and watch for error messages:

  cd /Library/FileMaker Server/Data/Scripts

  ./theNameOfMyScript.sh

This should reveal any gross errors.      If this works w/o fail, but doesn't work when run via FileMakerServer's schedules, then the bug is probably more subtle.   Look for permissions issues (which you may have done?)  -- make sure that any file read or written by the script has proper permissions for the FMSERVERD process.



2. If step 1 fails, then you should be able to deduce the error from the message(s).   If not, you might try executing the script one line at a time:   Create a new terminal window (so all your environment variables are clean).   Open your shell script in your editor (BBEdit, right?) then cut & paste one line at a time to the shell window and watch for errors.   Also, look at any files created and see if there are any problems.



3. Instead of running your commands, try just echoing them to the screen. e.g.

echo cp $root/file1 $root/file2

instead of 

cp $root/file1 $root/file2

This may help you see problems with environment variables, quoting problems, etc.

4. Don't forget to check Console.log and System.log (Mac OS X) and Events.log (/Library/FileMaker Server/Data/Logs) for error messages

[xochi]

RE: SuperDuper.

SuperDuper is a great tool to create a "snapshot" of a drive that can be a bootable drive. However, it's really just a copy tool, so if any files are open and/or in use when SuperDuper is copying them (such as FileMaker database files), they will not be usable.

I personally think SuperDuper is great -- I use it periodically to make a 4th backup copy of my boot drive (in addition to three other copies of the filemaker stuff). But, I make sure that I shut down all filemaker processed and filesharing processes before I run SuperDuper.

[xochi]

When I set up the script to run via an FM Server schedule, I just ended up with a 4KB zip file that wouldn't unzip. Obviously it had no data, but I'm not sure what I'm doing wrong, and have no experience with the command line. I've made sure that the script is in the scripts folder of FM Server, and that this script and the destination folder specified in it have the correct file permissions.

Can you shed any light on why this might not be working?

Thanks

Can you post a copy of your modified shell script here? Put it in between [ Code ] and [ / code ] tags so the formatting sticks?

[tv_kid]

1. cd to the folder it is in, and then execute it directly, and watch for error messages:

  cd /Library/FileMaker Server/Data/Scripts

  ./theNameOfMyScript.sh

This should reveal any gross errors.      If this works w/o fail, but doesn't work when run via FileMakerServer's schedules, then the bug is probably more subtle.   Look for permissions issues (which you may have done?)  -- make sure that any file read or written by the script has proper permissions for the FMSERVERD process.

Thanks for your advice. I tried the above, but just got a "Permission Denied" message. Is this because I am trying to run a script that I am not the owner of? I have checked that the files (the backup files and the shell script) and the folders they reside in all have the Owner as FMServer and Group as FMSAdmin. But still no luck. Here is the script:

I'd be grateful for any advice you can give me.

#!/bin/bash



# 5 Dec 2005 Rob Russell SumWare Consulting



# only change the next two lines of this script to suit your installation



source="/Library/FileMaker Server 7/Data/Backups/hourly/"

destination="/Library/Filemaker Server 7/Data/Backups/Compressed_Hourly_Backups/"



# source defines the folder you want to archive. Only archive 

# FileMaker Server backups, not the master files hosted by FMServer.



# destination is the place you want the archives sent

# ( best this be a separate volume )



# suffix is the time stamp to append to the destination



suffix="$(date +%Y%m%d%H%M%S).zip"



# create symlink files, in the tmp folder, to deal with file paths with spaces



sourcelink="/tmp/sumware001.ln"



ln -s "$source" $sourcelink



destinationlink="/tmp/sumware002.ln"



ln -s "$destination$suffix" $destinationlink



# copy with pkzip compression and preserve resources



ditto -c -k --rsrc $sourcelink $destinationlink



# clean up the symlinks we created earlier



rm -f $sourcelink

rm -f $destinationlink 

[xochi]

Thanks for your advice. I tried the above, but just got a "Permission Denied" message. Is this because I am trying to run a script that I am not the owner of? I have checked that the files (the backup files and the shell script) and the folders they reside in all have the Owner as FMServer and Group as FMSAdmin. But still no luck.

Sounds like permissions problems.

do this:

 cd /Library/FileMaker Server/Data/Scripts

ls -lag

Your script should show the following permissions for the script:

-rwxrwxr-x   1 fmsadmin (blah blah blah...)

if the x bits are not set, you need to set them

sudo chmod 775 myScriptFileName.sh

[tv_kid]

Sounds like permissions problems.

I checked using the code you specified, and indeed the x bits weren't set. I changed them and ran the script again from Filemaker Server, no luck. I then ran it directly from the Terminal application and got the following message:

ditto: /tmp/sumware002.ln: No such file or directory 

Does this make any sense?



I also tried your suggestion of running individual lines one by one, but none of them worked.  (I assume that the ones prefixed by # are comments only).



For instance runnning:

 source="/Library/FileMaker Server 7/Data/Backups/hourly/"

just gave the error:

tcsh: source=/Library/FileMaker Server 7/Data/Backups/hourly/;) command not found

By the way, I tried adding in a backslash after 'Filemaker' and 'Server' in the above file path, but this didn't seem to help.

If you have anymore suggestions, I'd be grateful

Thanks

STOP PRESS

I've now got it working. I decided to go back to the original script and not change anything! I set up a 'backups' folder in the Users/Shared directory and this works ok. The error seems to have been trying to save the compressed file in the Library/Filemaker Server 7/Data/Backups folder. Don't know why though? Still, at least it's working now. Thanks for your help.

Edited March 31, 2006 by Guest

[xochi]

Another thing to watch out for there is more than one "shell".

The first line of your shell script

#!/bin/bash

Tells the system which shell to execute the script in. Fun, huh?

Turns out that this may or may not be the same shell that your Terminal window is using, and it may or may not be the default shell that a daemon (like fmserverd) would be using.

And yes, the different shells have different syntax...

I think under Mac OS X 10.4, the default shell is indeed 'bash', so it looks like you are ok there...

[xochi]

For instance runnning:

 source="/Library/FileMaker Server 7/Data/Backups/hourly/"

just gave the error:

tcsh: source=/Library/FileMaker Server 7/Data/Backups/hourly/;) command not found

I take back my earlier comment -- looks like your default shell is 'tsch' instead of 'bash', which is why the one-line-at-a-time method ain't working. Are you on an old version of Mac OS X?

You can change your default shell via Terminal/Preferences.

[tv_kid]

[

I take back my earlier comment -- looks like your default shell is 'tsch' instead of 'bash', which is why the one-line-at-a-time method ain't working. Are you on an old version of Mac OS X?

You can change your default shell via Terminal/Preferences.

The machine is running OSX Server 10.3.9 As far as I know that is the default shell (I've not changed it). I'm just pleased to have got it going, and thanks once again for your advice, without which I would have given up.

[Fenton]

I (also ) have an eMac running 10.3.9 (client, not server). I just tried:

echo $SHELL

returned: /bin/bash

And that's all I know -]

[tv_kid]

Just reviewing your post:

A RAID is a great idea, but Server will perform best if the boot drive is not the data drive. So use the RAID as the data drive, keeping the internal drive for the boot drive.

I'm a bit confused. I thought that the 'usual' RAID configuration was a 'mirror', i.e having the same thing on both drives. How then could there be different information on the 'boot' and 'data' drives? Is the data drive merely a seperate drive that does not have the system software and is not 'bootable'? If so then surely it becomes more complicated to create a RAID setup from these two sources?

I don't really understand why having the data files on a seperate hard disk is a good idea, won't it take FM Server longer to locate them over firewire or other external interface?

Thanks in advance

[Ender]

Well, a RAID isn't necessarily a mirror of two drives. There are different levels depending on which you need to achieve: redundancy, performance, or both.

Since performance is a primary concern with the data drive, striping may be used instead of, or in addition to mirroring.

But in any case, booting the system off the same drive as the data creates more work and adds delays to that drive. Remember, next to the network, the hard drive is the slowest part of the process, so giving extra work to the data drive should be avoided.

I don't know what the performance of Firewire is like for database work, but I've been told that the SATA drives in PowerMac G5's are not that great because the bus is asynchronous, not allowing data transfer in both directions at once.

[tv_kid]

I originally posted this some time ago, but am having similar problems when trying to use the script on another clients server.

I cannot get the script to run from FM Server. After reading the earlier posts again, I have tried running it from the Terminal, but this did not work. I then tried performing the steps line by line, and this did work. I checked the permissions as described, and set the 'x' bits. I still cannot get it to run either from FM Server, or as a whole from the Terminal. I can run it line by line from the Terminal however.

In this case the shell appears to be 'bash' (?)

Can anyone throw any light on this or suggest an alternative solution?

Thanks in advance.

[Steven H. Blackwell]

On all these activities be sure that you do not copy any files while they are opened for hosting. This will damage them. Copy only closed files.

Steven

[tv_kid]

Hi Steven

Thanks for the advice. The script is only working on backups of the files that FM Server has created.