mac FMS's Progressive Backup questions

[Justin Close]

We are using FMS 12.03 on Mac OS X Server 10.6.8.

 

I had some questions about using progressive backups that doesn't appear to be too closely addressed:  in respect to Progressive Backups, how quickly do you have to realize, and respond, to a problem with you DB? 
    As I understand it, you really only have (2 * Interval) minutes to respond to a problem with your database.  If you wait longer than that to grab a copy of the backup file, then you will get a copy that contains the bad data or corrupted file structure.
   

Perhaps a side benefit to this:  Does Progressive backup BETTER protect you against file corruption, since it is (or sounds like it is) recreating changes made to the Live file instead of copying the Live file itself?  If you are working under the hood on the live file (yeah, I know, you aren't supposed to do Manage Database work in the live file) and something goes wrong (FM client crashes) that might corrupt the Live file, even if the data gets copied to the Progressive file at time of the interval, would it result in the same corruption?  Or, since it is a different file, would it just contain valid updates, and perhaps skip/ignore the transaction that was happening when it crashed?   If the system is using a 'log of changes' to make the changes to the Progressive backups, how does it handle a 'change' that was interrupted by the crashing of the client?

 

Doesn't this imply a few things about the interval that is chosen?  If the interval is really short then you will have a very short time to respond to problems.  (You may not even KNOW about problems for 10-30 minutes.  And then once you hear about the problem, you have to get to the server and make a copy of the Progressive BU file, or turn OFF the Progressive option so it doesn't overwrite the files.)  But, you don't want it too long or it wouldn't be a good safety net.

Thanks,
-- J
 

[Wim Decorte]

Progressive backups should not be your only means of backup.  It's meant to complement the regular backups and give you more options to have a valid backup set.  Not the ONLY way to have a valid backup set.

 

So if you set your progressive interval short - say 5 minutes - have a regular schedule run every 30 minutes with the option to keep a good number of sets.   That should cover you in case data gets lost.

 

There is nothing in progressive backups that would better protect you against file corruption.  The best way to avoid file corruption is to follow all the best practices.

[Justin Close]

Wim,

   Thanks for the response.  I wasn't implying that we were trying to use progressive backups (PBUs) in replacement of scheduled backups (SBUs) ones; we are looking at adding PBUs to our scheme and I am just trying to understand exactly how they function and what we can expect of them.

 

I am going to continue to assume that my perception is correct:  you have (2*interval) minutes to 'catch' a problem in order for your progressive backup to be useful.  Let's just go with an example of someone accidentally deleting something important.  In your example of 30 minute SBUs and 5 minute PBU interval, if you take 12 minutes to realize the problem and try to grab a backup, you will have to use your 30 minute backup; the PBUs will both have updated, and now the accidentally deleted information will, well, be deleted in both files. 

 

So maybe it is a better practice to divide your backup interval into thirds, to calculate a good PBU interval? E.g if your SBU is 30 minutes, then your PBU interval should be 10 minutes (30 / 3) in order to get you the most even coverage of that 30 minute interval.  Then, if you realize a problem in minutes 0-10, you can use PBU1 (and only lose a max of 10 minutes of data), in minutes 11-20 use PBU2 (and lose a max of 20 minutes), in minutes 21-30 use the 30 minute scheduled BU (and lose a max of 30 minutes).  If the interval is shorter than 1/3rd, you will lose a larger incremental portion of data if you catch the problem in that third zone, but you would gain (lose?  ) a smaller portion of data if you catch the problem in the first 2 zones.

 

I also wasn't trying to imply that I would actually USE any potential corruption benefits as a feature and safety net, it was more just a technical question and exploration. 

 

From the descriptions that I have read about PBU, it really sounds like the FM Server is executing a whole new set of transactions against a whole new file, based on what transactions were logged during that interval.  So, if you were working in the live file and it attempted to execute a transaction that got truncated, what would that mean for PBU1 (the next PBU to be updated)?  At what point does it 'log' a transaction, before or after it has executed?  If it is before it runs, then it will run it again; I would hope that it would succeed since it doesn't have to rely on a connection to a remote client.  (Does this imply, perhaps, that it WOULDN'T have logged the transaction in the first place, because the client didn't fully transfer it?)  And, for the other option, if it logs it after it finishes, but the client never finished sending it, wouldn't that mean that transaction wouldn't ever get executed against the PBU?  And thus wouldn't try to execute whatever it was that may have caused corruption in the live file?  I guess, in either scenario, it seems that the PBU would most likely NOT have incurred any corruption, assuming that it was the disconnection of the client that was causing it in the first place.  (It can't protect against fat fingered idiocy.    Well, at least not for longer than (2*interval). )

[Wim Decorte]

I am going to continue to assume that my perception is correct:  you have (2*interval) minutes to 'catch' a problem in order for your progressive backup to be useful.  

 

Actually, no.  You have less time than that.  You assumption above is based on an underlying assumption that the PBU clock starts ticking at the same time a data change happens.  It doesn't.  If I delete a record right now, the very next second may be when the PB runs and you will only have 5 minutes to go back to the oldest of the two progressive backups.  So you have to go with the worst case scenario and say that you only have the time of one interval to use a progressive backup to restore data from in case of an accidental edit or delete.

[Wim Decorte]

From the descriptions that I have read about PBU, it really sounds like the FM Server is executing a whole new set of transactions against a whole new file, based on what transactions were logged during that interval.  So, if you were working in the live file and it attempted to execute a transaction that got truncated, what would that mean for PBU1 (the next PBU to be updated)?  At what point does it 'log' a transaction, before or after it has executed?  

 

It obviously wouldn't log a transaction *before* it was executed...

There is no info on when a data change is logged in the PB logs.  But one can only assume that it happens when a record is committed.  And that may or may not correspond with what you call a transaction.  A long scripted transaction process to say update a batch of invoices is likely going to contain a number of commits.  If the network or the user's workstation conks out in the middle of that, some data changes will have been registered on the server and included in the next PB, but the transaction as a whole would have failed.

 

Makes the case for good approaches to transactional scripting and protecting against accidental data changes and deletes.  It would be nice if the PB feature provided us with an audit trail/log but that's not its purpose.  Providing that is still up to us...

[Justin Close]

I see what you mean about really only having 1 time interval.  At MOST you could have 2, but at a minimum you would only have 1.

 

It would be nice if they provided a trail of some sort.  If only.